"Description": "Discover how to implement AI management in your ABP Framework application, enhancing workspace dynamics with easy installation options."
"Description": "Manage persisted AI workspaces, providers, RAG data sources, MCP tools and remote AI clients with the ABP AI Management module."
}
}
```
```
@ -13,7 +13,7 @@ This module implements AI (Artificial Intelligence) management capabilities on t
## How to Install
## How to Install
The **AI Management Module** is not included in [the startup templates](../solution-templates/layered-web-application) by default. However, when creating a new application with [ABP Studio](../../tools/abp-studio/index.md), you can easily enable it during setup via the *AI Integration* step in the project creation wizard. Alternatively, you can install it using the ABP CLI or ABP Studio:
The **AI Management Module** is not included in [the startup templates](../../solution-templates/layered-web-application/index.md) by default. However, when creating a new application with [ABP Studio](../../studio/overview.md), you can easily enable it during setup via the *AI Integration* step in the project creation wizard. Alternatively, you can install it using the ABP CLI or ABP Studio:
**Using ABP CLI:**
**Using ABP CLI:**
@ -97,11 +97,13 @@ AI Management module packages are designed for various usage scenarios. Packages
## User Interface
## User Interface
This module provides UI integration for all three officially supported UI frameworks by ABP:
This module provides administration and client UI for the following UI stacks:
* **MVC / Razor Pages** UI
* **MVC / Razor Pages** UI
* **Angular** UI
* **Angular** UI
* **Blazor** UI (Server & WebAssembly)
* **Blazorise** UI (Server & WebAssembly)
* **MudBlazor** UI (Server & WebAssembly)
* **React Admin Console** management UI
### Menu Items
### Menu Items
@ -129,12 +131,12 @@ You can create a new workspace or edit an existing workspace in this page. The w
* **API Key**: Authentication key (if required by provider)
* **API Key**: Authentication key (if required by provider)
* **API Base URL**: Custom endpoint URL (optional)
* **API Base URL**: Custom endpoint URL (optional)
* **Application Name**: Associate with specific application
* **Required Permission**: Permission needed to use this workspace
* **Embedder Provider / Model**: Embedding generator used for RAG
* **Embedder Provider / Model**: Embedding generator used for RAG
* **Vector Store Provider / Settings**: Storage backend and connection settings for document vectors
* **Vector Store Provider / Settings**: Storage backend and connection settings for document vectors
The **Application Name** is assigned by the application that creates or synchronizes the workspace. Workspace access is managed from the resource permission action; it is not a field in the create/edit form.
#### Chat Interface
#### Chat Interface
The AI Management module includes a built-in chat interface for testing workspaces. You can:
The AI Management module includes a built-in chat interface for testing workspaces. You can:
@ -177,6 +179,21 @@ When a workspace has MCP servers associated, the AI model can invoke tools from
Use `McpClientFactoryOptions` when an MCP server needs more time to initialize or when a stdio connection test starts a slow process:
```csharp
Configure<McpClientFactoryOptions>(options =>
{
options.DefaultTimeoutMs = 180_000;
options.StdioInitializationTimeoutMs = 240_000;
options.StdioConnectionTestTimeoutSeconds = 300;
});
```
`DefaultTimeoutMs` defaults to 120 seconds and applies to HTTP transports and, unless overridden, stdio initialization. `StdioConnectionTestTimeoutSeconds` defaults to 180 seconds and is constrained to 30-600 seconds by the connection-test service. HTTP connection tests use a fixed one-minute timeout.
#### Workspace Data Sources
#### Workspace Data Sources
Workspace Data Sources page is used to upload and manage RAG documents per workspace. Uploaded files are processed and indexed in the background.
Workspace Data Sources page is used to upload and manage RAG documents per workspace. Uploaded files are processed and indexed in the background.
@ -207,12 +224,12 @@ When creating or managing a workspace, you can configure the following propertie
| `ApiKey` | No | API authentication key (required by some providers) |
| `ApiKey` | No | API authentication key (required by some providers) |
| `ApiBaseUrl` | No | Custom endpoint URL (defaults to provider's default) |
| `ApiBaseUrl` | No | Custom endpoint URL (defaults to provider's default) |
| `SystemPrompt` | No | Default system prompt for all conversations |
| `SystemPrompt` | No | Default system prompt for all conversations |
| `Temperature` | No | Response randomness (0.0-1.0, defaults to provider default) |
| `Temperature` | No | Provider-specific response randomness |
| `Description` | No | Workspace description |
| `Description` | No | Workspace description |
| `IsActive` | No | Enable/disable the workspace (default: true) |
| `IsActive` | No | Enable/disable the workspace (default: true) |
| `ApplicationName` | No | Associate workspace with specific application |
| `ApplicationName` | No | Application identity recorded during creation or synchronization |
| `RequiredPermissionName` | No | Permission required to use this workspace |
| `RequiredPermissionName` | No | Optional policy checked by client consumption services |
| `IsSystem` | No | Whether it's a system workspace (read-only) |
| `IsSystem` | No | Whether the workspace was synchronized from code |
| `OverrideSystemConfiguration` | No | Allow database configuration to override code-defined settings |
| `OverrideSystemConfiguration` | No | Allow database configuration to override code-defined settings |
| `EmbedderProvider` | No | Embedding provider name (e.g., "OpenAI", "Ollama") |
| `EmbedderProvider` | No | Embedding provider name (e.g., "OpenAI", "Ollama") |
| `EmbedderModelName` | No | Embedding model identifier (e.g., "text-embedding-3-small") |
| `EmbedderModelName` | No | Embedding model identifier (e.g., "text-embedding-3-small") |
@ -231,27 +248,38 @@ The AI Management module supports two types of workspaces:
* **Defined in code** using `PreConfigure<AbpAIWorkspaceOptions>`
* **Defined in code** using `PreConfigure<AbpAIWorkspaceOptions>`
* **Cannot be deleted** through the UI
* **Cannot be deleted** through the UI
* **Read-only by default**, but can be overridden when `OverrideSystemConfiguration` is enabled
* **Use the code-defined chat client by default**; persisted provider settings take effect only when `OverrideSystemConfiguration` is enabled
* **Useful for** application-critical AI features that must always be available
* **Useful for** application-critical AI features that must always be available
* **Created automatically** when the application starts
* **Synchronized automatically** when the application starts
Example:
Example:
```csharp
```csharp
PreConfigure<AbpAIWorkspaceOptions>(options =>
public override void PreConfigureServices(ServiceConfigurationContext context)
chatClientConfiguration.Builder = new ChatClientBuilder(
sp => new OpenAIClient(apiKey).GetChatClient("gpt-4")
);
});
});
});
}
```
Configure the code-defined keyed client with the [Framework AI provider integration](../../framework/infrastructure/artificial-intelligence/index.md) used by your application.
At startup, AI Management records the registered chat, embedding and vector-store providers and synchronizes the code-defined workspace names for the current application. When at least one code-defined workspace remains, names removed from that application's configuration are also removed from its synchronized set. An empty workspace collection skips the update, so removing the last code-defined workspace doesn't delete its persisted record automatically. Synchronization errors are logged without stopping application startup.
You can disable startup synchronization or override the recorded application name:
```csharp
Configure<WorkspaceUpdaterOptions>(options =>
{
options.AutoUpdateAtStartup = false;
options.ApplicationName = "MyAIGateway";
});
});
```
```
Disabling synchronization means code-defined workspaces are not added to or updated in the management database automatically. Their code-defined keyed clients can still be resolved by the Framework AI infrastructure.
#### Dynamic Workspaces
#### Dynamic Workspaces
* **Created through the UI** or programmatically via `ApplicationWorkspaceManager` and `IWorkspaceRepository`
* **Created through the UI** or programmatically via `ApplicationWorkspaceManager` and `IWorkspaceRepository`
@ -264,12 +292,16 @@ Example (data seeding):
```csharp
```csharp
public class WorkspaceDataSeederContributor : IDataSeedContributor, ITransientDependency
public class WorkspaceDataSeederContributor : IDataSeedContributor, ITransientDependency
> Provider API keys, embedding keys, vector-store connection settings, MCP headers and MCP credentials are sensitive persisted configuration. Do not hard-code them in source control. Restrict workspace and MCP administration permissions, protect the application database and configuration backups, and supply secrets from a secure configuration provider. Duplicating a workspace also duplicates its provider and RAG configuration, including credentials.
### Resolution Precedence
When an application requests a workspace chat client, AI Management resolves it in this order:
1. If no persisted configuration exists, the workspace is inactive, or it is a system workspace with `OverrideSystemConfiguration = false`, resolution first tries the code-defined keyed `IChatClient` and then the default `IChatClient`.
2. An active dynamic workspace, or an overridden system workspace, uses the factory registered for its persisted `Provider` value.
If neither fallback exists, an inactive workspace produces an inactive-workspace error; the other fallback cases produce a provider-not-found error. A configured provider without a registered factory also produces a provider-not-found error.
### Workspace Naming Rules
### Workspace Naming Rules
* Workspace names **must be unique**
* Workspace names **must be unique**
* Workspace names **cannot contain spaces** (use underscores or camelCase)
* Workspace names **cannot contain spaces** (use underscores or camelCase)
* Workspace names are **case-sensitive**
Workspace names are global identifiers in an AI Management database. Workspaces, MCP server configurations, data sources and the data-source blob container are not tenant-scoped entities. In a multi-tenant application, use workspace resource permissions and application-level design to control access; do not treat these records or blobs as tenant-isolated storage.
## RAG with File Upload
## RAG with File Upload
@ -367,7 +413,12 @@ RAG is enabled per workspace when both embedding and vector store settings are c
* `MongoDb`: Standard MongoDB connection string including database name.
* `MongoDb`: Standard MongoDB connection string including database name.
* `Pgvector`: Standard PostgreSQL/Npgsql connection string.
* `Pgvector`: Standard PostgreSQL/Npgsql connection string.
* `Qdrant`: Qdrant endpoint string (`http://host:port`, `https://host:port`, or `host:port`).
* `Qdrant`: Qdrant endpoint string (`http://host:port`, `https://host:port`, or `host:port`). The current provider discards the URL scheme and creates a non-TLS client from only the host and port. Do not rely on an `https://` value to enable TLS.
> [!IMPORTANT]
> The `MongoDb` vector-store provider requires MongoDB Atlas or an Atlas CLI local deployment because it uses `$vectorSearch`. Standard MongoDB Docker images do not provide this feature. Create an Atlas Vector Search index named `vector_index` on the `AIVectorEmbeddings` collection, with `Embedding` as the vector field, cosine similarity and dimensions matching the configured embedding model. The provider creates the collection and its regular workspace index, but it does not create the Atlas vector index.
The Pgvector provider creates the target database when its credentials permit it, enables the `vector` extension and creates workspace tables lazily. Qdrant creates a workspace-specific collection when the first vector is stored. The service account therefore needs the corresponding database, extension, table or collection privileges.
#### Document Processing Pipeline
#### Document Processing Pipeline
@ -376,9 +427,11 @@ When a file is uploaded as a workspace data source:
1. File is stored in blob storage.
1. File is stored in blob storage.
2. `IndexDocumentJob` is queued.
2. `IndexDocumentJob` is queued.
3. `DocumentProcessingManager` extracts text using content-type-specific extractors.
3. `DocumentProcessingManager` extracts text using content-type-specific extractors.
4. Text is chunked (default chunk size: `1000`, overlap: `200`).
4. Text is chunked with `1000` characters as a target size. The normal paragraph path carries up to `200` characters into the next chunk; oversized paragraphs are split by line with a `50`-character carry. An indivisible line can produce a chunk larger than the target.
5. Embeddings are generated in batches and stored through the configured vector store.
5. Embeddings are generated in ordered batches and stored through the configured vector store.
6. Data source is marked as processed (`IsProcessed = true`).
6. The data source is marked as processed (`IsProcessed = true`) after the final batch succeeds.
Each indexing run has an identifier and a per-data-source distributed lock. Stale and duplicate batches are ignored, while an out-of-order batch is retried by the background job system. A failed run can therefore leave partial vector data until a retry succeeds or the data source is re-indexed again.
#### Workspace Data Source HTTP API
#### Workspace Data Source HTTP API
@ -395,20 +448,45 @@ The module exposes workspace data source endpoints under `/api/ai-management/wor
#### Chat Integration Behavior
#### Chat Integration Behavior
When a workspace has embedder configuration, AI Management wraps the chat client with a document search tool function named `search_workspace_documents`.
When an active dynamic workspace, or a system workspace with `OverrideSystemConfiguration = true`, is resolved through its persisted provider factory and has embedder configuration, AI Management wraps the factory-created client with a document search tool function named `search_documents`. The keyed/default fallback branches described in [Resolution Precedence](#resolution-precedence) return their code-defined client directly and don't add persisted RAG or MCP tools.
* The tool delegates to `IDocumentSearchService` (`DocumentSearchService` by default).
* The tool delegates to `IDocumentSearchService` (`DocumentSearchService` by default).
* The search currently uses `TopK = 5` chunks.
* The search currently uses `TopK = 5` chunks.
* If RAG retrieval fails, chat continues without injected context.
* If RAG retrieval fails, chat continues without injected context.
The underlying chat client must be a `FunctionInvokingChatClient`. The built-in OpenAI and Ollama factories add function invocation automatically. A custom chat factory must build its client with `ChatClientBuilder.UseFunctionInvocation()` before RAG or MCP tools can execute.
#### Automatic Reindexing on Configuration Changes
#### Automatic Reindexing on Configuration Changes
When workspace embedder or vector store configuration changes, AI Management automatically:
When workspace embedder or vector store configuration changes, AI Management automatically:
* Initializes the new vector store configuration (if needed).
* Initializes the new vector store configuration (if needed).
* Deletes existing embeddings when embedder provider/model changes.
* Attempts to delete existing embeddings when the embedder provider or model changes.
* Re-queues all workspace data sources for re-indexing.
* Re-queues all workspace data sources for re-indexing.
Embedding cleanup and automatic re-index queueing are best-effort follow-up operations. Their failures are logged without rolling back the workspace update. Use the data-source page's **Re-index All** action after correcting the configuration if automatic re-indexing could not be queued.
### Configuring Indexing Options
`AIManagementIndexingOptions` controls the work performed by background indexing jobs:
```csharp
Configure<AIManagementIndexingOptions>(options =>
{
options.EmbeddingBatchSize = 100;
options.MaxConcurrentIndexingJobs = 2;
options.DistributedLockTimeoutSeconds = 15;
});
```
| Property | Default | Description |
| -------- | ------- | ----------- |
| `EmbeddingBatchSize` | `50` | Maximum chunks embedded and stored by one batch job |
| `MaxConcurrentIndexingJobs` | `1` | Maximum indexing batches running concurrently across the application |
| `DistributedLockTimeoutSeconds` | `0` | Time to wait for an indexing lock; `0` performs an immediate attempt |
Increase concurrency only after checking the embedding provider's rate limits and vector-store capacity. The concurrency limiter is application-wide; the per-data-source lock still prevents two batches from mutating the same data source concurrently.
### Configuring Data Source Upload Options
### Configuring Data Source Upload Options
The `WorkspaceDataSourceOptions` class allows you to customize the file upload constraints for workspace data sources. You can configure the allowed file extensions, maximum file size, and content type mappings.
The `WorkspaceDataSourceOptions` class allows you to customize the file upload constraints for workspace data sources. You can configure the allowed file extensions, maximum file size, and content type mappings.
@ -418,15 +496,13 @@ public override void ConfigureServices(ServiceConfigurationContext context)
{
{
Configure<WorkspaceDataSourceOptions>(options =>
Configure<WorkspaceDataSourceOptions>(options =>
{
{
// Add support for additional file types
// Allow log files to use the built-in plain-text extractor.
@ -439,6 +515,8 @@ public override void ConfigureServices(ServiceConfigurationContext context)
| `MaxFileSize` | `long` | `10485760` (10 MB) | Maximum file size in bytes |
| `MaxFileSize` | `long` | `10485760` (10 MB) | Maximum file size in bytes |
| `ContentTypeMap` | `Dictionary<string, string>` | `.txt`, `.md`, `.pdf` with their MIME types | Maps file extensions to MIME content types |
| `ContentTypeMap` | `Dictionary<string, string>` | `.txt`, `.md`, `.pdf` with their MIME types | Maps file extensions to MIME content types |
Adding an extension and MIME mapping only allows the upload. The mapped content type must also be supported by an `IDocumentTextExtractor`. Implement and register an extractor when adding a format that cannot use the built-in plain-text, Markdown or PDF extractors.
The options class also provides helper methods:
The options class also provides helper methods:
| Method | Description |
| Method | Description |
@ -447,9 +525,6 @@ The options class also provides helper methods:
| `GetAcceptAttribute()` | Returns a string for the HTML `accept` attribute (e.g., ".txt,.md,.pdf") |
| `GetAcceptAttribute()` | Returns a string for the HTML `accept` attribute (e.g., ".txt,.md,.pdf") |
> [!NOTE]
> Adding new file extensions also requires a matching content extractor to be registered for document processing. The built-in extractors support `.txt`, `.md`, and `.pdf` files.
#### Hosting-Level Upload Limits
#### Hosting-Level Upload Limits
`WorkspaceDataSourceOptions.MaxFileSize` controls the module-level validation, but your hosting stack may reject large uploads before the request reaches AI Management. If you increase `MaxFileSize`, make sure the underlying server and proxy limits are also updated.
`WorkspaceDataSourceOptions.MaxFileSize` controls the module-level validation, but your hosting stack may reject large uploads before the request reaches AI Management. If you increase `MaxFileSize`, make sure the underlying server and proxy limits are also updated.
@ -523,6 +598,7 @@ The AI Management module defines the following permissions:
@ -539,9 +615,17 @@ The module also defines workspace data source permissions for RAG document opera
| `AIManagement.WorkspaceDataSources.Download` | Download original uploaded file |
| `AIManagement.WorkspaceDataSources.Download` | Download original uploaded file |
| `AIManagement.WorkspaceDataSources.ReIndex` | Re-index one or all workspace files |
| `AIManagement.WorkspaceDataSources.ReIndex` | Re-index one or all workspace files |
### Workspace-Level Permissions
### Workspace Consumption Authorization
The `IChatCompletionClientAppService` client API, OpenAI-compatible API and MCP tool catalog authorize a workspace when the current user satisfies any one of these conditions:
1. The user has `AIManagement.Workspaces.Playground`.
2. The workspace has a `RequiredPermissionName` and the user has that policy.
3. The user has the `Volo.AIManagement.Workspaces.Workspace.Consume` resource permission for that workspace ID.
In addition to module-level permissions, you can restrict access to individual workspaces by setting the `RequiredPermissionName` property:
Use the **Manage Permissions** action on the workspace page to grant the per-workspace resource permission. `AIManagement.Workspaces.ManagePermissions` controls access to that action.
`RequiredPermissionName` is a programmatic alternative for workspaces tied to an application permission:
```csharp
```csharp
var workspace = await _applicationWorkspaceManager.CreateAsync(
var workspace = await _applicationWorkspaceManager.CreateAsync(
@ -549,14 +633,11 @@ var workspace = await _applicationWorkspaceManager.CreateAsync(
Administrative workspace and data-source endpoints still use the module permissions in the tables above. A workspace consumption grant does not grant permission to edit the workspace, manage its credentials or upload RAG documents.
* Only authorized users with that permission can access the workspace endpoints
* Users without the permission will receive an authorization error
## Usage Scenarios
## Usage Scenarios
@ -569,57 +650,7 @@ The AI Management module is designed to support various usage patterns, from sim
**Use this when:** You want to use AI in your application without any dependency on the AI Management module.
**Use this when:** You want to use AI in your application without any dependency on the AI Management module.
In this scenario, you only use the ABP Framework's AI features directly. You configure AI providers (like OpenAI) in your code and don't need any database or management UI.
In this scenario, use the ABP Framework AI infrastructure and configure provider clients in code. No AI Management database, administration UI or commercial client packages are involved. See [Artificial Intelligence](../../framework/infrastructure/artificial-intelligence/index.md) for the current static workspace registration and consumption APIs.
**Required Packages:**
- `Volo.Abp.AI`
- Any Microsoft AI extensions (e.g., `Microsoft.Extensions.AI.OpenAI`)
**Configuration:**
```csharp
public class YourModule : AbpModule
{
public override void ConfigureServices(ServiceConfigurationContext context)
public MyService(IChatClient<TWorkspace> chatClient)
{
_chatClient = chatClient;
}
public async Task<string> GetResponseAsync(string prompt)
{
var response = await _chatClient.CompleteAsync(prompt);
return response.Message.Text;
}
}
```
> See [Artificial Intelligence](../../framework/infrastructure/artificial-intelligence/index.md) documentation for more details about workspace configuration.
### Scenario 2: AI Management with Domain Layer Dependency (Local Execution)
### Scenario 2: AI Management with Domain Layer Dependency (Local Execution)
@ -646,49 +677,12 @@ In this scenario, you install the AI Management module with its database layer,
> Note: `Volo.AIManagement.EntityFrameworkCore` transitively includes `Volo.AIManagement.Domain` and `Volo.Abp.AI.AIManagement` packages.
> Note: `Volo.AIManagement.EntityFrameworkCore` transitively includes `Volo.AIManagement.Domain` and `Volo.Abp.AI.AIManagement` packages.
**Workspace Definition Options:**
Define dynamic workspaces through:
**Option 1 - System Workspace (Code-based):**
```csharp
public class YourModule : AbpModule
{
public override void ConfigureServices(ServiceConfigurationContext context)
No code configuration needed. Define workspaces through:
- The AI Management UI (navigate to AI Management > Workspaces)
- The AI Management UI (navigate to AI Management > Workspaces)
- Data seeding in your `DataSeeder` class
- Data seeding in your `DataSeeder` class
**Using Chat Client:**
For a system workspace, define the typed workspace and its code client as described in [System Workspaces](#system-workspaces). Enable `OverrideSystemConfiguration` on that workspace only when persisted provider settings should replace the code client.
@ -832,15 +807,14 @@ Each AI Management **workspace** appears as a selectable model in the client app
| `/v1/models` | GET | List available models (workspaces) |
| `/v1/models` | GET | List available models (workspaces) |
| `/v1/models/{modelId}` | GET | Retrieve a single model (workspace) |
| `/v1/models/{modelId}` | GET | Retrieve a single model (workspace) |
| `/v1/embeddings` | POST | Generate embeddings |
| `/v1/embeddings` | POST | Generate embeddings |
| `/v1/files` | GET | List uploaded files |
| `/v1/files` | POST | Upload a file |
| `/v1/files/{fileId}` | GET | Get file info |
| `/v1/files/{fileId}` | DELETE | Delete a file |
| `/v1/files/{fileId}/content` | GET | Download file content |
All endpoints require authentication via a **Bearer token** in the `Authorization` header.
All endpoints require an authenticated user. Non-browser API clients normally send an access token as a **Bearer token** in the `Authorization` header.
#### Usage
The value of `model` is an AI Management workspace name, not the provider's model name. `GET /v1/models` returns only workspaces the current user is authorized to consume. The same workspace authorization is applied before chat, completion, model and embedding operations.
The module also exposes file-management routes backed by workspace data sources. They require workspace-specific parameters and are not a drop-in implementation of the OpenAI Files API. Use the [Workspace Data Source HTTP API](#workspace-data-source-http-api) for portable upload, list, download, delete and re-index operations.
### Usage
The general pattern for connecting any OpenAI-compatible client:
The general pattern for connecting any OpenAI-compatible client:
@ -877,13 +851,13 @@ curl -X POST https://localhost:44336/v1/chat/completions \
}'
}'
```
```
> The OpenAI-compatible endpoints are available from both the `Volo.AIManagement.Client.HttpApi` and `Volo.AIManagement.HttpApi` packages, depending on your deployment scenario.
> The OpenAI-compatible `/v1` endpoints are provided by `Volo.AIManagement.Client.HttpApi`. `Volo.AIManagement.HttpApi` instead exposes the management and integration APIs.
## Client Usage
## Client Usage
AI Management uses different packages depending on the usage scenario:
AI Management uses different packages depending on the usage scenario:
- **`Volo.AIManagement.*` packages**: These contain the core AI functionality and are used when your application hosts and manages its own AI operations. These packages don't expose any application service and endpoints to be consumed by default.
- **`Volo.AIManagement.*` packages**: These contain the core AI functionality for applications that host and manage AI operations. The application and HTTP API packages expose management and integration services.
- **`Volo.AIManagement.Client.*` packages**: These are designed for applications that need to consume AI services from a remote application. They provide both server and client side of remote access to the AI services.
- **`Volo.AIManagement.Client.*` packages**: These are designed for applications that need to consume AI services from a remote application. They provide both server and client side of remote access to the AI services.
@ -923,6 +897,9 @@ You can customize the chat widget with the following properties:
- `Title`: The title of the chat widget.
- `Title`: The title of the chat widget.
- `ShowStreamCheckbox`: Whether to show the stream checkbox. Allows user to toggle streaming on and off. Default is `false`.
- `ShowStreamCheckbox`: Whether to show the stream checkbox. Allows user to toggle streaming on and off. Default is `false`.
- `UseStreaming`: Default streaming behavior. Can be overridden by user when `ShowStreamCheckbox` is true.
- `UseStreaming`: Default streaming behavior. Can be overridden by user when `ShowStreamCheckbox` is true.
- `DisableWhenNoConversation`: Disables input until a `ConversationId` is selected. Default is `false`.
- `ShowUsageDetails`: Shows token usage and tool calls for assistant messages. Default is `true`.
- `ShowDetailedSourceInformation`: Shows chunk-level RAG source details instead of grouping sources by file. Default is `false`.
```csharp
```csharp
@await Component.InvokeAsync(typeof(ChatClientChatViewComponent), new ChatClientChatViewModel
@await Component.InvokeAsync(typeof(ChatClientChatViewComponent), new ChatClientChatViewModel
The AI Management module should be imported and lazy-loaded in your routing array. It has a static `createRoutes` method for configuration. It is available for import from `@volo/abp.ng.ai-management`.
The AI Management module should be imported and lazy-loaded in your routing array. It has a `createRoutes` function for configuration and is available from `@volo/abp.ng.ai-management`.
`createRoutes` optionally accepts `AIManagementConfigOptions`. It provides entity-action, toolbar-action and entity-property contributors for workspaces and MCP servers, plus create/edit form contributors for workspaces. The Workspaces, Chat Playground, MCP Servers and Workspace Data Sources routes also use keys from `eAIManagementComponents`, so they can be replaced through ABP's replaceable component system.
#### Services / Models
#### Services / Models
AI Management module services and models are generated via `generate-proxy` command of the [ABP CLI](../../cli). If you need the module's proxies, you can run the following command in the Angular project directory:
AI Management module services and models are generated via `generate-proxy` command of the [ABP CLI](../../cli). If you need the module's proxies, you can run the following command in the Angular project directory:
The AI Management module remote URL configurations shown above are optional.
Use `AIManagement` for management and integration proxies. Non-streaming requests from the Angular chat widget use the generated `ChatCompletionClientService` proxy with `AIManagementClient`. When streaming is enabled, both the stream-start POST request and the subsequent EventSource connection use `default.url` instead of `AIManagementClient`.
> If you don't set the `AIManagement` property, the `default.url` will be used as fallback.
Both named entries are optional and generated proxy requests fall back independently to `default.url`. `default.url` is also the global fallback for other ABP and application requests that don't specify a configured named API, so changing it redirects those requests too.
For split hosting, configure `AIManagement` and `AIManagementClient` for their respective hosts. Before pointing `default.url` to the client API and enabling streaming, make sure every other request uses an appropriate named API or that the client host provides or forwards all endpoints that still rely on the global fallback.
#### The Chat Widget
#### The Chat Widget
@ -1103,12 +1087,19 @@ The `@volo/abp.ng.ai-management` package provides a `ChatInterfaceComponent` (`a
<abp-chat-interface
<abp-chat-interface
[workspaceName]="'mylama'"
[workspaceName]="'mylama'"
[conversationId]="'my-conversation-id'"
[conversationId]="'my-conversation-id'"
[showUsageDetails]="true"
[showStreamCheckbox]="true"
/>
/>
```
```
- `workspaceName` (required): The name of the workspace to use.
- `workspaceName` (required): The name of the workspace to use.
- `conversationId`: The unique identifier for persisting and retrieving chat history from client-side storage. When provided, the chat history is stored in the browser and restored when the user revisits the page. If `null`, the chat is ephemeral and will be lost when the component is destroyed.
- `conversationId`: The unique identifier for persisting and retrieving chat history from client-side storage. When provided, the history is restored for the current user and workspace.
- `providerName`: The name of the AI provider. Used for displaying contextual error messages.
- `providerName`: The name of the AI provider. Used for displaying contextual error messages.
- `allowEphemeral`: Allows sending without a `conversationId`. The default is `false`; set it to `true` for an in-memory conversation that is discarded with the component.
- `showUsageDetails`: Shows usage and tool-call details. Default is `true`.
- `showStreamCheckbox`: Shows the streaming toggle. Default is `true`.
The component emits `messageSent` and `messageReceived` events for application-level conversation orchestration.
### Blazor UI
### Blazor UI
@ -1117,26 +1108,22 @@ The `@volo/abp.ng.ai-management` package provides a `ChatInterfaceComponent` (`a
The AI Management module remote endpoint URLs can be configured in your `appsettings.json`:
The AI Management module remote endpoint URLs can be configured in your `appsettings.json`:
```json
```json
"RemoteServices": {
{
"Default": {
"RemoteServices": {
"BaseUrl": "Default url here"
"Default": {
},
"BaseUrl": "Default url here"
"AIManagement": {
},
"BaseUrl": "AI Management remote url here"
"AIManagement": {
"BaseUrl": "AI Management remote url here"
},
"AIManagementClient": {
"BaseUrl": "AI Management client remote url here"
}
}
}
}
}
```
```
For **Blazor WebAssembly**, you can also configure the remote endpoint URL via `AIManagementClientBlazorWebAssemblyOptions`:
Use the `AIManagement` remote service for management and integration clients. Use `AIManagementClient` for the remote chat and OpenAI-compatible client APIs. Blazor WebAssembly uses the standard ABP remote-service configuration shown above; it doesn't require a module-specific options class. If a named `BaseUrl` isn't set, ABP uses `Default.BaseUrl` as the fallback.
> If you don't set the `BaseUrl` for AIManagement, the `Default.BaseUrl` will be used as fallback.
#### The Chat Widget
#### The Chat Widget
@ -1152,10 +1139,15 @@ The `Volo.AIManagement.Client.Blazor` package provides a `ChatClientChat` Blazor
```
```
- `WorkspaceName` (required): The name of the workspace to use.
- `WorkspaceName` (required): The name of the workspace to use.
- `ConversationId`: The unique identifier for persisting and retrieving chat history from client-side storage. When provided, the chat history is stored in the browser's local storage and restored when the user revisits the page. If not provided or `null`, the chat is ephemeral and will be lost when the component is disposed.
- `ConversationId` (required to send messages): The unique identifier for persisting and retrieving chat history from client-side storage. The component disables message input while this value is null or empty.
- `Title`: The title displayed in the chat widget header.
- `Title`: The title displayed in the chat widget header.
- `ShowStreamCheckbox`: Whether to show a checkbox that allows the user to toggle streaming on and off. Default is `false`.
- `ShowStreamCheckbox`: Whether to show a checkbox that allows the user to toggle streaming on and off. Default is `false`.
- `OnFirstMessage`: An `EventCallback<FirstMessageEventArgs>` that is triggered when the first message is sent in a conversation. It can be used to determine the chat title after the first prompt like applied in the chat playground. The event args contain `ConversationId` and `Message` properties.
- `UseStreaming`: Initial streaming mode. Default is `true`.
- `ShowUsageDetails`: Shows token usage and tool calls. Default is `true` in the Blazorise component.
- `ShowDetailedSourceInformation`: Shows chunk-level RAG source information. Default is `true` in the Blazorise component.
- `OnFirstMessage`: An `EventCallback<FirstMessageEventArgs>` that is triggered when the first message is sent in a conversation. It can be used to determine the chat title after the first prompt like applied in the chat playground. The event args contain `ConversationId` and `Message` properties.
The MudBlazor package exposes the same core `WorkspaceName`, `ConversationId`, `Title`, `ShowStreamCheckbox` and `OnFirstMessage` parameters. The detailed usage/source display parameters above are specific to the Blazorise component.
```xml
```xml
<ChatClientChatWorkspaceName="mylama"
<ChatClientChatWorkspaceName="mylama"
@ -1165,10 +1157,9 @@ The `Volo.AIManagement.Client.Blazor` package provides a `ChatClientChat` Blazor
OnFirstMessage="@HandleFirstMessage" />
OnFirstMessage="@HandleFirstMessage" />
```
```
## Using Dynamic Workspace Configurations for custom requirements
## Reading Dynamic Workspace Configuration
The AI Management module allows you to access only configuration of a workspace without resolving pre-constructed chat client. This is useful when you want to use a workspace for your own purposes and you don't need to use the chat client.
Use `IWorkspaceConfigurationStore` when trusted server-side code needs the persisted workspace configuration without resolving an `IChatClient`. The store has local and remote implementations for the corresponding deployment scenarios.
The `IWorkspaceConfigurationStore` service is used to access the configuration of a workspace. It has multiple implementations according to the usage scenario.
// Get the configuration of the workspace that can be managed dynamically.
var configuration = await _workspaceConfigurationStore
var configuration = await _workspaceConfigurationStore.GetAsync("MyWorkspace");
.GetOrNullAsync("MyWorkspace");
// Do something with the configuration
return configuration?.ModelName;
var kernel = Kernel.CreateBuilder()
.AddAzureOpenAIChatClient(
config.ModelName!,
new Uri(config.ApiBaseUrl),
config.ApiKey
)
.Build();
}
}
}
}
```
```
The returned configuration can include provider credentials. Keep it inside trusted server-side services and do not serialize it into an application response. Use `IChatClient<TWorkspace>` or `IChatCompletionClientAppService` when you only need to execute a chat request.
## Implementing Custom AI Provider Factories
## Implementing Custom AI Provider Factories
While the AI Management module provides built-in support for OpenAI through the `Volo.AIManagement.OpenAI`package, you can easily add support for other AI providers by implementing a custom `IChatClientFactory`.
AI Management provides built-in OpenAI and Ollama factories through the `Volo.AIManagement.OpenAI`and `Volo.AIManagement.Ollama` packages. You can add another provider by implementing a custom `IChatClientFactory`.
### Understanding the Factory Pattern
### Understanding the Factory Pattern
The AI Management module uses a factory pattern to create `IChatClient` instances based on the provider configuration stored in the database. Each provider (OpenAI, Ollama, Azure OpenAI, etc.) needs its own factory implementation.
The AI Management module uses a factory pattern to create `IChatClient` instances based on the provider configuration stored in the database. Each provider name needs one registered factory implementation.
### Creating a Custom Factory
### Creating a Custom Factory
Here's how to implement a factory for Ollama as an example:
The following example registers a separate `CustomOllama` provider to demonstrate the factory contract. Use the built-in `Volo.AIManagement.Ollama` package when you only need the standard Ollama integration.
#### Step 1: Install the Provider's NuGet Package
#### Step 1: Install the Provider's NuGet Package
@ -1228,20 +1214,18 @@ using Volo.Abp.DependencyInjection;
namespace YourNamespace;
namespace YourNamespace;
public class OllamaChatClientFactory : IChatClientFactory, ITransientDependency
public class CustomOllamaChatClientFactory : IChatClientFactory, ITransientDependency
{
{
public string Provider => "Ollama";
public string Provider => "CustomOllama";
public Task<IChatClient> CreateAsync(ChatClientCreationConfiguration configuration)
public Task<IChatClient> CreateAsync(ChatClientCreationConfiguration configuration)
{
{
// Create the Ollama client with configuration from database
var builder = new ChatClientBuilder(sp => new OllamaApiClient(
> For production scenarios, you may want to add validation for the factory configuration.
> Validate every required configuration value in the factory. Build a `FunctionInvokingChatClient` with `UseFunctionInvocation()` when the workspace can use RAG or MCP tools.
### Available Configuration Properties
### Available Configuration Properties
@ -1270,6 +1254,7 @@ The `ChatClientCreationConfiguration` object provides the following properties f
The cache is invalidated when workspaces are created, updated or deleted. A rename invalidates both the old and current workspace names, and invalidation participates in the current unit of work.
### HttpApi Client Layer
### HttpApi Client Layer
- `IntegrationWorkspaceConfigurationStore`: Integration service for remote workspace configuration retrieval. Implements `IWorkspaceConfigurationStore` interface.
- `IntegrationWorkspaceConfigurationStore`: Integration service for remote workspace configuration retrieval. Implements `IWorkspaceConfigurationStore` interface.
The cache is automatically invalidated when workspaces are created, updated, or deleted.
## See Also
## See Also
- [Artificial Intelligence Infrastructure](../../framework/infrastructure/artificial-intelligence/index.md): Learn about the underlying AI workspace infrastructure
- [Artificial Intelligence Infrastructure](../../framework/infrastructure/artificial-intelligence/index.md): Learn about the underlying AI workspace infrastructure
maliming
2 months ago