You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
24 KiB
24 KiB
现在我已经收集了足够的信息来创建内部API网关部署文档。让我开始编写文档:
# 内部API网关部署文档 **本文档中引用的文件** - [Program.cs](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/Program.cs) - [InternalApiGatewayModule.cs](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/InternalApiGatewayModule.cs) - [InternalApiGatewayOptions.cs](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/InternalApiGatewayOptions.cs) - [DownstreamOpenApi.cs](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/Models/DownstreamOpenApi.cs) - [ocelot.json](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/ocelot.json) - [appsettings.json](file://gateways/internal/LINGYUN.MicroService.Internal.ApiGateway/src/LINGYUN.MicroService.Internal.ApiGateway/appsettings.json) - [docker-compose.yml](file://docker-compose.yml) - [start-internal-gateway.bat](file://aspnet-core/start-internal-gateway.bat) - [70.start-internal-gateway.bat](file://starter/70.start-internal-gateway.bat) - [UserinfoIdentitySession.cs](file://aspnet-core/modules/openIddict/LINGYUN.Abp.OpenIddict.AspNetCore.Session/LINGYUN/Abp/OpenIddict/AspNetCore/Session/UserinfoIdentitySession.cs) - [TenantHeaderParamter.cs](file://aspnet-core/services/LY.MicroService.IdentityServer.HttpApi.Host/TenantHeaderParamter.cs)目录
简介
本文档详细介绍了基于Ocelot的内部API网关部署配置,该网关是ABP Next Admin微服务架构的核心组件。网关提供了路由规则管理、服务发现、请求聚合、认证集成、限流熔断等关键功能,确保微服务之间的安全通信和高效协作。
内部API网关采用.NET Core技术栈,集成了ABP框架,支持多租户架构,并提供了完整的监控、日志和性能优化功能。
项目架构概览
graph TB
subgraph "客户端层"
UI[前端应用]
Mobile[移动应用]
end
subgraph "网关层"
IG[内部API网关]
OG[外部API网关]
end
subgraph "微服务层"
AS[认证服务]
BA[后台管理服务]
LS[本地化服务]
PM[平台管理服务]
MS[消息服务]
TS[任务管理服务]
WS[工作流服务]
WMS[Webhook管理服务]
WSS[微信服务]
end
subgraph "基础设施层"
DB[(数据库)]
Redis[(Redis缓存)]
ES[(Elasticsearch)]
MQ[(消息队列)]
end
UI --> IG
Mobile --> IG
IG --> AS
IG --> BA
IG --> LS
IG --> PM
IG --> MS
IG --> TS
IG --> WS
IG --> WMS
IG --> WSS
AS --> DB
BA --> DB
LS --> DB
PM --> DB
MS --> DB
TS --> DB
WS --> DB
WMS --> DB
WSS --> DB
AS --> Redis
BA --> Redis
LS --> Redis
PM --> Redis
MS --> Redis
TS --> Redis
WS --> Redis
WMS --> Redis
WSS --> Redis
AS --> ES
BA --> ES
LS --> ES
PM --> ES
MS --> ES
TS --> ES
WS --> ES
WMS --> ES
WSS --> ES
图表来源
- docker-compose.yml
核心组件分析
网关入口程序
内部API网关的入口程序位于Program.cs文件中,采用了现代化的.NET Core应用程序构建模式:
public async static Task<int> Main(string[] args)
{
try
{
Log.Information("Starting Internal ApiGateway.");
var builder = WebApplication.CreateBuilder(args);
builder.Host.AddAppSettingsSecretsJson()
.UseAutofac()
.ConfigureAppConfiguration((context, config) =>
{
var configuration = config.Build();
var agileConfigEnabled = configuration["AgileConfig:IsEnabled"];
if (agileConfigEnabled.IsNullOrEmpty() || bool.Parse(agileConfigEnabled))
{
config.AddAgileConfig(new AgileConfig.Client.ConfigClient(configuration));
}
config.AddJsonFile("ocelot.json", optional: true, reloadOnChange: true);
if (!context.HostingEnvironment.EnvironmentName.IsNullOrWhiteSpace())
{
config.AddJsonFile($"ocelot.{context.HostingEnvironment.EnvironmentName}.json", optional: true, reloadOnChange: true);
}
})
.UseSerilog((context, provider, config) =>
{
config.ReadFrom.Configuration(context.Configuration);
});
await builder.AddApplicationAsync<InternalApiGatewayModule>(options =>
{
var pluginFolder = Path.Combine(Directory.GetCurrentDirectory(), "Modules");
DirectoryHelper.CreateIfNotExists(pluginFolder);
options.PlugInSources.AddFolder(pluginFolder, SearchOption.AllDirectories);
});
var app = builder.Build();
await app.InitializeApplicationAsync();
await app.RunAsync();
return 0;
}
catch (Exception ex)
{
Log.Fatal(ex, "Host terminated unexpectedly!");
return 1;
}
finally
{
Log.CloseAndFlush();
}
}
网关模块配置
InternalApiGatewayModule是网关的核心模块,负责配置整个网关的功能:
[DependsOn(
typeof(AbpAutofacModule),
typeof(AbpSerilogEnrichersApplicationModule),
typeof(AbpSerilogEnrichersUniqueIdModule),
typeof(AbpCachingStackExchangeRedisModule),
typeof(AbpAspNetCoreSerilogModule)
)]
public partial class InternalApiGatewayModule : AbpModule
{
public override void ConfigureServices(ServiceConfigurationContext context)
{
var hostingEnvironment = context.Services.GetHostingEnvironment();
var configuration = context.Services.GetConfiguration();
ConfigureLocalization();
ConfigureJsonSerializer();
ConfigureVirtualFileSystem();
ConfigureCaching(configuration);
ConfigureApiGateway(configuration);
ConfigureKestrelServer(configuration, hostingEnvironment.IsDevelopment());
ConfigureSecurity(context.Services, configuration, hostingEnvironment.IsDevelopment());
ConfigureMvc(context.Services);
ConfigureSwagger(context.Services);
ConfigureCors(context.Services, configuration);
ConfigureOcelot(context.Services, configuration);
}
}
章节来源
- Program.cs
- InternalApiGatewayModule.cs
Ocelot配置详解
全局配置
Ocelot的全局配置包含了服务发现、限流、熔断、负载均衡等关键设置:
{
"GlobalConfiguration": {
"RequestIdKey": null,
"ServiceDiscoveryProvider": {
"Scheme": null,
"Host": null,
"Port": 0,
"Type": null,
"Token": null,
"ConfigurationKey": null,
"PollingInterval": 0,
"Namespace": null
},
"RateLimitOptions": {
"ClientIdHeader": "ClientId",
"QuotaExceededMessage": "您的操作过快,请稍后再试!",
"RateLimitCounterPrefix": "ocelot",
"DisableRateLimitHeaders": false,
"HttpStatusCode": 429
},
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 30,
"DurationOfBreak": 60000,
"TimeoutValue": 30000
},
"BaseUrl": "http://localhost:30000",
"LoadBalancerOptions": {
"Type": "RoundRobin",
"Key": null,
"Expiry": 0
},
"DownstreamScheme": "HTTP",
"HttpHandlerOptions": {
"AllowAutoRedirect": false,
"UseCookieContainer": false,
"UseTracing": true,
"UseProxy": true,
"MaxConnectionsPerServer": 2147483647
},
"DownstreamHttpVersion": null
}
}
路由配置示例
每个路由都包含详细的配置选项:
{
"DownstreamPathTemplate": "/api/abp/application-configuration",
"UpstreamPathTemplate": "/api/abp/backend-admin/application-configuration",
"UpstreamHttpMethod": ["GET"],
"DownstreamHttpMethod": null,
"AddHeadersToRequest": {},
"UpstreamHeaderTransform": {},
"DownstreamHeaderTransform": {},
"AddClaimsToRequest": {},
"RouteClaimsRequirement": {},
"AddQueriesToRequest": {},
"ChangeDownstreamPathTemplate": {},
"RequestIdKey": null,
"FileCacheOptions": {
"TtlSeconds": 0,
"Region": null
},
"RouteIsCaseSensitive": false,
"ServiceName": null,
"ServiceNamespace": null,
"DownstreamScheme": "http",
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 10,
"DurationOfBreak": 1000,
"TimeoutValue": 10000
},
"LoadBalancerOptions": {
"Type": "RoundRobin",
"Key": null,
"Expiry": 0
},
"RateLimitOptions": {
"ClientWhitelist": [],
"EnableRateLimiting": false,
"Period": null,
"PeriodTimespan": 0.0,
"Limit": 0
},
"AuthenticationOptions": {
"AuthenticationProviderKey": null,
"AllowedScopes": []
},
"HttpHandlerOptions": {
"AllowAutoRedirect": false,
"UseCookieContainer": false,
"UseTracing": true,
"UseProxy": true,
"MaxConnectionsPerServer": 2147483647
},
"DownstreamHostAndPorts": [
{
"Host": "127.0.0.1",
"Port": 30010
}
],
"UpstreamHost": null,
"Key": "backend-admin-configuration",
"DelegatingHandlers": [],
"Priority": 1,
"Timeout": 0,
"DangerousAcceptAnyServerCertificateValidator": false,
"SecurityOptions": {
"IPAllowedList": [],
"IPBlockedList": []
},
"DownstreamHttpVersion": null
}
章节来源
- ocelot.json
- ocelot.json
认证与授权集成
JWT认证集成
网关与认证服务深度集成,支持JWT令牌验证和租户隔离:
public async virtual ValueTask HandleAsync(OpenIddictServerEvents.HandleUserInfoRequestContext context)
{
var tenantId = context.Principal.FindTenantId();
using (CurrentTenant.Change(tenantId))
{
if (!await IdentitySessionChecker.ValidateSessionAsync(context.Principal))
{
context.Reject(Errors.InvalidToken, "The user session has expired.");
}
}
}
租户隔离配置
系统支持多租户架构,通过HTTP头部传递租户ID:
public class TenantHeaderParamter : IOperationFilter
{
private readonly AbpMultiTenancyOptions _multiTenancyOptions;
private readonly AbpAspNetCoreMultiTenancyOptions _aspNetCoreMultiTenancyOptions;
public void Apply(OpenApiOperation operation, OperationFilterContext context)
{
if (_multiTenancyOptions.IsEnabled)
{
operation.Parameters = operation.Parameters ?? new List<OpenApiParameter>();
operation.Parameters.Add(new OpenApiParameter
{
Name = _aspNetCoreMultiTenancyOptions.TenantKey,
In = ParameterLocation.Header,
Description = "Tenant Id in http header",
Required = false
});
}
}
}
权限检查机制
网关实现了细粒度的权限检查,确保只有授权用户才能访问特定资源:
sequenceDiagram
participant Client as 客户端
participant Gateway as 内部网关
participant Auth as 认证服务
participant Tenant as 租户服务
participant Target as 目标服务
Client->>Gateway : 请求带JWT令牌
Gateway->>Auth : 验证JWT令牌
Auth-->>Gateway : 返回用户信息和权限
Gateway->>Tenant : 获取租户信息
Tenant-->>Gateway : 返回租户上下文
Gateway->>Gateway : 应用权限检查
alt 权限验证通过
Gateway->>Target : 转发请求
Target-->>Gateway : 返回响应
Gateway-->>Client : 返回结果
else 权限验证失败
Gateway-->>Client : 返回403错误
end
图表来源
- UserinfoIdentitySession.cs
- TenantHeaderParamter.cs
章节来源
- UserinfoIdentitySession.cs
- TenantHeaderParamter.cs
限流与熔断策略
限流配置
网关实现了基于令牌桶算法的限流机制:
{
"RateLimitOptions": {
"ClientIdHeader": "ClientId",
"QuotaExceededMessage": "您的操作过快,请稍后再试!",
"RateLimitCounterPrefix": "ocelot",
"DisableRateLimitHeaders": false,
"HttpStatusCode": 429
}
}
熔断配置
熔断器保护下游服务免受故障影响:
{
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 30,
"DurationOfBreak": 60000,
"TimeoutValue": 30000
}
}
限流策略流程
flowchart TD
Request[接收请求] --> CheckRateLimit{检查限流规则}
CheckRateLimit --> |超出限制| Reject[拒绝请求<br/>返回429状态码]
CheckRateLimit --> |在限制内| CheckCircuit{检查熔断状态}
CheckCircuit --> |熔断开启| CircuitBreak[熔断器开启<br/>快速失败]
CheckCircuit --> |熔断关闭| ForwardRequest[转发请求]
ForwardRequest --> MonitorResponse{监控响应}
MonitorResponse --> |正常响应| UpdateMetrics[更新指标]
MonitorResponse --> |异常响应| IncrementFailure[增加失败计数]
IncrementFailure --> CheckBreak{检查是否达到熔断阈值}
CheckBreak --> |达到阈值| OpenCircuit[开启熔断器]
CheckBreak --> |未达到阈值| UpdateMetrics
UpdateMetrics --> ReturnResponse[返回响应]
OpenCircuit --> CircuitBreak
Reject --> End[结束]
ReturnResponse --> End
CircuitBreak --> End
负载均衡配置
负载均衡算法
网关支持多种负载均衡算法,默认使用轮询算法:
{
"LoadBalancerOptions": {
"Type": "RoundRobin",
"Key": null,
"Expiry": 0
}
}
多实例部署
graph TB
subgraph "负载均衡器"
LB[负载均衡器]
end
subgraph "内部API网关集群"
GW1[网关实例1<br/>端口:30000]
GW2[网关实例2<br/>端口:30001]
GW3[网关实例3<br/>端口:30002]
end
subgraph "后端服务"
AS1[认证服务1<br/>端口:30015]
AS2[认证服务2<br/>端口:30016]
BA1[后台管理1<br/>端口:30010]
BA2[后台管理2<br/>端口:30011]
end
LB --> GW1
LB --> GW2
LB --> GW3
GW1 --> AS1
GW1 --> BA1
GW2 --> AS2
GW2 --> BA2
GW3 --> AS1
GW3 --> BA1
图表来源
- docker-compose.yml
监控与日志配置
日志配置
网关使用Serilog进行结构化日志记录,支持多种输出目标:
{
"Serilog": {
"MinimumLevel": {
"Default": "Debug",
"Override": {
"System": "Warning",
"Microsoft": "Warning"
}
},
"Enrich": [
"FromLogContext",
"WithProcessId",
"WithThreadId",
"WithEnvironmentName",
"WithMachineName",
"WithApplicationName",
"WithUniqueId"
],
"WriteTo": [
{
"Name": "Console",
"Args": {
"initialMinimumLevel": "Information",
"standardErrorFromLevel": "Information",
"restrictedToMinimumLevel": "Information",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level:u3}] [{SourceContext}] [{ProcessId}] [{ThreadId}] - {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "File",
"Args": {
"path": "Logs/Debug-.log",
"restrictedToMinimumLevel": "Debug",
"rollingInterval": "Day",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level:u3}] [{SourceContext}] [{ProcessId}] [{ThreadId}] - {Message:lj}{NewLine}{Exception}"
}
}
]
}
}
健康检查配置
每个服务都配置了健康检查端点:
healthcheck:
test: ["CMD-SHELL", "wget --spider http://localhost/healthz || exit"]
interval: 10s
timeout: 5s
retries: 5
监控指标
graph LR
subgraph "监控指标"
RT[响应时间]
QPS[每秒查询数]
ERR[错误率]
CPU[CPU使用率]
MEM[内存使用率]
NET[网络流量]
end
subgraph "告警规则"
RTAlert[响应时间告警]
ErrorAlert[错误率告警]
ResourceAlert[资源使用告警]
end
subgraph "可视化"
Dashboard[监控面板]
Alert[告警通知]
end
RT --> RTAlert
QPS --> ErrorAlert
ERR --> ErrorAlert
CPU --> ResourceAlert
MEM --> ResourceAlert
NET --> ResourceAlert
RTAlert --> Dashboard
ErrorAlert --> Dashboard
ResourceAlert --> Dashboard
RTAlert --> Alert
ErrorAlert --> Alert
ResourceAlert --> Alert
章节来源
- appsettings.json
- docker-compose.yml
Kubernetes部署方案
Docker Compose部署
系统提供了完整的Docker Compose配置,支持一键部署:
version: '3.4'
services:
internal-apigateway:
hostname: apigateway
container_name: apigateway
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_HTTP_PORTS=80
- TZ=Asia/Shanghai
ports:
- "30000:80"
networks:
- abp-next-admin
extra_hosts:
- "host.docker.internal:host-gateway"
- "auth-server:host-gateway"
- "auth-server-api:host-gateway"
- "localization-api:host-gateway"
- "workflow-api:host-gateway"
- "webhooks-api:host-gateway"
- "wechat-api:host-gateway"
- "messages-api:host-gateway"
- "platform-api:host-gateway"
- "tasks-api:host-gateway"
- "admin-api:host-gateway"
Kubernetes部署清单
apiVersion: apps/v1
kind: Deployment
metadata:
name: internal-apigateway
spec:
replicas: 3
selector:
matchLabels:
app: internal-apigateway
template:
metadata:
labels:
app: internal-apigateway
spec:
containers:
- name: internal-apigateway
image: internal-apigateway:latest
ports:
- containerPort: 80
env:
- name: ASPNETCORE_ENVIRONMENT
value: "Production"
- name: ASPNETCORE_HTTP_PORTS
value: "80"
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /healthz
port: 80
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /healthz
port: 80
initialDelaySeconds: 5
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: internal-apigateway-service
spec:
selector:
app: internal-apigateway
ports:
- protocol: TCP
port: 80
targetPort: 80
type: LoadBalancer
Helm Chart配置
# values.yaml
replicaCount: 3
image:
repository: internal-apigateway
tag: latest
pullPolicy: IfNotPresent
service:
type: LoadBalancer
port: 80
ingress:
enabled: true
className: ""
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: api.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: api-tls
hosts:
- api.example.com
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 70
targetMemoryUtilizationPercentage: 80
章节来源
- docker-compose.yml
高可用性部署
集群部署架构
graph TB
subgraph "负载均衡层"
ELB[外部负载均衡器<br/>Nginx/AWS ALB]
ILB[内部负载均衡器<br/>Kubernetes Service]
end
subgraph "网关集群"
GW1[网关实例1<br/>可用区A]
GW2[网关实例2<br/>可用区B]
GW3[网关实例3<br/>可用区C]
end
subgraph "服务发现"
SD[服务发现<br/>Consul/Eureka]
end
subgraph "数据存储"
REDIS[Redis集群<br/>哨兵模式]
MYSQL[MySQL集群<br/>主从复制]
end
subgraph "监控告警"
PROM[Prometheus]
GRAFANA[Grafana]
ALERT[Alertmanager]
end
ELB --> ILB
ILB --> GW1
ILB --> GW2
ILB --> GW3
GW1 --> SD
GW2 --> SD
GW3 --> SD
GW1 --> REDIS
GW2 --> REDIS
GW3 --> REDIS
GW1 --> MYSQL
GW2 --> MYSQL
GW3 --> MYSQL
GW1 --> PROM
GW2 --> PROM
GW3 --> PROM
PROM --> GRAFANA
PROM --> ALERT
故障转移机制
sequenceDiagram
participant Client as 客户端
participant LB as 负载均衡器
participant GW1 as 网关实例1
participant GW2 as 网关实例2
participant GW3 as 网关实例3
participant Monitor as 监控系统
Client->>LB : 发送请求
LB->>GW1 : 转发请求
GW1-->>LB : 实例故障
LB->>Monitor : 报告实例故障
Monitor->>LB : 触发故障转移
LB->>GW2 : 转发到备用实例
GW2-->>LB : 正常响应
LB-->>Client : 返回结果
Note over Monitor,GW3 : 监控系统持续检测实例状态
Monitor->>GW1 : 检查实例健康状态
GW1-->>Monitor : 实例不可用
Monitor->>Monitor : 更新实例状态
数据备份与恢复
apiVersion: batch/v1
kind: CronJob
metadata:
name: gateway-backup
spec:
schedule: "0 2 * * *" # 每天凌晨2点执行
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: postgres:13
command:
- /bin/bash
- -c
- |
pg_dump -h postgres-primary -U postgres -d gateway_db > /backup/gateway-$(date +%Y%m%d).sql
aws s3 cp /backup/gateway-$(date +%Y%m%d).sql s3://backup-bucket/gateway/
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: password
volumeMounts:
- name: backup-storage
mountPath: /backup
volumes:
- name: backup-storage
persistentVolumeClaim:
claimName: backup-pvc
restartPolicy: OnFailure
故障排除指南
常见问题诊断
1. 网关启动失败
症状: 网关无法启动或启动后立即退出
排查步骤:
# 查看启动日志
docker logs apigateway
# 检查配置文件
cat ocelot.json | jq .
# 验证端口占用
netstat -tulpn | grep 30000
# 检查依赖服务
curl http://auth-server:44385/healthz
2. 路由配置错误
症状: 请求被拒绝或返回404错误
排查步骤:
# 检查路由配置
curl http://localhost:30000/api/ApiGateway/Basic/Routes
# 验证上游路径
curl -v http://localhost:30000/api/abp/backend-admin/application-configuration
# 检查下游服务状态
curl http://127.0.0.1:30010/healthz
3. 认证失败
症状: JWT令牌验证失败
排查步骤:
# 验证JWT令牌
jwt.io
# 检查认证服务
curl http://auth-server:44385/.well-known/openid-configuration
# 验证租户配置
curl -H