Open Source Web Application Framework for ASP.NET Core
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
github-actions[bot] cb40d5cf8e
Optimised images with calibre/image-actions
2 weeks ago
..
abp10 Optimised images with calibre/image-actions 2 weeks ago
abp11 Optimised images with calibre/image-actions 2 weeks ago
k6 test: trim AbpNet10vsNet11PerfTest 2 weeks ago
report Optimised images with calibre/image-actions 2 weeks ago
scripts test: rename benchmark targets from net10/net11 to abp10/abp11 2 weeks ago
.gitignore test: add AbpNet10vsNet11PerfTest benchmark suite 2 weeks ago
Directory.Packages.props test: add AbpNet10vsNet11PerfTest benchmark suite 2 weeks ago
README.md test: align benchmark README and report with the measured results 2 weeks ago
docker-compose.yml test: add AbpNet10vsNet11PerfTest benchmark suite 2 weeks ago

README.md

ABP .NET 10 vs .NET 11 Performance Benchmark

End-to-end load test comparing a template-based ABP MVC application on stable ABP 10.7.1 + .NET 10 (abp10) against preview ABP 11.0.0-rc.1 + .NET 11 (abp11) under identical hardware, database, data set, and workload conditions:

Target Application Build Runtime Role
abp10 ABP 10.7.1 (net10.0) .NET 10.0.13 Primary — Stable baseline
abp11 ABP 11.0.0-rc.1 (net11.0) .NET 11.0.0-rc.3 Primary — Preview stack
abp10-rt11 ABP 10.7.1 (net10.0) .NET 11.0.0-rc.3 (DOTNET_ROLL_FORWARD=LatestMajor) Supplementary — Isolates the .NET runtime difference from ABP package changes

Three-target benchmark design

Key Findings Summary

Metric / Test abp10 (ABP 10 + .NET 10) abp10-rt11 (ABP 10 on .NET 11) abp11 (ABP 11 + .NET 11) .NET 11 Advantage
6 it/s p95 Latency (27-run main-validation batch) 38.3 ms 34.5 ms 34.7 ms 9.4% faster
6 it/s Web CPU Usage 55% 50% 50% 5 pp lower
6 it/s GC Heap Size 355 MB 321 MB 311 MB ~44 MB less heap
7 it/s Saturation Knee (p95) 39.5 ms 40.3 ms 40.2 ms Stable at capacity edge (0 errors)
ABP 11 Package Overhead — Baseline +0.2 ms (0.5%) No measurable regression

Comparison of response time, CPU, and memory usage

Full benchmark methodology, tool evaluation, scenario correlation analysis, and findings are documented in report/README.md. Exact package, runtime, and hardware versions are recorded in report/versions.md.


Repository Structure

.
├── docker-compose.yml               # PostgreSQL 18.6 (digest-pinned) on port 5433
├── abp10/                           # ABP 10.7.1 + .NET 10 MVC project (EF Core, PostgreSQL, LeptonX Lite)
├── abp11/                           # ABP 11.0.0-rc.1 + .NET 11 MVC project (upgraded copy of abp10)
├── k6/
│   ├── tools/
│   │   └── summarize.mjs            # Aggregates k6, dotnet-counters, and docker stats into Markdown tables
│   └── scenario.js                  # Parameterized multi-VU load-test scenario (smoke, probe, fixed modes)
├── scripts/
│   ├── env.ps1                      # Shared Production measurement environment variables
│   ├── publish.ps1                  # Release-publishes Web and DbMigrator for abp10 and abp11
│   ├── db.ps1                       # Seeds or resets the database from Bench_seed_<target> template DBs
│   ├── seed-bench-data.sql          # Seeds 1,000 test accounts (bench_vu_1..1000) and global email settings
│   ├── start-app.ps1                # Starts a target with optional CPU affinity mask and background logging
│   ├── run-bench.ps1                # Automated benchmark runner (DB reset, affinity, counters, docker stats, k6)
│   └── versions.ps1                 # Generates report/versions.md from published artifacts and system info
└── report/
    ├── README.md                    # Benchmark report (methodology, tool evaluation, results, limitations)
    ├── versions.md                  # Generated environment, SDK, runtime, and package version table
    ├── probe.md                     # Capacity probe results (2 to 32 it/s)
    ├── results-main-validation.md   # Full 27-run marathon re-validation (abp10 vs abp10-rt11 vs abp11)
    ├── results-runtime.md           # Runtime isolation at 4 it/s
    ├── results-runtime6.md          # Runtime isolation at 6 it/s
    ├── results-verify-knee.md       # Saturation knee validation at 7 it/s (9 runs)
    └── results-verify-marathon6.md  # Targeted 6 it/s marathon validation (9 runs)

Prerequisites

  • OS: Windows 11 (CPU affinity and power mode inspection in scripts use Windows commands)
  • .NET SDKs & Runtimes:
    • .NET 10 SDK (10.0.402) and ASP.NET Core Runtime (10.0.13)
    • .NET 11 Preview/RC SDK and ASP.NET Core Runtime (11.0.100-rc.3 / 11.0.0-rc.3)
  • Diagnostics: dotnet-counters (dotnet tool install --global dotnet-counters)
  • Database: Docker Desktop (runs postgres:18.6 via docker compose)
  • Load Generator: Grafana k6 (v2.3.0+)
  • Node.js: v20+ (for k6/tools/summarize.mjs)

Setup & Preparation

Run all commands from the repository root in PowerShell.

1. Start PostgreSQL

docker compose up -d

2. Publish Both Projects (Release)

Publishes Bench.Web and Bench.DbMigrator for both abp10 and abp11 into publish/<target>/:

./scripts/publish.ps1

3. Seed Template Databases

Runs Bench.DbMigrator, applies scripts/seed-bench-data.sql (bench_vu_1..1000 + global email setting rows), and snapshots the result into Bench_seed_abp10 and Bench_seed_abp11. Subsequent runs use -Action reset to recreate Bench from the template in seconds:

./scripts/db.ps1 -Target abp10 -Action seed
./scripts/db.ps1 -Target abp11 -Action seed

4. Generate the Versions Report

./scripts/versions.ps1

Running the Scenario Manually (Smoke / Validation)

Set the seeded admin/test user password in the environment:

$env:BENCH_PASSWORD = '<seeded_admin_password>'

Reset the database and start the desired target (abp10, abp11, or abp10-rt11):

./scripts/db.ps1 -Target abp10 -Action reset
./scripts/start-app.ps1 -Target abp10

In a second terminal, run the load-test scenario with 1 VU (1 iteration) or 20 concurrent VUs:

# Single-user verification (1 iteration = 92 HTTP requests, 107 checks)
k6 run -e BENCH_PASSWORD=$env:BENCH_PASSWORD --vus 1 --iterations 1 k6/scenario.js

# Multi-user concurrency verification (20 VUs, 60 iterations, no think time)
k6 run -e BENCH_PASSWORD=$env:BENCH_PASSWORD -e THINK_TIME=0 -e SESSION_ITERATIONS=2 --vus 20 --iterations 60 k6/scenario.js

Running the Full Benchmark Suite

Before running measurements:

  • Plug the machine into AC power and set Windows Power Mode to Best performance.
  • Close browsers, IDEs, k6 Studio, and background apps; leave only Docker Desktop and the PowerShell terminal open.
  • Ensure $env:BENCH_PASSWORD is set.

Step 1: Capacity Probe (~32 minutes)

Steps through arrival rates (2,4,6,8,10,12,14,16,20,24,28,32 iterations/s, 60 s per step) on abp10 and abp11 to find the saturation point:

./scripts/run-bench.ps1 -Mode probe -Repeats 1 -Batch probe
node k6/tools/summarize.mjs results/probe report/probe.md

Inspect report/probe.md and pick three load levels (a, b, c iterations/s) corresponding to ~25%, ~50%, and ~75% of the lower saturation threshold before dropped iterations or error spikes occur.

Step 2: Runtime Isolation — abp10 vs abp10-rt11 vs abp11 (~1 hour 15 minutes)

Runs an arrival rate across all three targets × 3 repetitions to separate runtime gains/regressions from ABP package changes:

# Medium load (4 it/s):
./scripts/run-bench.ps1 -Mode fixed -Rates 4 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch runtime
node k6/tools/summarize.mjs results/runtime report/results-runtime.md

# High load (6 it/s):
./scripts/run-bench.ps1 -Mode fixed -Rates 6 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch runtime6
node k6/tools/summarize.mjs results/runtime6 report/results-runtime6.md

Step 3: Verification Batches — Knee & Marathon (~1 hour 15 minutes each)

Targeted validation runs addressing saturation threshold and sustained-load endurance:

# Saturation Knee (7 it/s):
./scripts/run-bench.ps1 -Mode fixed -Rates 7 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch verify-knee
node k6/tools/summarize.mjs results/verify-knee report/results-verify-knee.md

# Sustained Marathon (6 it/s):
./scripts/run-bench.ps1 -Mode fixed -Rates 6 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch verify-marathon6
node k6/tools/summarize.mjs results/verify-marathon6 report/results-verify-marathon6.md

# Full 3-Target Re-Validation Marathon (Rates 2, 4, 6 it/s, 27 runs, ~3.5 hours):
./scripts/run-bench.ps1 -Mode fixed -Rates 2,4,6 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch main-validation
node k6/tools/summarize.mjs results/main-validation report/results-main-validation.md