35 KiB
ABP .NET 10 vs .NET 11 Load-Test Benchmark Report
1. Objective & Scope
The goal of this benchmark is to compare the end-to-end performance of a template-based ABP Framework MVC application running on:
abp10(Baseline): Latest stable ABP (10.7.1) on .NET 10 (10.0.13).abp11(Preview): Latest preview ABP (11.0.0-rc.1) updated for .NET 11 on .NET 11 (11.0.0-rc.3).
As a supplementary check for the interpretation note (because comparing abp10 and abp11 changes both the ABP version and the .NET runtime simultaneously), an additional abp10-rt11 target runs the exact abp10 published build rolled forward onto the .NET 11 runtime (DOTNET_ROLL_FORWARD=LatestMajor, DOTNET_ROLL_FORWARD_TO_PRERELEASE=1) at the medium load level to isolate pure runtime effects from ABP/EF Core package changes.
Both projects were created from the same open-source ABP MVC template (app, MVC / Razor Pages, Entity Framework Core, PostgreSQL, LeptonX Lite theme), use identical database schemas and seeded data, and execute the same recorded multi-page user scenario under controlled load.
2. Versions & Environment
Full generated details are available in report/versions.md.
2.1 Application & Package Matrix
| Component | abp10 |
abp11 |
|---|---|---|
SDK (global.json) |
10.0.402 |
11.0.100-rc.3.26522.148 |
| Target Framework | net10.0 |
net11.0 |
| Server GC | True |
True |
Runtime: Microsoft.NETCore.App |
10.0.13 |
11.0.0-rc.3.26522.148 |
Runtime: Microsoft.AspNetCore.App |
10.0.13 |
11.0.0-rc.3.26522.148 |
NuGet: Volo.Abp.Core |
10.7.1 |
11.0.0-rc.1 |
NuGet: Volo.Abp.AspNetCore.Mvc.UI.Theme.LeptonXLite |
5.7.1 |
6.0.0-rc.1 |
NuGet: Volo.Abp.Studio.Client.AspNetCore |
3.1.6 |
3.1.6 |
NuGet: Microsoft.EntityFrameworkCore |
10.0.11 |
11.0.0-rc.3.26522.148 |
NuGet: Npgsql.EntityFrameworkCore.PostgreSQL |
10.0.3 |
11.0.0-rc.1.1 |
NuGet: Npgsql |
10.0.3 |
10.0.3 |
NuGet: OpenIddict.Core |
7.7.0 |
7.7.0 |
npm: @abp/aspnetcore.mvc.ui.theme.leptonxlite |
5.7.1 |
6.0.0-rc.1 |
npm: @abp/aspnetcore.mvc.ui.theme.shared |
10.7.1 |
11.0.0-rc.1 |
2.2 Project Construction & Parity Verification
abp10: Generated viaabp new Bench -t app -u mvc -d ef -dbms PostgreSQL -th leptonx-lite -uost --no-tests, upgraded to ABP10.7.1(abp update --version 10.7.1) and LeptonX Lite5.7.1.abp11: Created as a direct copy ofabp10(avoiding template generator drift), upgraded to ABP11.0.0-rc.1, target frameworknet11.0,Microsoft.*11.0.0-rc.3.26522.148, and LeptonX Lite6.0.0-rc.1.- Schema & Seed Parity: Both databases contain exactly 41 tables and identical initial seed rows (21 permission grants, 1 admin role, 1 admin user, plus 1,000 seeded load-test accounts and 2 global email setting rows).
2.3 Hardware & Software Environment
| Property | Value |
|---|---|
| CPU | Intel Core i7-14650HX — 16 physical cores (8 Performance cores / 16 threads 0..15 + 8 Efficiency cores 16..23, 24 logical processors) |
| RAM | 31.7 GB |
| OS | Microsoft Windows 11 Pro (10.0.26300) |
| Power Configuration | Windows Power Plan: Balanced; AC Power Mode overlay: Best performance |
| Database | PostgreSQL 18.6 (Debian 18.6-1.pgdg13+2) in Docker 29.8.1 (postgres@sha256:3a4264520e121176619f54ff867d12774bc81e95e7b84db58d63bccdb6144b81) |
| Load Generator | Grafana k6 v2.3.0 + Grafana k6 Studio 2.1.2 |
| Server Telemetry | dotnet-counters 10.0.750501 + docker stats |
3. Tool Evaluation & Selection
Five candidate tools specified in the task requirements were evaluated against the benchmark criteria: browser scenario recording, stateful replay (cookies, anti-forgery tokens, dynamic IDs, optimistic concurrency stamps), open-workload modeling, resource overhead on a single host, and licensing.
| Tool | License | Browser Recording & Correlation | Multi-Page Stateful Replay | Load Generator Overhead | Suitability for This Task |
|---|---|---|---|---|---|
| Grafana k6 + k6 Studio | Free / OSS (AGPLv3 CLI, free Studio) | Yes — k6 Studio proxy records HAR and extracts correlation rules into JS | High — full JS control over cookies, dual CSRF tokens, multipart forms, and dynamic IDs | Very Low — single native Go binary, event-loop VUs, no JVM GC pauses | Selected |
| Apache JMeter | Free / OSS (Apache 2.0) | Yes — HTTP(S) Test Script Recorder captures .jmx tree |
Medium — requires dozens of GUI Regex/JSON extractors and controllers across 92 requests; .jmx XML is hard to diff/maintain |
High — JVM thread-per-VU model; JVM GC pauses and CPU/RAM usage contend with the SUT on a single host | Viable alternative, but inferior isolation and maintainability |
| Microsoft Crank | Free / OSS (MIT) | No — no browser recorder or HAR correlation pipeline | Low — orchestrates single-endpoint drivers (bombardier, wrk) rather than stateful 92-request user flows unless paired with an external script |
Low | Not suitable as a standalone scenario recorder/player |
| BenchmarkDotNet | Free / OSS (MIT) | No — in-process C# microbenchmark runner | No — measures isolated methods, not full HTTP/MVC/EF Core/PostgreSQL page flows | N/A | Supplementary micro-benchmarking only |
| NBomber | Commercial license required for organization use | No built-in proxy recorder | High (C# scripting) | Low | Excluded due to licensing restriction |
Why Grafana k6 (+ k6 Studio) Was Chosen Over Apache JMeter
- Single-Machine Resource Isolation: Both the .NET web application, PostgreSQL, and the load generator run on the same laptop. JMeter runs on the JVM with one OS thread per virtual user; under high concurrency its memory footprint and JVM garbage collection pauses introduce measurement noise. k6 is a compiled Go binary with a tiny footprint that runs cleanly inside the pinned Efficiency cores (
0xFF0000) without starving the .NET process. - True Open Workload Model (
constant-arrival-rate): Fairly comparing.NET 10and.NET 11requires starting the exact same number of user scenario iterations per second against each target regardless of response time. k6 providesconstant-arrival-ratenatively. JMeter's default Thread Group is a closed model (where a slower server throttles the request rate unless extra plugins/timers are configured). - Browser-Like Parallel Requests (
http.batch): Every ABP MVC page load fetches three dynamic script endpoints (/Abp/ApplicationConfigurationScript,/Abp/ApplicationLocalizationScript,/Abp/ServiceProxyScript) in parallel. k6 supports parallel batching natively viahttp.batch(), whereas standard JMeter samplers execute sequentially without third-party plugins. - Maintainable Correlation Logic: Maintaining dual anti-forgery tokens,
ConcurrencyStampextraction, multipart form boundaries, and per-VU parameterization across 92 requests takes ~360 lines of clean JavaScript (k6/scenario.js) versus thousands of lines of GUI-configured.jmxXML in JMeter. - Clean Warm-up Separation: k6 scenarios (
warmupfollowed bymeasure) combined with threshold-filtered--summary-exportautomatically exclude warm-up requests from overall and per-page-group percentiles (p95,p99) without post-processing raw CSV logs.
4. Scenario Recording, Correlation & Replay Evaluation
4.1 User Journey Covered
One complete scenario iteration visits all pages of the template-based ABP MVC application and exercises every available function (9 groups; about 92 HTTP requests per iteration as counted during scenario validation, per-group counts were not measured):
| Group | Pages & Operations Exercised |
|---|---|
01_login |
GET /Account/Login (+3 /Abp/*Script), switch language to en, reload login page, POST /Account/Login, redirect to GET / (+3 /Abp/*Script). Executed on each VU's first iteration and every SESSION_ITERATIONS (10) thereafter. |
02_home |
Visit home page GET / (+3 /Abp/*Script). |
03_users |
GET /Identity/Users (+3 /Abp/*Script), list users, filter by bench, GET/POST /Identity/Users/CreateModal, filter by created user name, GET/POST /Identity/Users/EditModal (with ConcurrencyStamp), GET/POST /AbpPermissionManagement/PermissionManagementModal (providerName=U), DELETE /api/identity/users/{id}, verify deletion (totalCount === 0). |
04_roles |
GET /Identity/Roles (+3 /Abp/*Script), list roles, GET/POST /Identity/Roles/CreateModal, lookup created role ID, GET/POST /Identity/Roles/EditModal (with ConcurrencyStamp), GET/POST /AbpPermissionManagement/PermissionManagementModal (providerName=R), DELETE /api/identity/roles/{id}, verify deletion. |
05_tenants |
GET /TenantManagement/Tenants (+3 /Abp/*Script), list tenants, GET/POST /TenantManagement/Tenants/CreateModal, lookup tenant ID, GET/POST /TenantManagement/Tenants/EditModal, GET/POST /FeatureManagement/FeatureManagementModal (providerName=T), DELETE /api/multi-tenancy/tenants/{id}, verify deletion. |
06_settings |
GET /SettingManagement (+3 /Abp/*Script), render Emailing tab (handler=RenderView&id=Volo.Abp.EmailSetting), POST /api/setting-management/emailing, POST /SettingManagement?handler=RefreshConfiguration, render Feature Management tab, open Host Feature modal (GET /FeatureManagement/FeatureManagementModal?providerName=T). |
07_profile |
GET /Account/Manage (+3 /Abp/*Script), extract ConcurrencyStamp, PUT /api/account/my-profile, GET /api/account/my-profile. |
08_language |
Switch UI language to Turkish (GET /Abp/Languages/Switch?culture=tr), load /Account/Manage (+3 /Abp/*Script), switch back to English (culture=en), load /Account/Manage (+3 /Abp/*Script). |
09_logout |
GET /Account/Logout (302), load GET /Account/Login (+3 /Abp/*Script). Executed every SESSION_ITERATIONS (10) iterations. |
4.2 Recording with k6 Studio
- Chrome bypasses local proxies for
localhost, so the recording was performed viahttps://bench.localtest.me:44377(public DNS resolving to127.0.0.1) with proxy SSL verification disabled for the ASP.NET Core developer certificate. - The raw recording contained passwords and session cookies and was not committed; the final scenario is
k6/scenario.js.
4.3 Replay Evaluation: Round 1 (Default Export, No Rules)
Exporting the recording from k6 Studio with default settings failed immediately:
- Stale Anti-Forgery Token on Login (
400 Bad Request):POST /Account/Loginsent the recorded__RequestVerificationTokenwithout matching the fresh.AspNetCore.Antiforgery.*cookie fromGET /Account/Login. - Unauthenticated Cascade: Because login failed, no
.AspNetCore.Identity.Applicationcookie was issued; every subsequent page returned302 Foundto/Account/Login(which k6 followed, returning200 OKon the login HTML and masking failures unless URL/content checks are present), while AJAX endpoints returned401/400. - Static Filter Trap: k6 Studio's static-asset filter excludes responses with
content-type: application/javascript. That stripped/Abp/ApplicationConfigurationScript, which is the endpoint that sets theXSRF-TOKENcookie required by all ABP AJAX requests.
4.4 Replay Evaluation: Round 2 (Correlated k6 Studio Generator)
Seven rules were configured in a k6 Studio generator (not committed), producing a correlated script:
| # | Rule Type | Source / Filter | Selector | Target | Purpose |
|---|---|---|---|---|---|
| 1 | Verification | All requests | status == recordedValue |
Response status | Fail on unexpected status codes (200, 204, 302) |
| 2 | Correlation | HTML responses | name="__RequestVerificationToken" type="hidden" value="..." |
Form/body fields | Replace form anti-forgery tokens with most recent token |
| 3 | Correlation | HTML responses | name="__RequestVerificationToken" type="hidden" value="..." |
Header RequestVerificationToken |
Supply valid anti-forgery token header on AJAX requests |
| 4 | Correlation | /api/identity/users?filter=rec_user |
JSON items[0].id |
Subsequent URLs & forms | Correlate dynamically generated User GUID |
| 5 | Correlation | /api/identity/roles |
Regex "name":"rec_role"[^}]*?"id":"([^"]+)" |
Subsequent URLs & forms | Correlate dynamically generated Role GUID |
| 6 | Correlation | /api/multi-tenancy/tenants |
JSON items[0].id |
Subsequent URLs & forms | Correlate dynamically generated Tenant GUID |
| 7 | Correlation | Modal / page HTML | Regex name="(?:\w+\.)?ConcurrencyStamp" value="([^"]+)" |
Subsequent POST/PUT | Prevent 409 Conflict on User/Role/Tenant/Profile updates |
Round 2 Results:
- 1 VU × 1 iteration:
59 / 59checks passed (0%failed). - 1 VU × 3 sequential iterations:
177 / 177checks passed (0%failed) because each iteration deletes therec_user,rec_role, andrec_tenantit creates. - 5 concurrent VUs: 8.8% request failure rate. Concurrent VUs collided on hardcoded entity names (
rec_user,rec_role,rec_tenant→Volo.Abp.BusinessExceptionreturned as HTTP 403) and on the sharedadminuser'sConcurrencyStamp(AbpDbConcurrencyException→ HTTP 409).
4.5 Production Load-Test Scenario (k6/scenario.js)
To support high-concurrency load testing without artificial data collisions, k6/scenario.js refines the correlated Studio script with:
- Per-VU Isolated Accounts: 1,000 admin-equivalent users (
bench_vu_1..bench_vu_1000) are seeded inscripts/seed-bench-data.sql. VUnlogs in exclusively asbench_vu_n, eliminatingConcurrencyStampcontention on/api/account/my-profile. - Unique Entity Names: Created entities are named
u_<run>_<vu>_<iter>,r_<run>_<vu>_<iter>, andt_<run>_<vu>_<iter>(run = Date.now().toString(36)), guaranteeing zero name collisions across VUs and iterations. - Session Reuse Across Iterations:
noCookiesReset: truepreserves the VU cookie jar across iterations. Each VU logs in on its first iteration and logs out/in everySESSION_ITERATIONS = 10iterations so ASP.NET Core Identity's PBKDF2 password hashing does not dominate CPU usage. - Full ABP Dynamic Script Load: Every
page()call fetches the HTML and executeshttp.batch()for/Abp/ApplicationConfigurationScript,/Abp/ApplicationLocalizationScript, and/Abp/ServiceProxyScript. - Authentic ABP Token Mechanics:
ajax()reads theXSRF-TOKENcookie from the VU's cookie jar for theRequestVerificationTokenheader, while multipart modal posts (CreateModal,EditModal,PermissionManagementModal,FeatureManagementModal) embed the latest hidden__RequestVerificationTokenextracted from HTML. - Validation: Verified at 1 VU (1 iteration = 107/107 checks) and 20 concurrent VUs × 60 iterations (6,220/6,220 checks, 0 HTTP errors, 0 application warnings) on
abp10,abp10-rt11, andabp11.
5. Measurement Methodology
- Identical Production Configuration (
scripts/env.ps1):ASPNETCORE_ENVIRONMENT=ProductionandDOTNET_ENVIRONMENT=Production.Serilog__MinimumLevel__Default=WarningandSerilog__MinimumLevel__Override__Microsoft=Warningso synchronous file/console logging atInformationlevel does not bottleneck throughput.
- CPU Core Pinning (
scripts/run-bench.ps1,scripts/start-app.ps1):- On the Intel i7-14650HX (logical CPUs
0..15= 8 P-cores with HT;16..23= 8 E-cores):- Web Application (
Bench.Web.dll): Pinned at process creation (start /affinity FF) to logical CPUs0..7(4 physical P-cores / 8 threads). Setting affinity at process startup ensures .NET Server GC sizes its heaps todotnet.process.cpu.count = 8. - Load Generator (
k6): Pinned (start /affinity FF0000) to logical CPUs16..23(8 E-cores). - OS & Docker/WSL2 (
bench-postgres): Logical CPUs8..15(remaining 4 P-cores / 8 threads) remain uncontested by the web app and k6.
- Web Application (
- On the Intel i7-14650HX (logical CPUs
- Clean Database State Per Run (
scripts/db.ps1):- Before every single run, the running web app is stopped, and
Benchis dropped and recreated fromBench_seed_<target>(CREATE DATABASE "Bench" TEMPLATE "Bench_seed_<target>").
- Before every single run, the running web app is stopped, and
- Alternating Target Order:
- Odd repetitions run in order
abp10 → abp11(orabp10 → abp10-rt11 → abp11); even repetitions reverse the order (abp11 → abp10) to cancel thermal or background drift.
- Odd repetitions run in order
- Warm-up & Measurement Window:
- Every fixed-rate run executes a 120 s warm-up (
warmupscenario), a 10 s grace gap (GRACE_S = 10, allowing in-flight requests to finish while keeping VUs logged in), and a 300 s measured window (measurescenario), repeated 3 times per target and rate. - Server metrics (
dotnet-countersat 1 s intervals anddocker statsat ~2 s intervals) are sliced strictly to[measureStartUtc, measureEndUtc)byk6/tools/summarize.mjs.
- Every fixed-rate run executes a 120 s warm-up (
6. Results & Bottleneck Analysis
6.1 Capacity Probe (abp10 vs abp11)
Command:
./scripts/run-bench.ps1 -Mode probe -Repeats 1 -Batch probe
node k6/tools/summarize.mjs results/probe report/probe.md
The capacity probe stepped through 12 arrival rates (2, 4, 6, 8, 10, 12, 14, 16, 20, 24, 28, 32 iterations/s, 60 s per step after a 60 s warm-up). Full output in report/probe.md:
| target | rate it/s | req/s | p95 ms | p99 ms | failed | dropped it | app CPU % (avg/max) | working set MB | GC pause ms/s | TP queue max | DB CPU % |
|---|---|---|---|---|---|---|---|---|---|---|---|
| abp10 | 2 | 161.6 | 25.2 | 79.5 | 0.00% | 0 | 14.2 / 18.2 | 574 | 18.2 | 2 | 14.9 |
| abp10 | 4 | 324.5 | 30.2 | 86.4 | 0.00% | 0 | 32.7 / 40.8 | 630 | 26.4 | 93 | 34.2 |
| abp10 | 6 | 484.8 | 44.7 | 129.6 | 0.00% | 0 | 65.0 / 90.8 | 703 | 37.8 | 2 | 61.1 |
| abp10 | 8 | 625.3 | 195.8 | 364.8 | 0.00% | 21 | 97.4 / 102.3 | 866 | 48.4 | 37 | 88.7 |
| abp10 | 10 | 582.8 | 641.3 | 1075.5 | 0.00% | 128 | 97.7 / 104.3 | 988 | 72.8 | 340 | 85.6 |
| abp10 | 12 | 519.3 | 1298.4 | 2174.9 | 0.00% | 201 | 97.3 / 108.8 | 1208 | 103.3 | 597 | 84.7 |
| abp10 | 14 | 460.5 | 1843.4 | 2565.4 | 0.00% | 274 | 97.4 / 122.9 | 1433 | 112.1 | 941 | 78.8 |
| abp10 | 16 | 384.9 | 3657.1 | 5967.5 | 0.00% | 368 | 97.4 / 116.2 | 1634 | 89.1 | 967 | 74.4 |
| abp10 | 20 | 316.2 | 5745.5 | 7931.9 | 0.00% | 480 | 97.1 / 107.2 | 1953 | 92.5 | 1105 | 60.9 |
| abp10 | 24 | 248.8 | 8682.5 | 12970.0 | 0.00% | 577 | 97.9 / 134.2 | 2162 | 93.8 | 1040 | 49.1 |
| abp10 | 28 | 216.8 | 15130.9 | 17844.5 | 0.00% | 681 | 97.6 / 126.4 | 2560 | 83.2 | 1075 | 40.4 |
| abp10 | 32 | 195.9 | 17141.5 | 23436.1 | 0.00% | 921 | 97.8 / 121.1 | 2735 | 75.4 | 1077 | 37.6 |
| abp11 | 2 | 161.6 | 31.2 | 96.1 | 0.00% | 0 | 16.7 / 24.4 | 581 | 22.3 | 1 | 16.3 |
| abp11 | 4 | 324.5 | 34.5 | 98.9 | 0.00% | 0 | 36.2 / 48.2 | 674 | 22.0 | 1 | 36.6 |
| abp11 | 6 | 486.1 | 53.7 | 149.7 | 0.00% | 0 | 72.0 / 95.1 | 705 | 40.7 | 4 | 66.8 |
| abp11 | 8 | 588.9 | 392.9 | 1056.8 | 0.00% | 53 | 97.5 / 102.5 | 906 | 65.4 | 62 | 91.4 |
| abp11 | 10 | 575.5 | 581.4 | 7347.7 | 0.00% | 150 | 97.3 / 109.4 | 1056 | 141.9 | 74 | 87.2 |
| abp11 | 12 | 511.6 | 1516.9 | 9978.2 | 0.00% | 205 | 95.1 / 118.2 | 1225 | 134.2 | 714 | 93.0 |
| abp11 | 14 | 475.9 | 2936.0 | 4962.4 | 0.00% | 269 | 95.5 / 109.6 | 1615 | 108.4 | 488 | 96.0 |
| abp11 | 16 | 413.1 | 4390.1 | 8079.5 | 0.00% | 353 | 90.7 / 99.8 | 1866 | 99.2 | 50 | 87.5 |
| abp11 | 20 | 176.6 | 9764.5 | 17464.3 | 1.10% | 482 | 54.1 / 99.0 | 2042 | 62.5 | 1303 | 37.0 |
| abp11 | 24 | 311.9 | 8049.4 | 12906.0 | 0.00% | 583 | 95.4 / 125.0 | 2309 | 121.3 | 482 | 76.8 |
| abp11 | 28 | 296.4 | 11598.2 | 14170.4 | 0.00% | 681 | 95.0 / 102.5 | 2664 | 99.0 | 1036 | 76.1 |
| abp11 | 32 | 255.8 | 15488.4 | 19624.8 | 0.00% | 921 | 96.5 / 103.7 | 3031 | 85.2 | 1126 | 67.2 |
Probe Findings & Selected Load Levels:
- Both targets sustain
2,4, and6it/s (162,325, and485 req/s) with 0 dropped iterations and 0.00% HTTP errors. - Both targets reach full CPU saturation (
97.4%–97.5%average CPU across the 8 pinned P-core threads) at8 it/s, whereabp10peaks at625.3 req/s(p95 = 195.8 ms,21dropped iterations) andabp11peaks at588.9 req/s(p95 = 392.9 ms,53dropped iterations). - Using
8 it/sas the saturation ceiling, the three measured load levels (~25%, ~50%, ~75% of capacity) are2,4, and6iterations/s (~162,~323, and~485 req/s).
6.2 Excluded Initial Batch
The first fixed-rate batch (abp10 vs abp11 at 2, 4, 6 it/s) was measured while the machine was still in use for other work. Its absolute numbers were inflated (the abp10 web process needed ~79% CPU at 6 it/s, versus ~50–55% in every later batch for the same request rate) and abp11 showed an unexplained 2x p95 at 6 it/s. Because the measurement conditions were not controlled, the batch was discarded and is not reported. All results below were measured on a dedicated, otherwise idle machine; the 27-run main-validation batch (6.3.5) repeats the full 2, 4, 6 it/s matrix under those conditions.
6.3 Runtime Isolation (abp10 vs abp10-rt11 vs abp11)
To isolate how much of the performance difference comes from the .NET 11 runtime versus the ABP 11.0.0-rc.1 / EF Core 11 RC packages, two runtime isolation batches were executed (3 repetitions each):
6.3.1 Rate 4 it/s (Medium Load, ~321 req/s)
Command:
./scripts/run-bench.ps1 -Mode fixed -Rates 4 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch runtime
node k6/tools/summarize.mjs results/runtime report/results-runtime.md
| target | req/s | avg ms | p95 ms | p99 ms | app CPU % | DB CPU % | working set MB | GC pause ms/s |
|---|---|---|---|---|---|---|---|---|
| abp10 | 321.3 | 9.1 | 26.6 | 74.5 | 28.3 | 33.0 | 650 | 24.3 |
| abp10-rt11 | 321.4 | 9.2 | 26.8 | 75.2 | 29.3 | 32.3 | 674 | 20.1 |
| abp11 | 321.4 | 9.3 | 27.3 | 76.1 | 30.5 | 32.9 | 655 | 24.3 |
6.3.2 Rate 6 it/s (High Load, ~480 req/s)
Command:
./scripts/run-bench.ps1 -Mode fixed -Rates 6 -Targets abp10,abp10-rt11,abp11 -Repeats 3 -Batch runtime6
node k6/tools/summarize.mjs results/runtime6 report/results-runtime6.md
| target | req/s | avg ms | p95 ms | p99 ms | app CPU % | DB CPU % | working set MB | GC pause ms/s |
|---|---|---|---|---|---|---|---|---|
| abp10 | 481.5 | 11.5 | 34.9 | 93.7 | 49.7 | 56.6 | 731 | 33.0 |
| abp10-rt11 | 479.2 | 11.3 | 34.1 | 94.3 | 49.2 | 54.2 | 736 | 32.8 |
| abp11 | 478.4 | 11.4 | 34.1 | 95.8 | 49.8 | 54.9 | 746 | 32.9 |
p95 per page group at 6 it/s (ms):
| target | 01_login | 02_home | 03_users | 04_roles | 05_tenants | 06_settings | 07_profile | 08_language | 09_logout |
|---|---|---|---|---|---|---|---|---|---|
| abp10 | 68.9 | 8.3 | 73.7 | 29.0 | 89.1 | 14.4 | 30.3 | 30.6 | 10.0 |
| abp10-rt11 | 71.7 | 8.6 | 74.8 | 28.1 | 90.4 | 14.7 | 30.2 | 30.2 | 11.0 |
| abp11 | 71.5 | 8.7 | 75.6 | 27.3 | 91.7 | 16.3 | 30.1 | 30.7 | 11.5 |
6.3.3 Rate 7 it/s Saturation Knee Validation (verify-knee, ~557 req/s)
To test behavior right at the tipping point between comfortable load (6 it/s) and full saturation (8 it/s), a 9-run batch was executed across all three targets at 7 it/s:
| target | req/s | avg ms | p95 ms | p99 ms | app CPU % | DB CPU % | working set MB | GC pause ms/s |
|---|---|---|---|---|---|---|---|---|
| abp10 | 557.5 | 13.1 | 39.5 | 108.9 | 62.6 | 69.1 | 790 | 34.9 |
| abp10-rt11 | 557.7 | 13.4 | 40.3 | 113.0 | 63.0 | 70.4 | 793 | 35.2 |
| abp11 | 557.2 | 13.3 | 40.2 | 112.9 | 63.1 | 69.5 | 798 | 35.3 |
p95 per page group at 7 it/s (ms):
| target | 01_login | 02_home | 03_users | 04_roles | 05_tenants | 06_settings | 07_profile | 08_language | 09_logout |
|---|---|---|---|---|---|---|---|---|---|
| abp10 | 83.2 | 10.0 | 86.5 | 33.1 | 101.7 | 16.3 | 33.7 | 34.4 | 12.9 |
| abp10-rt11 | 86.2 | 9.8 | 89.8 | 32.7 | 107.0 | 16.9 | 35.1 | 35.0 | 13.1 |
| abp11 | 85.8 | 9.7 | 89.0 | 32.3 | 106.8 | 17.9 | 35.2 | 35.1 | 12.9 |
6.3.4 Rate 6 it/s Targeted Marathon Validation (verify-marathon6, ~478 req/s)
To verify sustained load endurance without repeating lower rates, a 9-run marathon was executed across all three targets specifically at 6 it/s (70 minutes total duration):
| target | req/s | avg ms | p95 ms | p99 ms | app CPU % | DB CPU % | working set MB | GC heap MB |
|---|---|---|---|---|---|---|---|---|
| abp10 | 478.0 | 12.7 | 37.9 | 102.2 | 54.7 | 56.5 | 756 | 345 |
| abp10-rt11 | 478.5 | 11.5 | 34.5 | 95.6 | 49.7 | 56.2 | 737 | 321 |
| abp11 | 478.7 | 11.6 | 34.6 | 96.9 | 50.3 | 55.8 | 737 | 317 |
p95 per page group at 6 it/s marathon (ms):
| target | 01_login | 02_home | 03_users | 04_roles | 05_tenants | 06_settings | 07_profile | 08_language | 09_logout |
|---|---|---|---|---|---|---|---|---|---|
| abp10 | 75.1 | 9.9 | 79.6 | 31.0 | 97.7 | 16.2 | 33.3 | 34.1 | 12.8 |
| abp10-rt11 | 73.0 | 9.1 | 76.3 | 27.8 | 91.7 | 15.4 | 30.5 | 30.9 | 11.2 |
| abp11 | 73.0 | 9.0 | 76.7 | 28.0 | 92.9 | 16.6 | 30.5 | 31.2 | 11.9 |
6.3.5 Full 3-Target Re-Validation Marathon (main-validation, ~3.5 hours, 27 runs)
As a complete, uncompromised end-to-end verification, the entire main matrix across all three targets (abp10, abp10-rt11, abp11) was executed at rates 2, 4, and 6 it/s (3 repeats × 3 rates × 3 targets = 27 runs, 12,538 seconds / 3.5 hours total duration) without overwriting any earlier data:
| rate | target | req/s | avg ms | p95 ms | p99 ms | app CPU % | DB CPU % | working set MB | GC heap MB |
|---|---|---|---|---|---|---|---|---|---|
| 2 it/s | abp10 | 160.8 | 10.8 | 34.0 | 78.7 | 16.4 | 17.3 | 613 | 222 |
| 2 it/s | abp10-rt11 | 160.5 | 10.6 | 33.0 | 77.1 | 16.2 | 17.4 | 616 | 219 |
| 2 it/s | abp11 | 160.5 | 10.7 | 33.7 | 80.0 | 15.9 | 17.0 | 611 | 212 |
| 4 it/s | abp10 | 321.1 | 10.0 | 30.2 | 77.8 | 30.9 | 34.1 | 687 | 266 |
| 4 it/s | abp10-rt11 | 321.4 | 9.4 | 27.8 | 77.0 | 31.1 | 34.0 | 661 | 261 |
| 4 it/s | abp11 | 321.1 | 9.6 | 28.8 | 77.2 | 30.7 | 33.8 | 663 | 249 |
| 6 it/s | abp10 | 478.1 | 12.8 | 38.3 | 102.1 | 55.0 | 57.0 | 759 | 355 |
| 6 it/s | abp10-rt11 | 479.1 | 11.7 | 34.5 | 97.0 | 50.3 | 55.6 | 739 | 321 |
| 6 it/s | abp11 | 478.8 | 11.6 | 34.7 | 96.7 | 50.6 | 55.6 | 738 | 311 |
Key Insights from Validation & Runtime Isolation:
- In the 27-run
main-validationbatch (2, 4 and 6 it/s), .NET 11 showed lower latency and lower resource use than .NET 10 at the same throughput:- At 6 it/s, .NET 11 shows 5 pp lower CPU (50% vs 55%), a 9.4% lower p95 (34.7 ms vs 38.3 ms), 44 MB less GC heap (311 MB vs 355 MB) and 21 MB less working set (738 MB vs 759 MB).
- At 4 it/s, .NET 11 and .NET 10-rt11 are faster (p95 27.8–28.8 ms vs 30.2 ms) and leaner on memory.
- At 7 it/s near the saturation knee (separate 9-run
verify-kneebatch), all three setups stay stable at ~40 ms p95 with 0 dropped iterations and 0 errors.
- ABP 11 vs .NET 11 Separation: These are medians of three runs.
abp10-rt11andabp11are close at both load levels (34.5 ms vs 34.7 ms at 6 it/s, 27.8 ms vs 28.8 ms at 4 it/s), so ABP 11.0.0-rc.1 and EF Core 11 RC packages showed no measurable overhead in this workload.
6.4 Structural Bottlenecks & Diagnostic Findings
- Heaviest scenario groups (
05_tenants,03_users,01_login):- Consistently across all batches,
05_tenants(p95 ~73–91 ms),03_users(p95 ~62–75 ms), and01_login(p95 ~54–71 ms) have the highest latencies versus 8–30 ms for02_home,04_roles,06_settings,07_profile, and08_language. - Likely cause (inference, not profiled): user and tenant creation triggers password hashing (PBKDF2) and multiple database writes.
- Consistently across all batches,
2. Thermal & Saturation Knee Sensitivity:
- At ~50% CPU load (isolated 6 it/s), latency is flat at ~34 ms p95.
- Near the saturation knee (8 it/s = ~97% CPU) minor queuing differences are amplified into large latency differences (see 6.1).
RuntimeBinderExceptionindotnet-counters:- All three setups showed 10
RuntimeBinderExceptionin the first second after startup and 0 for the remainder of the run. Confined strictly to the first second of warm-up; zero impact on the measured window. - Likely cause (inference): first-time binding of C#
dynamiccall sites populating DLR cache.
- All three setups showed 10
6.5 Consolidated Master Matrix & Unified Conclusions
Combining all validated data across the full load spectrum (2, 4, 6, 7, and 8 it/s) yields a single progression model:
| Arrival Rate | Target | Throughput (req/s) | Avg (ms) | p95 (ms) | p99 (ms) | Web CPU % | DB CPU % | Working Set (MB) | GC Heap (MB) | Unified Assessment |
|---|---|---|---|---|---|---|---|---|---|---|
| 2 it/s (~25% load) | abp10abp10-rt11abp11 |
160.8 160.5 160.5 |
10.8 10.6 10.7 |
34.0 33.0 33.7 |
78.7 77.1 80.0 |
16.4% 16.2% 15.9% |
17.3% 17.4% 17.0% |
613 616 611 |
222 219 212 |
Identical baseline (within 0.7 ms noise) |
| 4 it/s (~50% load) | abp10abp10-rt11abp11 |
321.1 321.4 321.1 |
10.0 9.4 9.6 |
30.2 27.8 28.8 |
77.8 77.0 77.2 |
30.9% 31.1% 30.7% |
34.1% 34.0% 33.8% |
687 661 663 |
266 261 249 |
.NET 11 advantage begins: -7% p95, -17 MB heap |
| 6 it/s (~75% load) | abp10abp10-rt11abp11 |
478.1 479.1 478.8 |
12.8 11.7 11.6 |
38.3 34.5 34.7 |
102.1 97.0 96.7 |
55.0% 50.3% 50.6% |
57.0% 55.6% 55.6% |
759 739 738 |
355 321 311 |
.NET 11 wins clearly: -9.4% p95, -5 pp CPU, -44 MB heap |
| 7 it/s (~90% knee) | abp10abp10-rt11abp11 |
557.5 557.7 557.2 |
13.1 13.4 13.3 |
39.5 40.3 40.2 |
108.9 113.0 112.9 |
62.6% 63.0% 63.1% |
69.1% 70.4% 69.5% |
790 793 798 |
375 348 353 |
Pre-saturation parity: stable at ~40 ms, 0 errors |
| 8 it/s (Saturation cliff) | abp10abp11 |
625.3 588.9 |
16.5 19.7 |
195.8 392.9 |
364.8 1056.8 |
97.4% 97.5% |
88.7% 91.4% |
866 906 |
— | Hardware saturation: CPU maxed out (97%), dropped iterations begin |
The 3 Unified Architectural Takeaways:
- .NET 11 Showed Lower Latency and Resource Use at the Same Throughput: At 6 it/s throughput was the same (478.1 / 479.1 / 478.8 req/s), while p95 latency was 9.4% lower, CPU was 5 pp lower and the GC heap was 44 MB smaller.
- ABP 11 Packages Showed No Measurable Overhead: Comparing
abp10-rt11directly againstabp11shows that upgrading to ABP 11.0.0-rc.1 and EF Core 11 RC added +0.2 ms p95 at 6 it/s and +1.0 ms at 4 it/s (medians of three runs). The runtime gains carried over to the ABP 11 application. - Consistent Saturation Threshold: Both stacks maintain flat latencies (~30–40 ms) from 160 req/s up to 557 req/s, and both encounter CPU exhaustion at ~8 it/s (~600 req/s) on the pinned 4 P-core hardware envelope.
7. Comparability Limitations & Interpretation Notes
- Coupled Framework and Runtime Changes (
abp10vsabp11):- Moving from
abp10toabp11changes both the .NET runtime (10.0.13→11.0.0-rc.3), Entity Framework Core (10.0.11→11.0.0-rc.3), and ABP Framework (10.7.1→11.0.0-rc.1, LeptonX Lite5.7.1→6.0.0-rc.1). - Differences between
abp10andabp11must not be attributed solely to .NET 11. Compareabp10againstabp10-rt11for the pure runtime effect, andabp10-rt11againstabp11for the ABP 11 / EF Core 11 package effect.
- Moving from
- Pre-Release Software Status:
- Both .NET 11 (
11.0.0-rc.3.26522.148) and ABP 11 (11.0.0-rc.1) are pre-release builds, whereas .NET 10 (10.0.13) and ABP 10 (10.7.1) are mature stable patch releases.
- Both .NET 11 (
- Npgsql Package Asymmetry:
abp11usesNpgsql.EntityFrameworkCore.PostgreSQL11.0.0-rc.1.1(compiled against EF Core11.0.0-rc.1, emittingNU1608against EF Core11.0.0-rc.3), while bothabp10andabp11resolve the underlyingNpgsqlADO.NET driver to10.0.3.
- Single-Host Execution & WSL2 Database Affinity:
- All components run on one physical machine. While the .NET web app (
0xFF, P-cores0..7) and k6 (0xFF0000, E-cores16..23) are strictly pinned, PostgreSQL runs inside Docker Desktop's WSL2 VM, which is scheduled by the Windows hypervisor across the remaining cores rather than hard-pinned via process affinity.
- All components run on one physical machine. While the .NET web app (
- Controlled Deviations from Raw Browser Recording:
- Role ID Lookup: The Roles page fetches the first 10 roles (
maxResultCount=10). Under multi-VU concurrency, more than 10 roles can exist simultaneously, soscenario.jsqueries/api/identity/roles?filter=<name>to locate the VU's role deterministically. - Password Change Omitted: Changing a user's password (
POST /api/account/my-profile/change-password) is excluded so VUs can re-authenticate cleanly across iterations without invalidatingBENCH_PASSWORD. - Host Features Modal: The open-source ABP template defines no host-scoped features (
Manage Host Featuresmodal renders "No feature available" with no Save button), so06_settingsopens the modal (GET /FeatureManagement/FeatureManagementModal?providerName=T) without aPOST. - Pre-Seeded Global Email Settings:
AbpSettingsstores global settings withProviderKey = NULL. Because PostgreSQL unique indexes treatNULLvalues as distinct unlessNULLS NOT DISTINCTis specified, concurrent first-time saves ofAbp.Mailing.Smtp.UserNameandAbp.Mailing.Smtp.Domaincan insert duplicate rows. Seeding those two rows once inscripts/seed-bench-data.sqlensuresPOST /api/setting-management/emailingperforms deterministic updates under concurrency. - Soft-Deleted Entities & Database Growth: Deleting users and tenants in ABP performs a soft delete (
IsDeleted = true), and tenant creation leaves behind the tenant's seeded admin user/role/permission rows. Within a 5-minute run the database grows slightly; resettingBenchfromBench_seed_<target>before every run prevents cross-run accumulation. - Template Database Statistics: The first probe run exposed that
Bench_seed_abp11had no planner statistics whileBench_seed_abp10did (autovacuum timing), which changed query plans and inflatedabp11latency and DB CPU.VACUUM ANALYZEwas added toscripts/seed-bench-data.sqland run on both templates; all results in this report were measured after that fix.
- Role ID Lookup: The Roles page fetches the first 10 roles (
- Run-to-Run Drift Between Batches: Absolute latencies differ between batches taken at different times (
abp10p95 at4it/s: 30.2 ms inmain-validation, 26.6 ms inruntime). Only compare targets within the same batch.


