# Backend Performance — Resolved

Items from the backend performance review that are done. Numbering matches
[todo.md](todo.md) — resolved items keep their original number so references stay valid.

Most entries are fixes. Items **6** and **14** are closed as *accepted* rather than fixed —
kept here so they are not re-reported as new findings.

---

### 4. Streaming export enriched contents one at a time — **FIXED**
`backend/src/Squidex.Domain.Apps.Entities/Contents/Queries/ContentQueryService.cs:49`

**Was:** `StreamAsync` called `contentEnricher.EnrichAsync(content, ...)` per item. The
single-item overload wraps the content in `Enumerable.Repeat(content, 1)` and runs the
whole pipeline for it — a new result `List`, a new schema-cache `Dictionary`, and every
`IContentEnricherStep` twice. Every batching optimisation in `ResolveReferences`,
`ResolveAssets` and `ConvertData` was defeated, so reference resolution degenerated to
one DB round trip per content. A 100k-content export meant 100k pipeline setups.

**Now:**

```csharp
await foreach (var batch in contents.Batch(50, ct).WithCancellation(ct))
{
    var enriched = await contentEnricher.EnrichAsync(batch, context, ct);
    foreach (var content in enriched)
    {
        yield return content;
    }
}
```

`Batch` yields `List<T>`, which binds to the `IEnumerable<Content>` overload, and that
overload calls `EnrichInternalAsync(contents, cloneData: false, ...)` — matching the
previous single-item behaviour. Reference resolution now amortises across 50 contents
instead of one DB round trip each.

**Follow-up:** 50 is conservative next to the 200-item batches used elsewhere
(`RuleEnqueuer.BatchSize`). Once profiled, a larger batch would amortise further.

---

### 5. `WriteManyAsync` iterated the unfiltered job list — **FIXED**
`backend/src/Squidex.Data.MongoDb/Domain/Apps/Entities/Contents/MongoContentRepository_SnapshotStore.cs:138`

**Was:** the method built `validJobs` via `jobs.Where(x => IsValid(x.Value)).ToList()`
and then looped over `jobs`. Two defects in one — the corrupt-data guard was bypassed
(the comment above it notes the data "might throw an exception if we do not ignore it"),
and the sequence was enumerated twice, re-running any upstream projection.

**Now:** `foreach (var job in jobs)` → `foreach (var job in validJobs)`.

---

### 7. Regex rebuilt per content write — **FIXED (the expensive part)**
`backend/src/Squidex.Domain.Apps.Core.Operations/ValidateContent/Validators/PatternValidator.cs`

**Was:** every content write constructs a fresh `ContentValidator` and with it a whole
validator object graph. For each pattern field that included
`new Regex($"^{pattern}$", options, Timeout)` in the constructor — a full pattern parse
and interpreter build. A 10k-item import with 5 pattern fields did 50k pattern parses.

**Now:** `PatternValidator` resolves its `Regex` from a process-wide, 1000-entry
`Squidex.Caching.LRUCache<(string Pattern, RegexOptions Options), Regex>`. The same
import does 5 parses.

**Why a plain static cache and not an async-local / request-scoped one.** The cacheable
unit turned out to be *only* the `Regex`, and a `Regex` has no dependency on the request
at all — it is a pure function of (pattern, options), and `Regex` instances are
thread-safe for matching. So a process-wide cache is both simpler and strictly more
effective than a request-scoped one, which would rebuild each pattern once per request.

**Why the validator tree itself is still rebuilt per write.** Caching the graph — even
per request — is not safe. It captures per-item state at several levels:

| Captured state | Where |
| --- | --- |
| `context.Root.PreviousData` | `DefaultValidatorsFactory` → `NotChangedValidator` |
| `context.Action` (Publish vs not) | `IsRequired` in both factories — changes *which* validators are emitted |
| `context.Mode` (Optimized) | `DependencyValidatorsFactory` short-circuits entirely |
| `context.Root.App` / `.Schema` | closures in `CheckAssets` / `CheckContentsByIds` / `CheckUniqueness` |

A bulk import is a single request but each item carries its own `PreviousData` and
`CommandId`, so even an `ILocalCache` keyed by schema would hand back a graph wired to
the previous item. The remaining per-write cost is a few hundred small gen-0 allocations
(dictionaries and `AggregateValidator` arrays) — real, but an order of magnitude below
the pattern parse that was removed. Reworking the factories to split
"schema-shaped, cacheable" from "context-bound" validators is the follow-up if profiling
says the churn still matters.

`RegexOptions.Compiled` was deliberately *not* added: it moves cost into IL emit and the
generated code can never be unloaded, which is a bad trade for user-authored patterns.

**The cache access is locked, and has to be.** `LRUCache` is a plain `Dictionary` plus a
`LinkedList` with no synchronisation, and its `TryGetValue` *mutates* the recency list —
so there is no lock-free read path. Verified empirically against the shipped
`Squidex.Caching` 8.0.3 assembly: 8 threads hammering an unguarded instance produced
`InvalidOperationException: The LinkedList node does not belong to current LinkedList`,
`ArgumentException: An item with the same key has already been added`, and repeated
`NullReferenceException`s. (The assembly *does* reference `Monitor`, but from other types
in the package — not `LRUCache`.) Validators are constructed concurrently on every
content write, so this path is genuinely contended.

`new Regex(...)` is built *outside* the lock, so pattern parsing is never serialised
across threads; a cold race can build the same pattern twice, which only wastes a little
work and never returns anything incorrect. The critical section is just the dictionary
and linked-list updates.

**Verified:** `dotnet build` clean (0 warnings); a harness mirroring `GetRegex` ran 1.6M
operations over 8 threads against 3000 distinct patterns in a 1000-entry cache
(continuous eviction) with 0 exceptions, 0 wrong matches and the cache correctly bounded
at 1000; full `Squidex.Domain.Apps.Core.Tests` suite green (1247), plus 25 validation
tests in `Squidex.Domain.Apps.Entities.Tests`.

---

### 8. GraphQL field-selection data loader never matched its results — **FIXED**
`backend/src/Squidex.Domain.Apps.Entities/Contents/GraphQL/GraphQLExecutionContext.cs`

**Was:** two separate defects in the `GetContentsLoaderWithFields` path, which serves
every GraphQL reference resolved under the `@optimizeFieldQueries` directive.

1. `BuildKeys` wrote `keys[i] = (ids[0], fields)` — every key in the batch was the
   *first* id, so one content was requested N times and the other N−1 never were.
2. The batch callback keyed its result dictionary by a freshly merged field set:

   ```csharp
   var fields = batch.SelectMany(x => x.Fields).ToHashSet();
   return result.ToDictionary(x => (x.Id, fields));
   ```

   `NonCachingBatchLoader` then looks the results up with the *original* key. The key
   type is `(DomainId, HashSet<string>)` and `HashSet<T>` has no structural equality, so
   the tuple comparer fell back to reference equality and **no lookup ever matched**.
   Contents were fetched from the database and thrown away; every field-selected
   reference resolved to `null`.

   This was unconditional, not a race: `SharedExtensions.FieldNames()` builds a *new*
   `HashSet` per resolver invocation (`new FieldNameResolver(...).Iterate(...)`), so the
   requested instance and the merged instance were never the same object.

**Now:** `(1)` was fixed to `ids[i]`. For `(2)`, the key is compared by value:

```csharp
private static readonly IEqualityComparer<HashSet<string>> FieldsComparer = HashSet<string>.CreateSetComparer();

private sealed class ContentWithFieldsComparer : IEqualityComparer<(DomainId Id, HashSet<string> Fields)>
{
    public bool Equals((DomainId Id, HashSet<string> Fields) x, (DomainId Id, HashSet<string> Fields) y)
        => x.Id.Equals(y.Id) && FieldsComparer.Equals(x.Fields, y.Fields);

    public int GetHashCode((DomainId Id, HashSet<string> Fields) obj)
        => HashCode.Combine(obj.Id, FieldsComparer.GetHashCode(obj.Fields));
}
```

and the callback groups by field selection instead of merging:

```csharp
var result = new Dictionary<(DomainId Id, HashSet<string> Fields), EnrichedContent>(ContentWithFieldsComparer.Instance);

foreach (var byFields in batch.GroupBy(x => x.Fields, FieldsComparer))
{
    var contents = await QueryContentsByIdsAsync(byFields.Select(x => x.Id), byFields.Key, ct);

    foreach (var content in contents)
    {
        result[(content.Id, byFields.Key)] = content;
    }
}
```

Grouping rather than merging matters for correctness: a batch can hold several different
field selections, and merging them would hand a caller fields it did not request. Because
the grouping is by *value*, identical selections coming from different resolvers still
collapse into a single query — which the old reference-equality behaviour could not do.

`HashSet<string>.CreateSetComparer()` is cached in a static; it allocates a new comparer
on every call.

**Verified:** a new regression test,
`GraphQLQueriesTests.Should_resolve_referenced_contents_when_field_queries_are_optimized`,
resolves a reference under `@optimizeFieldQueries`. It **fails on the pre-fix code** and
passes after — red-to-green, not just green. Full GraphQL suite (79) and full
`Squidex.Domain.Apps.Entities.Tests` (1527) green.

---

### 9. Generic query-model cache key collided across apps — **FIXED**
`backend/src/Squidex.Domain.Apps.Entities/Contents/Queries/ContentQueryParser.cs:276-294`

**Was:** the cross-schema (`schema == null`) cache key was the constant
`"EDM/__generic"` / `"JSON/__generic"`. The cached model is built from
`context.App.PartitionResolver()`, so whichever app populated the cache first imposed its
languages on every other app's cross-schema `/contents` queries for the 60-minute cache
lifetime — wrong filters accepted, correct ones rejected, across tenants.

An intermediate fix replaced it with `$"EDM/{app.Version}/{withHidden}"`, which did not
close the hole: `App.Version` is `Entity.Version`, a per-aggregate event-stream position,
so two apps with the same event count still collided.

**Now:** the key carries the app identity (commit `b7103a12`):

```csharp
return $"EDM/{app.Id}/{app.Version}/{withHidden}";
return $"EDM/{app.Id}/{app.Version}/{schema.Id}_{schema.Version}/{withHidden}";
```

`app.Id` is a globally unique `DomainId`, so no two apps can share a key.

**Deliberately not changed: the `app.Version` over-invalidation.** Keying on `app.Version`
means any app-level event (a contributor edit, a settings tweak) rebuilds the EDM models
of every schema in the app. Narrowing it to a language-specific token looked attractive —
`PartitionResolver` is just `app.Languages.ToResolver()` — but `BuildDataSchema` also
reads `partitioning.GetName(...)` and `IsOptional`, so a key built from the language
*codes* alone could serve a stale model after a language rename or fallback change.
`app.Version` is conservative but provably correct: it changes whenever anything about
the app does. Trading guaranteed correctness for a cache-hit-rate win is the wrong
direction here, so it stays until someone establishes the model's exact dependency set.

---

### 6. Sync-over-async on the authentication path — **CLOSED: ACCEPTED, WON'T FIX**
`backend/src/Squidex/Areas/IdentityServer/Config/Dynamic/DynamicSchemeProvider.cs:129`

```csharp
var scheme = GetSchemeCoreAsync(name, default).Result;
```

`Get(string? name)` blocks a thread-pool thread on a DB round trip, which in a hot path
is a classic thread-pool starvation source.

**Closed as accepted, not fixed.** This is dynamic OIDC scheme resolution — reached only
for team-level auth domains, not on ordinary API traffic — so the risk does not justify
the rework. Recorded here rather than deleted so it is not re-reported as a new finding.

If it ever moves onto a hot path, the fix is to cache scheme results synchronously
(populated by an async initializer / background refresh) so `Get` can return without
blocking.

Same pattern elsewhere, also accepted:
- `Squidex.Domain.Apps.Entities/Contents/DomainObject/Guards/ScriptingExtensions.cs:144` — `.Wait()` on full content validation inside a script callback.
- `Squidex.Data.MongoDb/Infrastructure/MongoRepositoryBase.cs:26` — `InitializeAsync(default).Wait()`.

---

### 12. `ReaderWriterLockSlim` used exclusively for write locks in the ETag path — **FIXED**
`backend/src/Squidex.Web/Pipeline/CachingManager.cs`

**Was:** `CacheContext` guarded `AddDependency`, `AddDependency<T>`, `AddHeader` and
`Finish` with `ReaderWriterLockSlim` — but every one of them took `EnterWriteLock`. No
code path ever took a read lock, so the reader/writer bookkeeping was pure overhead at
roughly 2–3× the cost of a plain monitor. `AddDependency` is called once per content,
once per schema and once per resolved reference, so a 200-item list with references took
on the order of a thousand write-lock round trips per request.

**Now:** a plain `Lock` (`System.Threading.Lock`, matching `DisposableObjectBase`), with
each `EnterWriteLock`/`try`/`finally`/`ExitWriteLock` block collapsed to `lock (...)`.

Two incidental improvements fell out of the rewrite:

- `Dispose()` no longer has a lock to dispose, so `CacheContext` only disposes the hasher.
- `AddHeader` had its `EnterWriteLock` *inside* the `try`, so a throw from the acquire
  would have hit `ExitWriteLock` on an unheld lock and masked the original error with a
  `SynchronizationLockException`. `lock` cannot express that shape.

Nothing about the concurrency contract changed — every operation mutates the hasher and
the sets, so there was never anything a read lock could have protected.

**Verified:** build clean, `Squidex.Web.Tests` green (167).

---

### 13. Rules dictionary rebuilt per event inside the batch loop — **FIXED**
`backend/src/Squidex.Domain.Apps.Entities/Rules/RuleEnqueuer.cs`

**Was:** `On(...)` receives batches of 200 events and ran
`Rules = rules.ToReadonlyDictionary(x => x.Id)` for *each* one — a full `Dictionary`
build plus a wrapper allocation per event, even though the events in a batch are
overwhelmingly from the same app.

Note the rules *lookup* was already cheap: `RulesCacheDuration` defaults to 10s, so
`appProvider.GetRulesAsync` was memoized. The waste was purely the per-event indexing.

**Now:** the batch is grouped by app, so rules are resolved and indexed once per app and
the context is built once per group:

```csharp
foreach (var byApp in events.GroupBy(GetAppId))
{
    if (byApp.Key == null) { continue; }

    var rules = await GetRulesAsync(byApp.Key.Id);
    if (rules.Count == 0) { continue; }

    var context = new RulesContext { AppId = byApp.Key, Rules = rules.ToReadonlyDictionary(x => x.Id), ... };

    foreach (var @event in byApp) { ... }
}
```

`GetAppId` returns `null` for restored events and non-`AppEvent` payloads, so they all
collect into one group that is skipped — replacing the two per-event `continue` guards.

**Why `GroupBy` rather than memoizing per app inside the original loop.** The first
attempt kept the original per-event loop and cached the indexed dictionary in a
`Dictionary<DomainId, ...>`, specifically to avoid reordering events. `GroupBy` does
reorder across apps, so that had to be checked rather than assumed:

- Rules are scoped to a single app (`context.AppId`, `context.Rules`), so a rule cannot
  observe another app's events.
- `RuleQueueWriter` is app-agnostic — it accumulates `CreateFlowInstanceRequest` values
  and flushes every 100 regardless of origin.
- `ruleUsageTracker.TrackAsync` is an additive counter per (app, rule, day).
- `GroupBy` preserves source order *within* each group, which is the ordering that can
  actually matter.

Nothing cross-app is order-sensitive, so `GroupBy` is safe — and it is both simpler and
slightly more correct than the memo: keying on `NamedId<DomainId>` (a `sealed record`,
so value equality over id *and* name) means an app renamed mid-batch yields two groups
each carrying its own correct name, where the memo keyed on `.Id` would have reused the
first name seen.

**Verified:** a new test,
`RuleEnqueuerTests.Should_handle_events_of_multiple_apps_with_the_rules_of_each_app`,
feeds an interleaved two-app batch and asserts each event is handled with its own app's
rules, that the grouped order is what reaches the service, and — via `Assert.Same` on the
`Rules` instance — that indexing happens once per app rather than once per event. It
**fails on the pre-fix code** (the ordering assertion shows `app1, app2, app1, app2`
against the expected `app1, app1, app2, app2`) and passes after. Existing coverage did
not include a multi-app batch at all: `Should_handle_events_in_batches` repeats the *same*
event ten times. Full `Squidex.Domain.Apps.Entities.Tests` green (1528).

---

### 20. Script cache key embedded the entire script source — **FIXED**
`backend/src/Squidex.Domain.Apps.Core.Operations/Scripting/Internal/CacheParser.cs:20`

**Was:**

```csharp
var cacheKey = $"{typeof(CacheParser)}_Script_{script}";
```

Every parse allocated a new string holding a full copy of the script body, and
`IMemoryCache` then retained that copy as the key — so each cached script was held twice.

**Now:** `var cacheKey = (typeof(CacheParser), script);`

The tuple boxes once (one small allocation) but holds a *reference* to the existing
script string, so nothing is copied and the cache no longer keeps a second copy alive.

**Honest limit:** this removes the allocation and the duplicate retention, not the hash.
`ValueTuple.GetHashCode` still calls `string.GetHashCode()` on the source, which is O(n)
— .NET does not cache string hash codes. Removing that too would mean keying by schema id
+ script version, which needs that context plumbed into `CacheParser` and changes its API.
Not worth it unless profiling says the hash itself shows up.

---

### Tuple cache keys — sweep of the other call sites

Same change applied where the key was an interpolated string and the cache accepts
`object`. Beyond skipping the string build, a tuple also avoids *formatting* non-string
parts (`DateOnly`, `long`), which the interpolation did on every call.

| Site | Key before | Key now |
| --- | --- | --- |
| `CachingUsageTracker.GetForMonthAsync` | `$"{typeof(..)}_UsageForMonth_{key}_{date}_{category}"` | `(typeof(..), nameof(GetForMonthAsync), key, date, category)` |
| `CachingUsageTracker.GetAsync` | `$"{typeof(..)}_Usage_{key}_{fromDate}_{toDate}_{category}"` | `(typeof(..), nameof(GetAsync), key, fromDate, toDate, category)` |
| `EventEnricher.FindUserAsync` | `$"{typeof(..)}_Users_{actor.Identifier}"` | `(typeof(EventEnricher), actor.Identifier)` |
| `RuleEnqueuer.GetRulesAsync` | `$"{typeof(..)}_Rules_{appId}"` | `(typeof(RuleEnqueuer), appId)` |
| `UsageGate.CacheKey` | `$"{appId}_Plan"` | `(typeof(UsageGate), nameof(GetPlanForAppAsync), appId)` |
| `UsageGate` notified flag | bare `DomainId` | `(typeof(UsageGate), nameof(TrackNotified), appId)` |
| `CachingGraphQLResolver` | `$"GraphQLModel_{appId}_{etag}"` | `(typeof(CachingGraphQLResolver), app.Id, app.Version)` |
| `AppProvider` × 11 | `$"APPS_ID_{appId}"`, `$"GetSchemasAsync({appId})"`, … | `(nameof(AppProvider), "APPS_ID", appId)`, … |

Notes:

- `CachingUsageTracker.GetForMonthAsync` runs on **every API request** (via
  `UsageGate.IsBlockedAsync`) and its old key formatted a `DateOnly` — a culture lookup
  plus an allocation — before building an ~80-character string.
- `CachingGraphQLResolver` no longer needs
  `app.Version.ToString(CultureInfo.InvariantCulture)`; the tuple carries the `long`
  directly, so `System.Globalization` was dropped from the file.
- `UsageGate`'s notified flag previously used a bare `DomainId` as the key. It was safe
  only because that `MemoryCache` is private to the class; it is now explicit.
- `AppProvider` keys carry `nameof(AppProvider)` plus the lookup name, preserving the
  namespacing the old string prefixes provided. The two `TeamCacheKey` overloads and
  `CachingGraphQLResolver.CreateCacheKey` had a single call site each and were inlined;
  `AppCacheKey` and `SchemaCacheKey` have three each and stayed as helpers.

**Three sites were deliberately left as strings:**

- `MongoCountCollection.GetOrAddAsync(string key, …)` — used by `QueryByQuery` and
  `MongoAssetRepository`. That key is **persisted as a MongoDB document id**, not an
  in-memory cache key. Changing it would change stored data.
- `DataLoaderContext.GetOrAddLoader(string loaderKey, …)` — the GraphQL.DataLoader API
  takes a `string`, so `GraphQLExecutionContext.GetContent` cannot use a tuple.
- `Singletons<IMongoClient>.GetOrAdd(string, …)` — typed `string`, and startup-only.

**Verified:** build clean (0 warnings). `Squidex.Domain.Apps.Core.Tests` (1243),
`Squidex.Domain.Apps.Entities.Tests` (1528), `Squidex.Infrastructure.Tests` (1031) and
`Squidex.Web.Tests` (167) all green.

---

### 14. `AppProvider` copies cached schema/rule lists on every call — **CLOSED: ACCEPTED**
`backend/src/Squidex.Domain.Apps.Entities/AppProvider.cs`

`GetSchemasAsync` and `GetRulesAsync` end with `?.ToList() ?? []`, a defensive copy of the
cached list on every call including cache hits, and `GetRuleAsync` copies the whole rule
list just to `Find` one element.

**Closed as accepted, not fixed.** The copy is a single shallow `List` allocation of
already-immutable elements; returning the cached instance directly would expose it to
mutation by callers, which is a worse trade than the allocation. Recorded here so it is
not re-reported as a new finding.

---

### 15. Faulted tasks were cached permanently in `CollectionProvider` — **FIXED**
`backend/src/Squidex.Data.MongoDb/Domain/Apps/Entities/Contents/CollectionProvider.cs`

**Was:**

```csharp
return collections.GetOrAdd((appId, schemaId), CreateCollectionAsync);
```

Two defects. `CreateCollectionAsync` creates indexes, so it can fail transiently — and
`GetOrAdd` stored the returned `Task` including a *faulted* one for the process lifetime,
so a single Mongo hiccup on first access permanently broke queries for that app/schema
until restart. Separately, `GetOrAdd` may invoke its factory concurrently for the same
key, issuing duplicate `CreateManyAsync` calls.

**Now:** the dictionary holds `Lazy<Task<...>>` with `LazyThreadSafetyMode.ExecutionAndPublication`,
so the factory runs exactly once per key even under concurrent access, and the entry is
evicted when it fails:

```csharp
var collection = collections.GetOrAdd(key, CreateLazyCollection);

return AwaitCollectionAsync(key, collection);
...
try
{
    return await collection.Value;
}
catch
{
    collections.TryRemove(new KeyValuePair<...>(key, collection));
    throw;
}
```

The removal uses the `TryRemove(KeyValuePair)` overload, which only removes when the value
is still the *same* `Lazy` instance. The plain `TryRemove(key)` would race: a second thread
that had already retried and succeeded would have its good entry discarded by the first
thread's cleanup.

A `using` alias for the key tuple was tried first, but StyleCop's SA1008 rejects the space
before the parenthesis in `using X = (A, B);`, so the tuple type is written out instead.

**Verified:** build clean, `Squidex.Data.Tests` (180) and all other suites green.

---

### 16. `IsFrontendClient` re-scanned claims on every access — **FIXED (verified)**
`backend/src/Squidex.Domain.Apps.Entities/Context.cs:32,51`
`backend/src/Squidex.Infrastructure/Security/Extensions.cs:70`

**Was:** `public bool IsFrontendClient => UserPrincipal.IsInClient(DefaultClients.Frontend);`
— a computed property whose implementation was `principal.Claims.Any(x => ...)`, walking
every identity and every claim and allocating an enumerator plus a delegate per call. It is
read from several enrichment steps and from `ConvertData.GenerateConverter` per schema
group, so it ran many times per request against a value that cannot change.

**Now:** a get-only auto-property assigned once in the private constructor, and
`IsInClient` rewritten from LINQ `Any` to a plain `foreach`, dropping the closure.

**Verification found the commit did not compile.** Line 32 read
`public bool IsFrontendClient { get; };` — a stray semicolon, `error CS1597: Semicolon
after method or accessor block is not valid`. Removed the semicolon.

Beyond compiling, the assignment is correct for every construction path: the public
`Context(ClaimsPrincipal, App)` chains to the private constructor via `: this(...)`,
`Anonymous` and `Admin` both go through that public one, and `HeaderBuilder.Build` calls
the private 4-argument constructor directly. All four paths therefore set the field.

---

### 17. `ResolvingReferences()` re-evaluated per content — **FIXED**
`backend/src/Squidex.Domain.Apps.Entities/Contents/Queries/Steps/ResolveReferences.cs`

**Was:** `SchemaExtensions.ResolvingReferences` is a lazy
`Fields.OfType<...>().Where(...)` that is never materialized, and `AddReferenceIds` called
it *inside* the per-content loop — so the full field scan plus two LINQ iterator
allocations happened once per content instead of once per schema.

**Now:** hoisted out of the loop.

```csharp
var fields = schema.ResolvingReferences().ToList();

foreach (var content in contents)
{
    content.Data.AddReferencedIds(fields, ids, components);
}
```

(The other call site, the outer `foreach` in `ResolveReferencesAsync`, enumerates the
sequence exactly once and was left alone.)

**The double `GroupBy` was deliberately left alone.** `ResolveReferences.EnrichAsync` and
`ConvertData` each build `contents.GroupBy(x => x.SchemaId.Id)` twice. This does *not*
cause duplicate schema fetches: `ContentEnricher` passes a `ProvideSchema` delegate backed
by a per-call `schemaCache` dictionary, so the second grouping resolves every schema from
memory. The only real cost is re-materializing the LINQ `Lookup` — one extra pass over the
contents and one set of bucket allocations per step.

Deduplicating it was tried and reverted: the gain is small enough that it does not justify
threading a materialized `List<IGrouping<...>>` through the method signatures.

**Verified:** build clean, all suites green.

---

### 10. Unbounded in-memory request-log queue — **FIXED**
`backend/src/Squidex.Infrastructure/Log/BackgroundRequestLogStore.cs`
`backend/src/Squidex.Infrastructure/Log/RequestLogStoreOptions.cs`

**Was:** `jobs` was an unbounded `ConcurrentQueue<Request>`. `LogAsync` enqueues on every
API request while the flush timer drains only once per `WriteIntervall` (1s by default).
If `InsertManyAsync` threw — Mongo unreachable, disk full — the drain aborted and the
surviving entries stayed queued while new ones kept arriving. A sustained storage outage
under load grew the queue until the process ran out of memory: the request *log* taking
down the whole server.

**Now:** a soft bound with an explicit drop counter.

```csharp
if (Volatile.Read(ref jobsCount) >= options.MaxPendingItems)
{
    Interlocked.Increment(ref jobsDropped);
    return Task.CompletedTask;
}

Interlocked.Increment(ref jobsCount);

jobs.Enqueue(request);
```

`jobsCount` is decremented as the drain dequeues, so the queue accepts entries again once
it has been written. Each drain reports what it dropped via a new
`LogRequestLogDropped` message, so the gap in the request log is visible rather than
silent. `MaxPendingItems` defaults to 50,000 — roughly 50 seconds of headroom at 1000
requests/second — and is configurable.

The bound is deliberately *soft*: two threads can both observe `jobsCount < max` and both
enqueue, so the queue can overshoot by the number of concurrent writers. That is fine for
a backpressure limit and avoids a lock on the hot path.

A `Channel` with `BoundedChannelFullMode.DropWrite` was the alternative. The counter was
chosen because it keeps the existing drain loop unchanged and makes the drop explicit at
the call site instead of hiding it behind a channel option.

**Verified:** two new tests —
`Should_drop_logs_when_pending_queue_is_full` and
`Should_accept_logs_again_after_pending_queue_has_been_written`. Both **fail on the
pre-fix code**. The second was additionally mutation-checked: removing the
`Interlocked.Decrement` from the drain loop kills it and nothing else, confirming it
really covers the recovery path rather than passing incidentally. This required splitting
the test helper, because the existing `WaitForCompletion` disposes the store and so cannot
be used to drain twice. `Squidex.Infrastructure.Tests` green (1033).

---

### 11. Cross-schema content queries never used the cached total — **FIXED**
`backend/src/Squidex.Data.MongoDb/Domain/Apps/Entities/Contents/Operations/QueryByQuery.cs`

**Was:**

```csharp
var (filter, isDefault) = CreateFilter(app.Id, schemas.Select(x => x.Id), ...);
```

`isDefault` was computed and then discarded. The multi-schema overload had no
`else if (isDefault)` branch, unlike the single-schema overload thirty lines below which
routes through `countCollection.GetOrAddAsync`. So the "all schemas" `/contents` endpoint
ran a full uncached `CountDocumentsAsync` over every content in the app on each page.

**Now:** the branch is mirrored, keyed by app plus the schema set:

```csharp
else if (isDefault)
{
    var totalKey = CreateTotalKey(app, schemas);

    contentTotal = await countCollection.GetOrAddAsync(totalKey, ct => Collection.Find(filter).CountDocumentsAsync(ct), ct);
}
```

**The key needs care, which is why it is not just an interpolated list.** The schema set
depends on the caller's permissions and arrives in no guaranteed order, so the ids are
sorted before hashing — otherwise the same query would produce different keys and never
hit. And the key becomes the `_id` of the count document, where MongoDB caps index keys at
1024 bytes; a raw join of 37-character ids would exceed that at roughly 27 schemas. Hashing
gives a bounded, deterministic key:

```csharp
var schemaIds = schemas.Select(x => x.Id.ToString()).Order(StringComparer.Ordinal);

return $"{app.Id}_Schemas_{string.Join('_', schemaIds).ToSha256Base64()}";
```

The `_Schemas_` marker keeps this key space distinct from the single-schema overload's
`$"{appId}_{schemaId}"`. The two must not share entries in any case: their filters differ
(`Filter.In` vs `Filter.Eq`, and different existence guards), so the counts are not
interchangeable.

**Verified:** build clean, all suites green.