Table of Contents

Quality Reports

AsiBackbone uses quality reports to make the validation surface easier to inspect from the documentation site.

Coverage, mutation analysis, smoke testing, and concurrency validation answer different questions:

Report Question answered Purpose
Coverage Report Did the tests execute this code? Shows the tested surface area and highlights unvisited code paths.
Core Branch Coverage Did Core tests exercise the decision branches that protect governance behavior? Enforces a stricter branch gate for the framework-neutral Core package without applying that same threshold to every adapter or sample.
Mutation Analysis Would tests fail if behavior changed? Checks assertion strength by introducing small code mutations and verifying tests catch them.
External Consumer Smoke Test Can a clean host consume package-shaped artifacts? Validates package ergonomics, DI registration, host-owned EF persistence, in-memory audit storage, and HTTP allow/deny/acknowledgment flows.
EF Core Outbox Concurrency Validation What happens under concurrent EF Core writes, retryable failures, and drain-worker contention? Provides CI-friendly relational evidence for outbox/lifecycle persistence and documents the current non-claiming drain boundary.

For a governance package, these views all matter. Coverage helps show that policy, acknowledgment, audit, capability-token, DLP/classification, provider-neutral emission, durable outbox, signing, verification, and canonical-hashing paths are exercised. The Core branch coverage gate protects the framework-neutral governance engine from missed policy, decision, signing, capability, acknowledgment, audit, emission, outbox, DLP/classification, and verification branches. Mutation analysis helps show that tests are strong enough to detect behavior changes in selected high-value decision logic. External consumer smoke testing helps prove that package-shaped adoption works from outside the repository's normal project-reference graph. EF Core outbox concurrency validation adds repeatable evidence for durable outbox behavior under concurrent local persistence and worker contention without claiming exactly-once delivery.

Current quality posture

The 1.2.1 stable package family carries forward the 1.2.0 quality surface and adds release-readiness coverage around Source Link repository-commit metadata, package metadata validation, package-signing wording, GitHub Actions dependency refresh, workflow line-ending normalization, documentation alignment, and project-governance readiness.

Current Core coverage includes provider-neutral governance emission contracts, durable outbox contracts, DLP/classification failure policy primitives, signing-ready metadata, canonical hashing/signing seams, verification-policy primitives, and capability grant hardening. The targeted mutation reports remain quality signals for selected high-value behavior; they are not full-repository certification.

Tracked Core coverage-hardening work includes:

Available reports

Coverage Reports

The repository-wide coverage report is generated from the full test suite using Coverlet and ReportGenerator. It is published with the documentation site when the documentation workflow runs successfully.

The Core branch coverage report is generated separately from AsiBackbone.Core.Tests, filtered to AsiBackbone.Core, and enforced as a 90% branch coverage gate. The repository-wide 75% line coverage gate remains in place; the stricter Core gate does not apply to every adapter, storage provider, telemetry provider, or sample package.

Mutation Analysis

The mutation analysis reports are generated with Stryker.NET for targeted governance behavior. The Core report focuses on evaluator and policy-pipeline behavior, including denial precedence, decision outcome selection, reason-code preservation, and related edge cases. The ASP.NET Core report focuses on acknowledgment challenge round-trip behavior, including safe-default challenge shaping, correlation preservation, and conversion of host acknowledgment responses back into Core acknowledgment language.

The historical pre-1.0.0 mutation boundary and accepted 1.x expansion deferrals are documented in Mutation Coverage Scope and Deferrals. Post-1.1.0 xUnit coverage-hardening work is tracked separately from mutation-scope expansion so readers can distinguish execution coverage improvements from mutation-testing growth.

External Consumer Smoke Test

The external consumer smoke test packs the repository projects into local NuGet packages, generates a temporary external xUnit project, installs the local packages without project references, and validates Core, ASP.NET Core adapter, EF Core ledger, in-memory audit, and HTTP decision-flow ergonomics.

EF Core Outbox Concurrency Validation

The EF Core outbox concurrency validation tests run inside AsiBackbone.EntityFrameworkCore.Tests. They use SQLite shared in-memory relational persistence to exercise concurrent outbox/lifecycle writes, retryable drain failures, and multi-worker drain contention. The validation intentionally preserves the current boundary: the provider-neutral drain is not an exactly-once delivery system and does not claim or lease work before provider emission.

Note

If a report link is unavailable, the related workflow may not have generated that report yet. Generate the reports locally or rerun the Publish Quality Reports workflow after the report-producing steps are configured.

Local validation

Restore tools before running report commands:

dotnet tool restore

Generate the standard coverage report through the normal test and ReportGenerator flow:

dotnet test ./AsiBackbone.slnx --configuration Release --collect:"XPlat Code Coverage" --results-directory ./artifacts/test-results

dotnet reportgenerator \
  -reports:"./artifacts/test-results/**/coverage.cobertura.xml" \
  -targetdir:"./artifacts/coverage-report" \
  -reporttypes:"Html;MarkdownSummaryGithub;Cobertura" \
  -assemblyfilters:"-*.Tests" \
  -filefilters:"-**/bin/**;-**/obj/**;-**/*.g.cs"

Run the Core branch coverage gate and generate the Core-only report:

dotnet test ./tests/AsiBackbone.Core.Tests/AsiBackbone.Core.Tests.csproj \
  --configuration Release \
  /p:CollectCoverage=true \
  /p:CoverletOutputFormat=cobertura \
  /p:CoverletOutput="./artifacts/core-coverage/" \
  /p:Include="[AsiBackbone.Core]*" \
  /p:Exclude="[*.Tests]*" \
  /p:Threshold=90 \
  /p:ThresholdType=branch \
  /p:ThresholdStat=total

dotnet reportgenerator \
  -reports:"./artifacts/core-coverage/coverage.cobertura.xml" \
  -targetdir:"./artifacts/core-branch-coverage-report" \
  -reporttypes:"Html;MarkdownSummaryGithub;Cobertura" \
  -assemblyfilters:"+AsiBackbone.Core;-*.Tests" \
  -filefilters:"-**/bin/**;-**/obj/**;-**/*.g.cs"

Run the initial Core mutation analysis from the Core test-project folder:

cd ./tests/AsiBackbone.Core.Tests
dotnet tool run dotnet-stryker -- --config-file stryker-config.json

The Core mutation report is written to:

artifacts/mutation-report/core

Run the external consumer smoke test from the repository root:

bash ./eng/smoke-tests/external-consumer-smoke.sh

Run the EF Core outbox concurrency validation from the repository root:

dotnet test ./tests/AsiBackbone.EntityFrameworkCore.Tests/AsiBackbone.EntityFrameworkCore.Tests.csproj --configuration Release --filter FullyQualifiedName~EfCoreOutboxConcurrencyValidationTests

Interpreting mutation results

Mutation testing should be reviewed as a quality signal, not as a raw score chase.

Surviving mutants should be classified as:

  • missing assertion or weak test behavior;
  • equivalent or non-actionable mutant;
  • acceptable gap to document for a later issue;
  • intentional behavior that should be protected by a stronger test.

For Issue #134, see Core Test Triage for the focused Core survivor-triage pass covering Evaluation, Decisions, Audit, and Handshakes.

Interpreting external consumer smoke-test results

External consumer smoke-test failures should be treated as package-consumer ergonomics failures first. The generated test project is intentionally outside the solution's normal project-reference graph, so failures can expose packaging, dependency, registration, or host-boundary regressions that normal repository tests may not catch.

Interpreting EF Core outbox concurrency validation results

EF Core outbox concurrency validation failures should be reviewed as reliability-boundary regressions or documentation-boundary mismatches. Passing results show repeatable evidence for the tested relational path; they do not prove universal production throughput, exactly-once delivery, or distributed locking across all database providers.

Edit this page