Skip to main content

Engine comparison: structural validation

Four engines were run against the same 51-case structural corpus on the same host, each in its own container constrained to 2 vCPU / 4 GiB (NGINX 1.26.2, libmodsecurity 3.0.13, Coraza 3.7.0 via Caddy 2.11.3, CRS 4.25.0; run of 2026-08-24, commit 08851ee):

EngineWhat it is
Baselineplain NGINX, no JSON processing at all
JSON GuardNGINX + ngx_json_guard (this module)
ModSecurity standaloneNGINX + ModSecurity JSON body processor, no rules
CorazaCaddy + Coraza JSON body processor, no rules

The question is narrow on purpose: when a structurally invalid or abusive JSON body arrives, who rejects it at the edge — and who lets it through?

Structural validation matrix​

Every cell is what the proxy did. "Pass" means the body was forwarded; the test upstream also parses JSON, so some forwarded bodies were eventually rejected by the application — that is the baseline's row in a nutshell: it validates nothing itself.

Violation classBaselineJSON GuardModSecurityCoraza
Malformed syntax (trailing comma, bad brackets)pass400400400
Invalid escape / invalid numberpass400400400
Invalid UTF-8 in stringspass400400pass
Unpaired UTF-16 surrogate ("\ud800")pass400passpass
Trailing garbage after documentpass400400400
Nesting depth > 100pass413400†400†
Array/object width > 10,000pass413413413
Key length > 4096pass413passpass
Number length > 256pass413passpass
String length > 64 MiBpass413passpass
Total tokens > 100,000pass413passpass
Classes caught at the edge (of 11)01165

† ModSecurity's and Coraza's depth and width rejections come from SecRequestBodyJsonDepthLimit 100 and SecArgumentsLimit 10001, set in the benchmark profiles (benchmark/modsecurity/*.conf, benchmark/coraza/*.conf). Neither engine's JSON body processor caps key, number or string length or total tokens (CRS has optional argument-length rules, but they run after the full parse); the guard's thresholds for all of these are directives in nginx.conf.

Structural violation classes rejected at the edge (of 11)
  • Baseline0 of 11
  • JSON Guard11 of 11
  • ModSecurity6 of 11
  • Coraza5 of 11
Count of 400/413 cells in each column of the matrix above. ModSecurity and Coraza depth and width rejections depend on the limits set in the benchmark profiles (see †).

Three takeaways:

  1. Everyone catches plain syntax errors. If malformed JSON were the only concern, any JSON-aware engine would do.
  2. Only the guard limits key, number, string and token size. Depth and argument count are configurable in ModSecurity and Coraza too (SecRequestBodyJsonDepthLimit, SecArgumentsLimit), but neither body processor caps a single key, number or string, or the total token count, so a 100 MiB single string reaches their parser (and your app) unless you add this layer.
  3. The guard is the only engine rejecting unpaired surrogates. That is strict RFC 8259 behavior; the others tolerate it. Know your producers — if yours emit lone surrogates legitimately, do not enable that path without checking (there is no off switch; it is inherent to strict validation).

What rejection costs​

From the processor suites (20 MiB bodies, medians; per-case table in artifacts/cross-waf/20260824-004254/COMPARISON.md, local run output that is gitignored and not committed):

CaseJSON GuardModSecurity aloneGuard + ModSecurity
Clean 20 MiB, accepted476 ms696 ms829 ms
Malformed tail 20 MiB, rejected395 ms542 ms323 ms
High-cardinality 20 MiB, accepted294 ms515 ms519 ms
20 MiB bodies, no CRS: client-observed median latency
Clean 20 MiB, accepted
  • JSON Guard476 ms
  • ModSecurity alone696 ms
  • Guard + ModSecurity829 ms
Malformed tail 20 MiB, rejected
  • JSON Guard395 ms
  • ModSecurity alone542 ms
  • Guard + ModSecurity323 ms
High-cardinality 20 MiB, accepted
  • JSON Guard294 ms
  • ModSecurity alone515 ms
  • Guard + ModSecurity519 ms
Same values as the table above. These are end-to-end HTTP latencies seen by the client (upload, proxy and upstream included), not parser time; only the guard exposes its own timer. Lower is better.

Reading: with the guard first, a malformed 20 MiB body is rejected 40% faster than ModSecurity's processor manages alone (323 vs 542 ms) — the guard's walk is cheaper than a full libmodsecurity parse, and rejected bodies skip rule evaluation entirely. The honest counterweight: an accepted body is validated twice (guard, then processor), costing ~19% more end-to-end in the processor-only configuration.

With full CRS in front of large JSON​

Under full CRS 4.25.0 with 20 MiB bodies, adding the guard in front does not change CRS throughput (1.77 vs 1.80 MiB/s — noise) and cuts the structural-correctness suite wall time from 150 s to 104 s, because malformed bodies are 400'd before rules run. A single 64 MiB string under full CRS: 268 s with Coraza vs 46.6 s with ModSecurity — both dominated by rule evaluation over the huge value, which is exactly the workload class the guard's json_guard_max_string_length lets you cap before any engine starts.

The complete methodology, raw tables, and reproduction steps are in BENCHMARK.md in the repository root, and the headline numbers are on the Performance page.