Accuracy
Does the trace report known delays and the best-supported WordPress owner within each documented capability boundary?
Current evidence status: methodology defined; public results pending
Until representative runs, environment details, sample sizes, results, and limitations are published, the website should not state a fixed overhead number or describe WP Flame as production-safe.
Evidence questions
The benchmark must test measurement and interference.
Overhead
How do sampled-out, Safe, Standard, Deep, persistence, and dashboard paths change wall time, memory, trace size, and response behavior?
Operational safety
Do status, output, storage, retention, dropped spans, and failure behavior remain bounded across representative environments?
Representative scenarios
Test the workflows the product intends to serve.
| Scenario | Why it matters | Evidence required |
|---|---|---|
| Vanilla WordPress | Establishes a small, controlled baseline. | Disabled, sampled-out, Safe, Standard, Deep, and persistence deltas. |
| WooCommerce | Exercises frontend, checkout, admin, sessions, queries, and supported HTTP paths. | Separate workflow results; no general store-wide conclusion. |
| Elementor | Exercises a complex page-builder environment used in the intended market. | Capability report and known ownership fixtures. |
| REST and AJAX | Validates normalized dynamic operations beyond normal frontend pages. | Route/action identity and response equivalence. |
| Cron or Action Scheduler | Exercises bounded background work and long-running paths. | Job identity, capture expiry, storage and truncation behavior. |
| 100 queries | Tests compatible query capture and aggregation. | Known query count, measured overhead, trace bytes, and attribution limits. |
| 2,000 spans / maximum trace | Tests graph navigation, limits, dropped data, storage, and rendering. | Bounded behavior, disclosure, and browser rendering results. |
| WPGraphQL | Only relevant if public operation/resolver support is claimed. | Current end-to-end fixtures or explicit removal from advertised scope. |
Run method
How results should be produced
- Document environment, WordPress, PHP, database, plugin, theme, and WP Flame versions.
- Warm the environment using a published, repeatable procedure.
- Use at least 100 measured runs per representative scenario unless methodology justifies more.
- Randomize or alternate conditions to reduce time-order bias where practical.
- Record failures, outliers, dropped/truncated spans, and output/status differences.
- Publish fixtures and scripts needed to reproduce the claim where licensing permits.
Reported metrics
What each result must show
- p50 and p95 wall-time delta, not a single best run.
- CPU only where the environment actually measures CPU.
- Peak-memory delta, trace bytes, and database growth.
- Sample size, time window, warm-up, and comparison basis.
- Response body/status equivalence and observed interference.
- Limitations and the exact claim the evidence supports.
Public evidence register
No gate is implied complete.
| Evidence | Current public status | Required before claim |
|---|---|---|
| Mode-specific overhead | Not published | Dated methodology, representative environments, samples, results, and limitations. |
| Accuracy comparison | Not published | Known fixtures and careful comparison with independent tools. |
| Compatibility matrix | Not published | Expected, degraded, and unavailable capabilities per tested row. |
| Scale limits | Not published | Maximum trace, storage, retention, pruning, and rendering evidence. |
| Case studies | Not published | Separate permission and compatible before/after evidence for each case. |