State is part of the experiment.

A result is hard to interpret if you cannot reconstruct what the system knew when it made a decision.

Market-data projects give me a useful setting for studying realtime software: ingestion, state transitions, event ordering, and recovery. This note concerns offline experiments, not investment performance.

One event, several clocks

Event time, receipt time, and processing time describe different moments. Recording them separately helps distinguish stale data from delayed processing. An experiment should use only information available at the point it claims to simulate.

Make failure cases observable

• Do duplicate or out-of-order events change the final state?
• Can a restart reconstruct the state used in an experiment?
• Are gaps and stale observations recorded explicitly?
• Does the experiment stop when required inputs are missing?

Review the trace before the headline

I’m interested in traces and tests that expose failures in these controls. Claims about strategy performance need their own data, costs, and evaluation assumptions. An architecture diagram or a successful replay does not establish profitability.

The same principle applies to ML systems: make state transitions inspectable, preserve failure evidence, and keep the claim as narrow as the measurement.