Define the prediction
Define the target, time horizon and units. Compare like-for-like prediction tasks.
Loading…

Research · Evaluation framework
Test against a baseline. Repeat the run. Publish the failures and limits with the result.

The question
Compare the same task, with and without added context. Keep model, scenario and evaluation rules fixed.
Define the target, time horizon and units. Compare like-for-like prediction tasks.
Record versions, inputs, calibration and compute. Give the baseline the same evaluation conditions.
Change only the input under test. Record time, coordinates and uncertainty. Include missing and stale signals.

Proposed protocol
Keep training and tuning separate from evaluation. Record the split. Check for leakage.
Repeat with recorded seeds. Test sensing loss, delays and unfamiliar conditions. Report failures.
Show sample counts, distributions and uncertainty. Better prediction does not automatically mean higher throughput or improved safety.

Published research / Scope & provenance
Fleet and decision-path results have separate experiments. Model cards retain each baseline, artifact and boundary.
Identify data or scenarios, evaluation code, versions, hardware, configuration and exclusions.
Separate measurements, synthetic data and simulation. Record permissions, calibration and disclosure limits.
Make the method reviewable. If data is restricted, say what can still be reproduced.

Start with your operation
Bring a question, a dataset or a repeatable scenario. Start without sensitive data.