walkforwardResearch
Method

Trained on the past, scored on what it had never seen

The rules are written down before the forecast, so they cannot drift afterwards to flatter the result.

What walk-forward scoring is

Train a model on the past. Test it on the next slice of time — data it could not have seen. Step forward and repeat. Every score here is produced that way: the model makes its call the way it would on a Monday morning, before the answer exists.

Walk-forward validation

data the model learned from the slice it was tested on, never seen in training

Train on the past, test on the next slice, step forward, repeat. The model never sees the future.

Pre-registered

  • Target, metric, benchmark and horizon written down first
  • The bar is set while it can still be missed

Published, then scored

  • Forecasts out before the outcome, marks up after
  • Hits and misses alike, and the ledger is append-only

Domain-agnostic

  • One harness scores public finances, capacity and demography
  • Shared machinery is what makes these one body of work

Why backtests are not evidence

A model tested on data it was fitted to will always look good — the exam was written after seeing the answers. Retrospective accuracy is easy to manufacture, and most published claims of it are worthless as evidence.

The standard

The only honest test is the one where the future was still the future

  • Every claim here links to a registered entry, or it is not made
  • That discipline is unglamorous, and it is the whole of the method

How the ledger works

Registration, marking and withdrawal, in the order they happen.

Registration

Before the outcome exists

  • The claim, the benchmark and its value on that day, the data used, the marking dates and the scoring rule, committed as a file
  • The commit is the timestamp, and because the repository is public the push is recorded independently, so the date can be checked without taking our word for it

Marking

Against the number it set out to beat

  • Marks are added by scripts from the source data as it is published
  • The benchmark names the publisher and the vintage, not just the series
  • Marks will be computed with the conventions in the Bank of England’s published forecast-evaluation package, so the arithmetic is not ours to argue about

Afterwards

Nothing is edited or deleted

  • A forecast wrong in construction is withdrawn with a reason and stays on the ledger
  • A change of view is a new entry that supersedes the old one, and both are scored
  • The record counts everything, including withdrawn entries and misses

Status

Registered — no marks yet Open — marks arriving Closed — final mark landed Withdrawn — kept, with a reason

A marked entry also carries the headline word: beat, missed or tied.