Pre-registration (locked before any forecast)
# PREREG.md — pre-registration of the gold-standard Kalshi forecaster
**Registered 2026-07-28 (America/Chicago), before any `source='ai'` forecast row exists.**
The git history is the proof: this file is committed at Phase 5, and the first production
(`source='ai'`) forecast is not created until Phase 7, after this commit. Any change to the rules
below requires a dated amendment appended to the "Amendments" section AND the owner's sign-off — the
original rules above the amendment line are never edited in place.
This is a paper-only research instrument. Nothing here places real orders. Educational / hypothetical.
---
## The pre-registered rules (copied verbatim from PLAN.md "LOCKED decisions")
1. **The model never sees the market price** in its prompt. We are testing independent skill;
showing the price lets the model echo the crowd and fakes calibration. Log `mkt_p` separately
at call time. (A later "sees-price" variant may be added as a separate scored segment. Not v1.)
2. **Question selection:** status open, `close_time` ≤ 30 days out, `volume_fp` ≥ 500, bid-ask
spread ≤ $0.10, category in: Economics, Financials, Commodities, Climate and Weather, Politics,
Science and Technology, World. Excluded: Sports, Entertainment, Mentions, Crypto, Elections,
Companies, Social, Health. Max **2 markets per event** (for ladders pick the 2 strikes nearest
$0.50 mid = nearest the money). Max **40 new forecasts per night**.
3. **One forecast per market**, made at first qualifying sighting, never revised. The model may
**skip** any question (no informational basis); skips are logged and cost nothing.
4. **Paper trade rule:** fills at the **ask** (not mid). Buy YES if `our_p − yes_ask − fee(yes_ask)
> 0.02`; buy NO if `(1 − our_p) − no_ask − fee(no_ask) > 0.02`. Kalshi fee per $1 contract:
`fee(p) = ceil_to_cent(0.07 · p · (1 − p))`. Flat $10 notional per trade, max 1 trade per event.
5. **Scoring:** Brier of `our_p` vs Brier of `mkt_p` (mid at call time) on identical resolved sets,
segmented by category and by context kind (fred / nws / prices / news / none), plus calibration
buckets. Voided/unresolvable markets are excluded. Only rows with `source='ai'` count
(`source='dev'` = build-time tests, never scored).
6. **Gates (evaluate at ≥150 resolved AI forecasts):** skill gate = our Brier < market Brier;
money gate = paper P&L > 0 net of fees over ≥100 trades. Both pass → owner may fund a real
bankroll ($500–2k, allowed to go to zero; separate authorization, separate build). Either fails
→ kill, write the post-mortem, keep the track record. Interim peeks fine; no action before n.
7. **Prompt discipline:** `PROMPT_VER` constant logged on every forecast; any prompt change bumps
it. LOUD failures only: a failed/malformed LLM call writes a `runs` row and stderr; **never**
silently fall back (market-radar's silent-fallback bug is the cautionary tale).
---
## Exact prompt text (PROMPT_VER = "v1")
The nightly batched call (`brain.py`, `--model sonnet`) emits ONE prompt built from a fixed
preamble plus one block per question. The preamble is **verbatim**:
> You are a calibrated event forecaster. For each question output a probability 0-1 that it
> resolves YES. You have NO market price and must not guess one - reason only from the evidence
> given. If you have no informational basis, set skip=true (this costs nothing and is better than
> a blind guess). Today is {UTC date} (UTC). Output STRICT JSON ONLY: an object mapping each
> ticker to {"p":<0..1>,"why":"<=20 words","skip":<bool>}. No prose outside the JSON.
Each per-question block that follows the preamble contains ONLY: the ticker, the question title,
its sub-title, its resolution rules, and the Phase-2 context pack (external evidence — FRED / NWS /
prices / news — which is itself price-free by construction; `context.py` contains zero Kalshi price
fields). **No market price, bid, ask, or implied probability is ever placed in the prompt.** This is
enforced structurally: `build_prompt()` is a separate pure function and the price fetch lives in
`_fetch_mid()`; a grep proves the string `_dollars` never appears in the prompt path.
If v1 is ever changed, `PROMPT_VER` is bumped (e.g. "v2") and forecasts made under different
versions are scored as separate segments — v1's track record is never retroactively altered.
---
## Gate arithmetic (exactly what `score.py` computes)
Scored set = resolved `forecasts` rows with `source='ai'`, `outcome ∈ {0,1}` (void `-1` and
unresolved `NULL` excluded), and both `our_p` and `mkt_p` populated.
- `brier_ours = mean((our_p − outcome)²)` over the scored set.
- `brier_mkt = mean((mkt_p − outcome)²)` over the identical set (the crowd's mid at call time).
- `edge = brier_mkt − brier_ours` (positive ⇒ we beat the market; lower Brier is better).
- Paper P&L: over settled `paper_trades` (`pnl` not NULL), `total_pnl = Σ pnl`, `n_trades = count`.
Per-trade P&L (LOCKED #4 `$1`-contract math, `contracts = $10 / entry_p`,
`total_fee = contracts · fee(entry_p)`): WIN `pnl = 10·(1/entry_p − 1) − total_fee`;
LOSS `pnl = −10 − total_fee`; VOID `pnl = 0`.
**Decision (only at `n_resolved ≥ 150`):**
- **skill_gate_pass** ⟺ `n_resolved ≥ 150` AND `brier_ours < brier_mkt`.
- **money_gate_pass** ⟺ `n_trades ≥ 100` AND `total_pnl > 0`.
- **Both pass** → the forecaster has earned a real-bankroll funding decision ($500–2k, may go to
zero; separate authorization + separate build). **Either fails** → kill it, write the post-mortem,
keep the documented track record as the durable asset.
- Before `n_resolved = 150`: interim peeks are allowed but **no action is taken** — `score.py`'s
verdict reads "accruing — N of 150 resolved; no decision yet".
---
## Note on the fee example in PLAN.md
PLAN.md's Phase-4 acceptance line ("fee(0.50) on 20 contracts = $0.70") is an arithmetic slip that
contradicts the LOCKED 0.07 coefficient. The **coefficient is unchanged** (`fee(p) =
ceil_to_cent(0.07·p·(1−p))`, i.e. $0.02/contract at p=0.50). The implemented order fee is
`contracts × fee(entry_p)` = 20 × $0.02 = **$0.40** at the example point. This clarification does not
alter any pre-registered rule; it records the correct arithmetic under the LOCKED formula.
---
## Amendments
(none — the rules above are the original registration. Append dated, owner-signed amendments below
this line; never edit the original rules in place.)