measurement plan

The measurements that decide Storage and Materialization’s open questions. Each has a hypothesis, a method and a failure criterion. They extend Benchmark Plan (B-06 compact encoding, B-09 snapshots, B-16 real code, B-17 queries). 2026-10-20.

Sources: Current Inventory (the baseline numbers); Materialization Policy.

#QuestionHypothesisMethodFails ifStatus
M-1How much smaller is a store that keeps only authority, ids, kinds and one copy of the edges?set.mm under 70 MB (from 567 MiB); LinearAlgebra under 2 MB (from 17.7 MB)a second SQLite layout behind the same Backend trait; ingest both corpora; compare dbstatthe saving is under 3×baseline measured (Current Inventory)
M-2What does rebuilding canonical bytes cost, per definition and per file?below 1 ms per definition for Julia (parse the file once, normalise one definition)time module::ingest per file and per definition; time the WP-66 encoding alonerebuild for one Moonkale “show canonical form” request exceeds 50 ms (an interactive budget)open
M-3Which derived objects are requested how often, at what cost?a few query shapes dominate; most results are used oncelog every run_cached call: query id, snapshot, rows, steps, bytes, time, hit/miss— (it is a measurement)open; needs a Moonkale workload (B-15)
M-4For evidence, how does finding cost compare to checking cost?Metamath: checking is seconds, finding is human worktime verify proof *; later the same for an SMT certificate—Metamath measured: metamath.exe checks all 48 083 set.mm proofs in 2.85 s; open for SMT
M-5How fast does the log grow under editing, and what does compaction save?ingest dominates the log today (5 % of set.mm’s store); edits are smallB-15 edit loop; count log bytes per editlog growth per edit above 1 KBopen
M-6When is a compiled derivative worth keeping?small functions: compile ≫ run but both are milliseconds, so cache only; compiled models (Reactant/XLA): compile is seconds to minutes, persistE-AD-0 and its successors (AD as a Materialization Case)compile time does not dominate run time for any backendE-AD-0 done, hypothesis partly refuted: even for toy functions producing D(f) takes 0.1–5 s (not milliseconds), 10²–10⁶ runs’ worth; so persisting across sessions pays for small functions too. Reactant (CPU): 0.7–1.3 s to compile, 2.5 µs to run; models still to measure (E-7)
M-7How much of a folder store is garbage after real history?most of it is old source textsfolder_garbage on Moonkale’s replays (471 and 130 commits)— (it is a measurement)measured 2026-10-24 (Folder Store Garbage): Moonkale 91.2 MB replayed, 38.4 MB after removing non-live records, 36.2 MB fresh; source texts are 73 % of the garbage

A measurement that refutes its hypothesis is a result, not a failure: it goes into Benchmark Results and the policy changes.