What exists, what is being decided, and what is waiting. Check the date. Where the repository disagrees with this page, the repository wins, so fix this page.
Next session
Overwritten at the end of every task (see Backlog). 2026-11-18. B-017 is done (0f761b9; B-014 in 97c8721): untrusted queries can run under a hard memory bound in a worker process. sophia_query::bounded::run_bounded starts sophia-query-worker, which sets RLIMIT_DATA on itself, opens the store read-only (open_read_only, new on all four backends) and runs the query; over the bound the caller gets Error::MemoryBound (bounded). Tested on SQLite, redb, LMDB and DuckDB (tests/bounded.rs). Codex’s review found three bugs (an oversized request, non-UTF-8 paths, a deadline overflow), all reproduced and fixed (Review 2026-11-18 Codex B-017 (external)). Limits: Linux only; redb and DuckDB lock their file, so the worker reads them only while no other process holds them for writing; the raw sql dialect is not routed through it. 391 tests pass. Still waiting from earlier: B-061 (SR-126), B-060 on Daniel’s choice about B-010 (SP-008); SPEC-011’s five questions. Next task: B-018, M3b, the agent server sophia-mcp (size L: the first session splits it into rows and builds the first one). Read Milestones M3b, SR-62, SR-100 (agents use the query API, not backend SQL), SPEC-007 §8, LLMs and Mixed-Language Code, and bounded (untrusted queries go through run_bounded; it needs the worker binary’s path and a store file, and redb/DuckDB cannot be read while the server holds them). Which MCP library, transport, and edit tools are in scope are design choices to put to Daniel first. ../Sophia-mistral is on branch mistral/b014-review (the B-003 state is in its git stash).
As of 2026-11-09.
-
Direction: Sophia is the store; sophialang is its language (S-ADR-0007). The first priority is the Moonkale interface (SPEC-004 Moonkale Interface). The plan is Milestones M0 to M7.
-
Code: M0 and M1 done (2026-10-20):
sophia-hash,sophialang-coreandsophia-store(SQLite and redb), plussophia-benchand thesophiaCLI;- 58 tests pass;
- first benchmarks in Benchmark Results.
M2’s interface is done (2026-10-20):
sophialang-syntax(the Julia and Rust profiles round-trip);- rendering profiles in the database;
sophia-interface(SPEC-004 §9 on both backends);- Moonkale has been sent the revision.
M2 is complete: four backends compared (Database Comparison).
M3a, Sophia’s own query language, is done in its first form (2026-10-20), after Daniel’s query-language proposal (Review 2026-10-20 Query Language (external)):
sophia-query: stratified Datalog over base relations, a Julia-flavoured syntax, content ids, a native executor on all four backends, magic sets, SQL translation (SPEC-007 Sophia Query Language);- Moonkale’s
querydialect, and queries stored, printed and edited as definitions; - B-17 measured (Benchmark Results);
- S-ADR-0010 accepted in amended form (SQLite default, LMDB experimental, no permanent choice yet); S-ADR-0011 accepted; S-ADR-0012 proposed.
M3a stabilised (2026-10-20) after its review (Review 2026-10-20 M3a (external)): cache race fixed, exact canonical form, resource budget, property and differential tests.
B-16 rung 1 done (2026-10-20): Metamath set.mm and iset.mm in Sophia, with real queries (the axioms behind 2 + 2 = 4, per library) and four design corrections learned from real data (Real Code Corpora §4). Daniel: Moonkale, Lenticulum and Sophia are to be thought of together; Lenticulum running in Sophia is a later goal (SR-114, SR-115).
B-16 rung 2 started (2026-10-20): Julia LinearAlgebra through the normaliser shared with Moonkale (identical ids on Moonkale’s fixtures). 69 % of real definitions got an id under v2; Moonkale’s v4 (loops, scopes, varargs, destructuring) raised it to 99 % (Real Code Corpora §5).
v6 in both tools (2026-10-20): Sophia and Moonkale give identical ids on LinearAlgebra (3 435), with module-level references (2 469, 948 across files).
After the real-code review (2026-10-20, Review 2026-10-20 Real Code (external)):
- Metamath checked: Sophia’s “which axioms does T rest on” agrees with metamath.exe on every theorem of set.mm and iset.mm (64 566) (Metamath Overview);
- generic functions are (module, name): a real bug, fixed; agreement with Julia’s own method table on LinearAlgebra 53.7 % → 95.1 %; the check found a bug in the shared normaliser, fixed by Moonkale in v7, which Sophia now uses (3 416 identical ids) (Julia Conformance);
- record kinds enforced (SP-012);
- new sections: Storage and Materialization (finding: the store is about 10× its source and mostly derivable) and Languages (support levels per language);
- AD measured (E-AD-0): producing a derivative costs 10² to 10⁶ runs of it (AD as a Materialization Case).
After the second real-code review (2026-10-20, Review 2026-10-20 Real Code 2 (external)):
- reconstruction before compaction: sources and an ingest manifest are stored; every LinearAlgebra (3 284) and set.mm (98 136) record rebuilds identically from the store alone, tested on four backends;
- a compiled derivative restored in a fresh process (Reactant, bit-identical); Enzyme via Julia’s pkgimage segfaults;
- Mathlib exported and measured: 71 % of its term graph is needed only by proofs; nothing has been kernel-checked locally yet;
- S-ADR-0012 accepted.
- B-15 measured with Moonkale (its
edit_loop, Moonkale26752448on Sophia202e137a): the whole edit loop’s p99 is 2.4–3.5 ms on 10⁵ definitions (target < 50 ms); Moonkale’s 500 ms change-feed polling is what a user would wait on (Benchmark Results).
After Moonkale’s desktop check (2026-10-21), four findings on Sophia’s side, all fixed: Julia records and ingest manifests now print (the source text a definition was read from, through a new
source-spanfact; re-ingest needed for old stores; Moonkale re-ingested LinearAlgebra and all 945 names print, pinned at4c2aaab3); listing a namespace is a skip scan (root 32 → 0.1 ms on 10⁵ names); parse errors name tokens as written;letandmatchfollow the Rust profile’s braces. Found on the way: 7 % ofsophia gendefinitions print but do not parse back (SP-015 Stored definitions deeper than the parse limit cannot be edited).Moonkale ADR-0029 (accepted by Daniel, 2026-10-23): Sophia becomes Moonkale’s code-graph store, one derived store per folder, files the truth. Sophia’s answer is SPEC-009 Moonkale Folder Store, accepted by Daniel on 2026-10-24 with SP-015 (revised the same day: no depth limit, depth must cost memory not stack, SR-145) and a
v0.1.0tag after the gates; implemented on 2026-10-24:replace_files, the derived mode,text_source; SP-015 step 1a (any depth on native targets); sophia-ingest on Moonkale’s shared naming (09ce6d9c). SP-015 resolved: no pass recurses on the call stack (parser and elaborator included), any depth on native and wasm (Node, Firefox), printing and parsing linear;stackerremoved.v0.1.0tagged on 2026-10-24 atc3c48e02(after the fact-provenance fix found by Moonkale’s Codex) (finished records from Moonkale, per-file replacement in one transaction, an enforced derived mode, SQLite for wasm, a first tag). The whole workspace except the ingest compiles forwasm32-unknown-unknown; nothing has run in a browser yet. Afterv0.1.0, Moonkale’s history replay (P-231) found generic memberships drifting from a fresh build: repeated members were subtracted more than once, and names changing between generic function and definition left orphan memberships or lost live ones. Both are fixed and tested on all four backends, and each step is compared with a fresh build (folder.md). Moonkale reports 0 differences at every checkpoint after the fix.v0.1.1tagged on 2026-10-24. Folder store garbage measured (Folder Store Garbage): about 60 % of a replayed store is unreachable, mostly old source texts. Collection implemented (SR-146): the source-of-truth setting decides. In derived stores, where the files are the truth,replace_filescollects what no current file reaches, andcollect_foldersweeps old stores. On Moonkale’s replay the sweep leaves exactly the fresh build’s records: 39.2 MB against 91.2 MB. Where the store is the truth, history is kept. Moonkale’s replays with collection: 0 differences, no measurable cost per batch, 41.5 MB against 92.6 MB.Interface::allon many namespaces: 2.1 s → 0.12 s (namespace children were listed and discarded; two SQLite prefix reads scanned their tables, Moonkale P-232).v0.1.2tagged on 2026-10-24 (collection by source of truth, theallfixes). A Codex review of the changes sincev0.1.0found three things, all checked and fixed: facts kept records that a fresh build would not have, which made a dropped fact leave garbage; and one file listing a generic function twice could retire another file’s method.v0.1.3tagged on 2026-10-25 with these fixes; Moonkale pinned it (48c188c9), both replays clean. Research questions section added (2026-10-25, SR-147): Research Questions, RQ-001 to RQ-003. Writing them found two things: a recursive group’s ids respect automorphisms but not bisimilarity (RQ-001, now a test), and collection keeps a record a file provided but nothing reaches, so the store can differ from a fresh build (RQ-002). Daniel chose to reject them:replace_filesrefuses such a file withUnreachedRecord(2026-10-26). Moonkale’s replays ondcf8660: 0 errors, 0 differences, record counts unchanged (5 940 and 1 770), its 75 tests pass (Moonkale’s report).v0.1.4tagged on 2026-10-26. Where the truth lives (Moonkale ADR-0030, Daniel, 2026-10-26; SR-148): legacy folders keep files as the truth, everything else has the store as the truth. Sophia’s part is SPEC-010 Store-Truth Folders and Generated Files, accepted by Daniel on 2026-10-26, implemented on 2026-10-28 exceptpropose_textand the SR-149 text export: file tokens onreplace_files, an in-place switch of the truth, generated files and outside edits as proposals; Moonkale normalises foreign code, as in SPEC-009. Acceptance 2 passes (Moonkale’s report, 2026-10-28): Moonkale’s history replay on08e6ffb, switched to store truth halfway, checks the replay against a twin store built fresh and switched at the same commit. At every checkpoint after the switch it also compares the replay with a fresh build: 0 differences for both Lenticulum.jl (130 commits) and Moonkale (471), 0 errors, and no token conflicts in 65 and 236 commits written withreplace_files_checked. Keeping history, the stores grow to 2 473 and 7 840 records.give_truthremoves 703 and 1 900 of them, leaving exactly a fresh build’s 1 770 and 5 940 records. Command and raw reports are on Moonkale’s branchreplay-truth(08e23152),packages/services/index/measurements/history-2026-10-28-*-spec010.jsonl.v0.1.5tagged on 2026-10-28 (SPEC-010 acceptance 1 and 2; acceptance 3 waits forpropose_text). Codex’s review of SPEC-010 (2026-10-28) found six bugs, all checked and fixed, each with a test that fails before the fix except the export race: generic names and a group’s other names escapedLaidOut;give_truthkept names no file has (nowNotInFiles); export could pair a new token with old text; a recursive member’s second name got a wrong span (nowaliases); proposals did not list generic functions created or removed.fetch_texthad the same token race and now reads the token first.v0.1.6tagged on 2026-10-28 with these fixes. Moonkale’s replay onv0.1.6: the same counts and 0 differences, 0 errors, no token conflicts; Moonkale master6d1b1eb3pinsv0.1.6(Moonkale’s report, 2026-10-29; reportshistory-2026-10-28-*-v0.1.6.jsonl). A second Codex review (2026-10-29) of the v0.1.6 fixes found five things. Three were true and are fixed: a rule could add a method to a laid-out generic function,give_truthchecked owner rows but not targets, and export’s retry missed changes outside the exported names. One was not reachable (now tested). One was a cost, removed with thegive_truthfix. Text tokens do not cover the names a text references.v0.1.7tagged on 2026-10-29 with these fixes. Moonkale’s replay onv0.1.7: the same counts, 0 differences, 0 errors; Moonkale mastercaa2fcaepinsv0.1.7(Moonkale’s report, 2026-10-29; reportshistory-2026-10-29-*-v0.1.7.jsonl). Daniel chose to warn instead (SR-150, option C): a save whose reference now points at other content than at the last save warns inWritten.warnings, implemented and tested on 2026-10-29.v0.1.8tagged on 2026-10-29. Moonkale master36572ccepinsv0.1.8. Annotations (2026-10-30): Moonkale’s ADR-0031 (proposed) puts comments and proofs in Sophia annotations for store-truth folders and in its jsonl elsewhere. Sophia’s hook for moving them with the truth is drafted as SPEC-010 Store-Truth Folders and Generated Files §7. Daniel accepted ADR-0031 and §7 on 2026-10-30 (SR-151); not built yet. His remark that one result can be both a costly computation and a claim is answered by a draft, SPEC-001 Sophia Store v0 §8a: the cost of recomputing is a second axis, and a per-user cache keeps content-only facts. Daniel answered its questions on 2026-10-31 (SR-152): the cache policy is a setting, chosen by experiment; local and per user at first. The experiment is planned, and nothing is built. Gemini’s review (2026-11-07), triaged 2026-11-08 (Review 2026-11-07 Gemini (external)): of its Sophia findings, 7 were right (2 already known, SP-008 and SP-014) and 9 were wrong or out of date. Its cross-repo X-BUG-01 was right. Moonkale’s normaliser gave symmetric members of a recursive component one id but kept the full size, so a 2-member component was stored with 1 member. Moonkale fixed it in06fd112c(normaliser 8). Sophia moved its pin there: all golden rows reproduce, and LinearAlgebra re-ingested gives the same names and counts, with no mismatched components. On Daniel’s go (SR-153), Sophia now:- refuses incomplete components on write;
- has
Store::verifyreport Moonkale records asForeigninstead of failing them; - reads annotations by primary key;
- takes a name’s source text from its owning file;
- waits on a
Condvarfor the DuckDB time budget (2.1 → 0.69 ms per query); - limits DuckDB queries to the views, checked on the parse tree.
333 tests pass, and wasm passes under Node.
v0.1.9tagged on 2026-11-08 (c19862a). Moonkale master639f8d0cpins it; its replays: Lenticulum.jl 130 commits and Moonkale 471, 0 findings, 0 errors each (Moonkale’s report;history-2026-11-08-*-v0.1.9.jsonl, summary lines checked). A read-only review of v0.1.9 by Mistral (throughvibe, 2026-11-08, Review 2026-11-08 Mistral v0.1.9 (external)) found five real bugs and no way past the view check. All five are fixed, four with a test that fails without its fix: the DuckDB timer outlived an error path; the row cap overflowed; anymoonkale-normaliser skipped re-hashing;text_sourcesplit backquoted operators; and a batch was checked only for its first claimed size.v0.1.10tagged on 2026-11-08 (499e5f3). Moonkale could not pin it: its folder store writesmoonkale-wp66-imports-v1@8, which the narrowed check did not count as foreign, so every definition verifiedInvalid. Fixed inv0.1.11, tagged on 2026-11-08 (cc9e426). Moonkale’s replays onv0.1.11(store truth from halfway): Lenticulum.jl 130 commits and Moonkale 471, every checkpoint with 0 differences, 0 errors, 0 findings.give_truthremoves 703 and 1 900 records, leaving a fresh build’s 1 770 and 5 940, the same as onv0.1.9(Moonkale’s report;history-2026-11-08-*-v0.1.11.jsonlin Moonkale’sMoonkale-dev-claudecheckout, summary lines and difference lists checked). The pin reaches Moonkale master with its CI fix. Scope (2026-11-09, SR-154): Daniel restated sophialang as one language with three parts: programming, queries, and a language for type systems. Three external reviews (Review 2026-11-09 Scope (external)) agree that it is still in scope, sequenced rather than reduced. The third part is specified nowhere, and is now RQ-004 A Language for Defining Type Systems. SP-014, option 2 (2026-11-09): a miss that stores a query result now removes the results of older snapshots. Mistral wrote it throughvibewith file tools only; Claude built, tested and reviewed it. Option 3, an in-memory cache by default, is still open. Fable’s review (2026-11-10), triaged in Review 2026-11-10 Fable (external). Of its Sophia findings, most were right. The store accepts a record whose bytes do not match its id (S-1). Raw SQL has no memory limit (S-3). One file may bind a name twice (S-4).set_settingscan write the store’s own keys (S-7). There is no CI. On Daniel’s go, the small fixes and the gates are done inv0.1.12: S-2, S-3, S-7, S-8, S-12, S-13, S-14, a CI workflow, clippy clean under-D warnings,rustfmt.toml, and a consumer test with Moonkale’s normaliser names.v0.1.12tagged on 2026-11-10 (a28bf5d). The first CI run passed in 25½ min. Moonkale mastercecdfcd4pins it, and both replays matchv0.1.11: 0 differences, 0 errors, no counter orInvalidBatcherror.give_truthremoves 703 and 1 900 records (Moonkale’s report;history-2026-11-11-*-v0.1.12.jsonl, summary lines and difference lists checked). The open decisions are listed in the note: S-1, S-4, S-5, X-1 and SP-008. Moonkale’s findings went to the Moonkale agent. Sharing a store-truth folder: a text export for now (SR-149), its own spec to come. Also fixed for Moonkale:Interface::allon large stores (P-229: no term recursion, no quadratic edge check).Next (SR-124 to SR-126): the Semantic Boundary Experiments (small, each with an oracle), and B-15 (Moonkale edits) as soon as Moonkale is on the current revision. LuxCore is dropped as a step; full Lux.jl after the AD and Reactant experiments, Lenticulum after that. No storage-layout change without Daniel’s agreement.
-
Specifications: ten (SPEC-001 to SPEC-010) (Specifications).
-
Decisions: Decisions. Accepted: S-ADR-0004 (Rust only), S-ADR-0007 (store vs language), S-ADR-0008 (sophialang first), S-ADR-0009 (relations tracked), S-ADR-0010 (amended), S-ADR-0011 (own query algebra).
-
Requirements: Requirements: SR-01 to SR-29 and SR-51 to SR-74 from Daniel; SR-30 to SR-50 from Moonkale; SR-75 to SR-89 from the first external review; SR-90 to SR-106 from Daniel’s query-language proposal; SR-107 to SR-113 from the review of M3a; SR-114 to SR-118 (the three projects); SR-119 to SR-132 from Daniel’s storage and language requests and the real-code review.
-
Daniel’s answers of 2026-10-20 (second message): push; delete the Julia skeleton (done); M2 starts with the interface crate. Relations: S-ADR-0009 proposed (tracked), awaiting confirmation.
-
Daniel’s answers so far:
- Q1: keep both Unison and Sophia.
- Q2: Julia-flavoured, display as settings stored in the database.
- Q3: Rust only.
- 2026-10-20: the language principles (SR-51 to SR-61), research questions (SR-62 to SR-74), and the go-ahead to implement.
-
Research notes answering the questions of 2026-10-20:
-
Benchmarks: Benchmark Plan; Benchmark Results.
-
External review (2026-10-20): triaged in Review 2026-10-20 (external). It found one real bug (symmetric recursion), which is fixed; and led to SR-75 to SR-89, SPEC-006, corrections to SPEC-001 to SPEC-004 and the theory notes, edit tokens, world snapshots, and the P0 gate in Milestones.
-
Coordination: the Moonkale agent has the 2026-10-17 interface. It is to be told about the split into SPEC-001 to SPEC-005.