Which real code B-16 uses, in what order, and why. Daniel, 2026-10-20: “Moonkale was meant as an editor for Lenticulum.jl. Now Sophia is somewhat intermingled with it, so it would be a good idea to think the three things together. Having Lenticulum run in Sophia would be a goal, but a later one; there should be easier Julia packages.” He suggested BLAS-like linear algebra (“linear algebra might be quite fundamental”), Metamath, and then Lean for theorem libraries. Requirements: SR-114 to SR-118.
Sources:
~/Code/CategoryTheory/Lenticulum.jl(about 17 000 lines of Julia:Lenticulumplus five packages underlib/); Moonkale’s WP-66 normaliser (packages/services/index/src/content/, tree-sitter grammars for Rust, Julia and Python viaarborium); metamath/set.mm (GitHub). code:crates/sophia-ingest(Metamath first).
1. The three projects as one system
| Role | What it needs from the others | |
|---|---|---|
| Lenticulum.jl | the ML library: the program to be edited, stored and eventually run | nothing yet; later, to be stored and run from Sophia |
| Moonkale | the editor and graph view, originally built for Lenticulum | a code store with identity and queries (Sophia, SPEC-004) |
| Sophia | the store and, later, the language and compiler | real code to store: Julia first, because of Lenticulum |
Consequence for Sophia: Julia is the first foreign language that matters.
- Moonkale already computes WP-66 content ids for Julia, Rust and Python definitions with tree-sitter.
- Sophia’s Julia ingestion should use the same normal form, ideally the same code (SR-116), so that a definition has one id in both.
2. The ladder
Each rung is a benchmark (B-16) and an experiment with a written result (SR-73).
| # | Corpus | Why | What it tests | Needs |
|---|---|---|---|---|
| 1 | Metamath set.mm (≈ 45 000 theorems; also iset.mm) | A real theorem library with a trivially parseable format and a huge, real dependency graph. Daniel named it | content addressing of a real library (renaming invariance, duplicate theorems); ingest at 10⁵-scale; deep transitive queries (“which axioms does 2p2e4 use?“) | a Metamath frontend (sophia-ingest), no external tools |
| 2 | Julia LinearAlgebra (generic parts), then Octavian.jl (BLAS written in Julia) | “Linear algebra is fundamental”; pure Julia; multiple dispatch everywhere | methods and generic functions (nominal entities, SP-006), dispatch-heavy reference graphs | a Julia frontend on the WP-66 normal form |
| 3 | Rust matrixmultiply (a pure-Rust gemm) | the same algorithm as rung 2 in another language | cross-language comparison: does Sophia find the same structure in Rust and Julia? (SR-73, the reuse dream) | a Rust frontend on WP-66 |
| 4 | the goal (SR-115) | a real project of the user’s own, edited in Moonkale through Sophia | rung 2’s frontend, plus macros and modules | |
| 5 | Lean 4 / Mathlib (via lean4export) | a dependently typed theorem library | terms with real dependent types; Mathlib’s scale | a Lean toolchain (not installed here) and an export reader |
Rung 1 does not contradict S-ADR-0008 Sophialang before Metamath or Lisp. That decision is about which language Sophia implements first. Here Metamath is data to store and query, through a small frontend.
3. Rung 1: Metamath into Sophia
Each assertion becomes one record of the language metamath:
$astatements: syntax, axiom or definition;$pstatements: theorems.
The design, as built after four corrections (§4):
- A statement is the identity: mandatory hypotheses in order, disjoint-variable pairs, typecode and math string. Variables are numbered by first occurrence; labels and variable names are display data. Its kind is
statementorsyntax. Statements are namedmm.<db>.<label>. - A proof is evidence: a separate record named
mmproof.<db>.<label>. It references the proofs of the theorems it uses (the justification the author chose) and the statements of the axioms and definitions it uses. Aproofannotation joins it to the statement it proves. - A library is a world (SPEC-006 §1): three nominal collections,
mmlib.<db>.proofs,.axiomsand.definitions.
“Which axioms does 2p2e4 rest on, in set.mm” is then a plain walk:
query rests_on(proof::Id, axioms::Text)
below(x) = refs(proof, x)
below(y) = below(x) && def(x, :proof) && refs(x, y)
select n where below(s) && def(s, :statement) && member(c, s) && name(axioms, c) && name(n, s)
end4. Rung 1: results (2026-10-20)
What the real library taught, in order. Each point was found by running a real query, and each changed the design:
- Proofs are not identity. The first normal form hashed each theorem’s proof into its id. Then 12 059 of the 13 138 labels that set.mm and iset.mm share looked different, although the statements are equal: iset.mm proves them intuitionistically. Sophia’s own principle (SPEC-002 §3) says a proof is evidence, not identity. With statements as identity, 12 927 of 13 138 are shared, and the other 211 really differ.
- “Proves” is not “uses”. A proof referenced both the statement it proves and the statements it uses, as plain references. The query “the proof of s” then matched every proof that uses s: tens of thousands for
ax-mp, and over the memory budget. Now aproofannotation carries “proves” (SPEC-001 §8). This is the review’s point about typed relations (TypeQL’s roles), met in real data. - Being an axiom belongs to the library, not to the statement. The record kind (axiom, theorem) was first-writer-wins, so set.mm theorems that state exactly what an axiom states were labelled axioms. Kinds now follow from content; roles are library collections.
- A step must name its justification. With statements as identity, a proof step that referenced a statement lost which justification the author used. set.mm’s personal sections prove
ax-1from other axiom systems, and the walk followed those proofs. Now a step references the used theorem’s proof, so a proof’s identity covers its whole chain. Identical proofs between the libraries drop to 121, which is honest: their foundations differ.
The answers.
- In set.mm, 2 + 2 = 4 rests on 15 axioms:
- logic:
ax-mp,ax-1toax-9andax-gen, including classicalax-3; - set theory:
ax-ext; - complex numbers:
ax-1cn,ax-addcl,ax-addass.
metamath.exe(2026-10-20, SR-128): the same 15 axioms and 29 definitions. See below. - logic:
- In iset.mm, the same statement rests on intuitionistic axioms (
ax-ia1,ax-io, …) and not onax-3. - The fundamental theorem of algebra rests on 46 set.mm axioms.
B-16 on set.mm (51 159 assertions, 48 083 proofs, 98 136 records; median of 5):
| backend | assertions | proofs | records | parse | ingest (parse + store) | size | queries (median of 5) |
|---|---|---|---|---|---|---|---|
| sqlite | 51159 | 48083 | 98136 | 2.25s | 15.26s | 564 MiB | axioms behind 2p2e4: 15 rows, 3.87 ms; axioms behind fta (fundamental thm of algebra): 46 rows, 79.34 ms; proofs using ax-mp directly: 5361 rows, 14.53 ms; count by kind: 3 rows, 97.06 ms |
| redb | 51159 | 48083 | 98136 | 2.17s | 10.91s | 1028 MiB | axioms behind 2p2e4: 15 rows, 2.42 ms; axioms behind fta (fundamental thm of algebra): 46 rows, 46.53 ms; proofs using ax-mp directly: 5361 rows, 13.53 ms; count by kind: 3 rows, 66.98 ms |
| lmdb | 51159 | 48083 | 98136 | 2.17s | 6.21s | 791 MiB | axioms behind 2p2e4: 15 rows, 1.85 ms; axioms behind fta (fundamental thm of algebra): 46 rows, 37.14 ms; proofs using ax-mp directly: 5361 rows, 10.04 ms; count by kind: 3 rows, 52.63 ms |
Two weaknesses found (SP-011 Reference edges dominate storage on real code):
- The store is large: 564 MiB on SQLite for a 52 MB source. About 1.5 M reference edges (about 31 per proof, against 3 in the synthetic generator) take 42 % of the database, stored twice with 32-byte ids, plus a third copy on each record. H3 (“local ids save little”) was refuted on synthetic code; on real code it is the main cost.
- The planner guessed selectivity wrongly twice: a lookup by kind, and by collection, is wide. It now knows that, and magic sets also apply to negated calls (SPEC-007 §6). Cost-based planning is the real fix.
Cross-check against metamath.exe (2026-10-20). The external review asked for it before trusting the dependency analysis. sophia-mm-check (sophia-mm-check) builds no shared code with metamath.exe (built from source, commit 9898f5d) and compares “which axioms and definitions does T rest on”, by statement id:
- set.mm, 302 theorems (incl.
2p2e4,fta): all agree (8 503 axiom and 24 471 definition dependencies); - set.mm, 3 000 theorems: all agree (83 671 and 242 975);
- iset.mm, 1 001 theorems: all agree (35 676 and 48 995);
- all theorems of both libraries: all agree — set.mm 48 083 of 48 083 (1 346 571 axiom and 3 934 015 definition dependencies), iset.mm 16 483 of 16 483 (587 185 and 802 318). Sophia 1 139 s and 184 s for the 96 166 and 32 966 queries; metamath.exe 913 s and 142 s. Six theorems prove syntax rather than
|-statements (weq,wel,bj-0,sn-wcdeq;weq,welin iset.mm); for those metamath.exe also lists the syntax constructor they are built from, which the checker now counts separately.
This checks the dependency walk, not the proofs: Sophia does not verify proof steps. metamath.exe verifies all 48 083 set.mm proofs in 2.85 s, which makes it the obvious first checker for trust records (Metamath Overview).
Next rungs: a Julia frontend on Moonkale’s WP-66 normal form (rung 2).
Agreed with the Moonkale agent (2026-10-20, SR-116). Moonkale extracts its normaliser into a small public crate, moonkale-content-id (packages/services/content-id in MathStruct/Moonkale):
- contents: the per-language normal forms, the SCC canonicalisation and hashing, the limits and
NORMALIZER_VERSION; - dependencies: only the arborium tree-sitter grammars and blake3;
- output:
[u8; 32]digests, byte-identical to Moonkale’s ids today; - input: a parsed definition, plus a map from byte ranges to opaque target keys and their digests.
5. Rung 2: Julia LinearAlgebra (2026-10-20)
Update, normaliser v4 (Moonkale f09b3c40). Moonkale fixed both reported points:
- the self-edge is gone (v3);
- Rust struct expressions are uses (v3);
- every Julia local scope now has its own frame, and loops, varargs, destructuring and anonymous parameters are supported (v4).
Sophia’s resolver matches Moonkale’s on all golden fixtures, including the new scopes.jl, with no exceptions left. LinearAlgebra: 3 947 of about 3 983 definitions get ids (99 %), with 36 refused (nested named functions, a few syntax errors). 894 generic functions with 3 484 methods, and 2 012 same-file references. The numbers below are from v2 and are kept as the record of why v4 was needed.
sophia ingest-code julia LinearAlgebra <stdlib>/LinearAlgebra/src, through the shared crate (crates/sophia-ingest/src/code.rs).
Same ids as Moonkale.
- On Moonkale’s golden fixtures, with Moonkale’s reference spans, every digest is identical in Julia, Rust and Python.
- With Sophia’s own resolver, one difference remains: Rust
Point { … }is a use for Sophia, not for Moonkale. - Moonkale’s Julia struct self-edge turned out to enter the id (the agent had thought not). Sophia mirrors it.
- Both points are reported to the Moonkale agent.
Coverage of real Julia. Of about 3 980 definitions in 34 files:
- 2 758 get an id (69 %);
- 1 225 are refused by the WP-66 v2 normal form, which conservatively rejects constructs whose scoping it does not model:
forloops (399) and anonymous functions (64);- about 729 for binding forms: varargs (
f(A...)), tuple and named-tuple destructuring (m, n = size(A)), anonymous typed parameters (::Type{T}); - a few
do,let,catchand comprehensions.
What is stored.
- Records: 2 300 distinct ones. 458 definitions are exact duplicates of others: the same forwarding method written for several types.
- Generic functions: functions are grouped by name into 597 generic functions with 2 303 methods.
*has 103 methods,/71,\54. This is SP-006’s model, with the method table as a nominal relation. - Shared bodies: 134 method bodies are members of two or more generic functions (for example
hpmv!andspmv!). The function’s own name is not part of the content, so identical bodies under different names are one record, the Unison idea. - References: 913 same-file references. Cross-file references (one package spread over
included files) are not resolved yet, as in Moonkale.
v5 in both tools (2026-10-20). Sophia now uses the crate’s module resolver and reproduces Moonkale’s golden module data exactly. On LinearAlgebra both tools report 3 435 ids and 2 199 references (867 across files). Methods record their calls to generic functions as nominal uses (1 187), after filtering three kinds of false generic calls found here (SPEC-008 §6).
Module-level resolution (2026-10-20, SPEC-008 Module-Level Julia Resolution).
- Per-file resolution is not only incomplete for Julia; it is partly wrong. About 1 940 per-file references in LinearAlgebra link a call to one method of a generic function, or a field
x.nameto a definition of that name. - The module resolver makes 2 301 references, 880 of them across files.
- Proposed to Moonkale as normaliser v5.
Generic functions keyed by module (2026-10-20). Methods were grouped by bare name, which merged LinearAlgebra.dot with BLAS.dot and filed 898 methods of Base’s functions as LinearAlgebra’s own. Now a generic function is (module path, name), and methods written Base.show(…) or after import Base: show join the function they extend. Against Julia’s own method table, the right owner for 94.5 % of methods (53.7 % before): Julia Conformance. The check also found units inside macro calls; Moonkale fixed that in v7, which Sophia now pins: 3 416 identical ids, 798 generic functions (123 of them other modules’ functions it extends) with 2 907 methods, 95.1 % agreement.
The caveat: ids also depend on reference resolution. So “one id in both” needs the same resolver too. The plan has two steps:
- Sophia uses the shared normaliser with its own resolver. Golden fixtures from Moonkale show where the two diverge, and the differences get recorded.
- Moonkale moves its Julia resolver into the crate behind a plain-data interface, once Sophia ingests Julia.