Crate root of
sophia-interface: what Moonkale’ssources/sophiaadapter wraps. Source:sophia_interface.rs.Sources: SPEC-004 Moonkale Interface; Moonkale
packages/core/src/source/. code:sophia_interface.rs, edit, profiles
Shape
Interface<B> is a store opened for one reader; the reader id is given at open (SPEC-004 §3a). Its public types are:
Key:Content(digest)→ Moonkale’sNodeId::from_content;Derived(s)→NodeId::derive(source, s);NodeView,EdgeView,Subgraph;Direction,Change,Risk;Written,EditError, andDiagnosticwith byte spans.
None of them is a Moonkale, driver or async type.
Operations, mapped to Moonkale’s Source
| Moonkale | Interface |
|---|---|
Query::Node | node(key) |
Query::Children | children(key): sub-namespaces, then names, each sorted; the root also holds sophia:Settings. A skip scan over names_from (below) |
Query::Neighbours | neighbours(key, depth, dir, limit) |
Query::All | all(limit): namespaces, names, definitions and named nominal entities, with Contains, Names and the dependency edges among them; truncated when the cap cuts |
Query::Text { "sql" } | sql(query, limits) (SQLite): table plus subgraph |
fetch_text | fetch_text(key): names print their whole recursive group in the reader’s profile; definitions print read-only; settings print the reader’s profile; records that are not sophialang print as described below |
apply(WriteText) | write_text(key, expected, text) |
apply(CreateText) | create_text(ns, name, text) |
apply(CreateDir) | create_dir(ns, name) |
apply(Rename) | rename(key, to) (/ ↔ .) |
apply(Delete) | delete(key) |
changes_since | changes_since(seq, limit), with the keys affected and reprint_all for profile changes; a collected record (remove-definition, SR-146) affects its content key; a removed fact (fact with only an old id) affects the fact’s id, as an added one does (Fable S-11, 2026-11-18; a_removed_fact_reaches_the_feed_with_its_key) |
classify | Interface::classify(dialect) |
Edit tokens
fetch_text returns, as a name’s version, the edit token from text_token(path): a hash of every name the text covers, with versions and targets (SPEC-004 §3b). write_text requires the same token. The profile is deliberately not part of it, because the parser reads every editable profile.
Tests
tests/acceptance.rs covers SPEC-004 §9 item by item, on SQLite and redb (11 tests):
- browsing, including relations as nominal entities with
MethodOfmembers; - editing, including a no-op re-save, a parse error, a conflict, and an edit inside a recursive group;
- a rename in the change feed;
- switching profiles (reprint, edit in Rust, another reader still sees Julia);
- creating definitions, a relation and a rule from text;
sql.
Difficulties met
- A deadlock:
neighbourscalledchildreninside an open read, and SQLite’s connection mutex is not reentrant. Fixed withchildren_in(tx, …). Rule kept for this crate: never call a public method from inside astore.readclosure. - Not atomic across steps: an edit is several store transactions (add records, update names, attach relation members). A failure halfway leaves a consistent but partial state, never a corrupt one. A single-transaction batch API is a later improvement (SP-008 Edits span several transactions).
The query dialect (2026-10-20, SPEC-007 Sophia Query Language §8): query(text, limits) runs Sophia’s own query language on any backend and returns the table (ids as hex text) plus a node for every id cell. classify("query") is ReadOnly. A stored query’s name prints in the query syntax (print_target).
Records that are not sophialang (2026-10-21)
Moonkale’s desktop check found that fetch_text failed for every Julia record of an ingested package (146 of 945 names in LinearAlgebra: structs, abstract types, consts) and for the ingest manifest: print_target decoded every record as sophialang. Now foreign_text prints:
- a source file (
lang = source): its bytes, after a#header; - a definition ingested from another language: a
#header (language, kind, id, normaliser, file and byte range) and the source text it was read from, through itssource-spanfact (types). One id can have several spans, because the name is not part of a definition’s identity (struct LAPACKException … endandstruct RankDeficientException … endare one record in LinearAlgebra); the span whose text contains the printed name is chosen; - anything else (an ingest manifest, a Metamath statement, a Julia record ingested before spans were recorded): a
#summary with display, size and references.
Saving such a text parses it as sophialang and fails or translates; there is no Julia write path. Stores ingested before 2026-10-21 have no spans: re-ingest the package.
Listing (2026-10-21)
children lists a namespace’s direct children with a skip scan: names in batches of 512 along the name index (names_from), and on meeting a sub-namespace, a jump past it. The cost grows with the number of children, not with the number of names below them. Nodes are built from the scanned entries, without a second lookup per name. Measured with examples/listing_timing.rs on a copy of B-15’s 100 000-definition store (SQLite, best of 5, warm):
| Call | Before | After |
|---|---|---|
children(ns:) (2 nodes) | 32 ms | 0.1 ms |
children(ns:gen) (100 000 names) | 133 ms | 75 ms |
all(100000) | 418 ms | 326 ms |
Moonkale first measured 0.9 s, 0.9 s and 3.3 s through its adapter; that was a cold page cache (the 680 MB file evicted). Warm, through the adapter (Moonkale’s examples/browse.rs, reported by the Moonkale agent): root 44 ms → 0.3–1 ms, ns:gen 270 → 240 ms, all 572 → 490–500 ms. The adapter adds about 110 ms per 100 000 nodes (conversion, id-to-key map, a mutex lock per node), on Moonkale’s side.
tests/acceptance.rs now also covers listing around namespace boundaries (a.b, a.b!, a.b.c, a.`b.c` , a.b0, ab.x), foreign records printing, and diagnostics; 45 tests on four backends.
Many namespaces (2026-10-24). Moonkale saw All(1000) take about 6.2 s on its own folder store. That store has 481 files and 5 569 names in deep module paths. On a collected copy, all(100000) took 2.1 s in release. Of that, 1.85 s went to namespace keys: the edge loop asked each namespace for its out-edges, which lists its children (children_in). Those are all Contains edges, which all draws itself and the loop then discards. The loop now skips namespaces: 2127 → 280 ms (listing_timing … rs 3, best of 3). The output is the same: 16 427 nodes, 23 154 edges, and an equal digest of the sorted nodes and edges, from a scratch example run before and after the change. Measured by temporarily timing all’s phases; that timing was not committed. Then two SQLite table scans found by Moonkale’s Codex (P-232) were turned into index seeks, meta_prefix and names_under (sqlite): 280 → 120 ms, same output.
Deep definitions and large graphs (2026-10-23)
Moonkale’s coarse graph query called all(2 000 000) on a copy of B-15’s 100 000-name store, and a Tokio blocking worker (2 MiB stack) overflowed (Moonkale P-229). Two causes, both fixed:
- Recursion per nesting level:
edges_atdecoded each definition and walked its term to findCalls. Generated definitions nest up to 700 levels (encoded fields); decoding, walking and dropping the term recurse once per level. Reproduced: in a debug build, decoding and walking every record on a 2 MiB thread overflows; in release it does not.Callsnow come fromencode::called_refs, a scan of the bytes with an explicit stack. - A quadratic edge check:
allde-duplicated edges withVec::contains. With 739 273 edges it did not finish in several minutes. Now aHashSet(PropandEdgeViewderiveEqandHash).
all(2 000 000) on that store now returns 199 256 nodes and 739 273 edges on a 2 MiB thread: 2.6 s in a release build, 14.9 s in a debug build. Still recursive and unbounded: printing (fetch_text) and decoding a deep definition for display. Such texts are also beyond the parse limit (SP-015 Stored definitions deeper than the parse limit cannot be edited).
Folder stores (2026-10-24, SPEC-009)
- In a store marked derived (
Store::set_derived),write_text,create_text,create_dir,rename,deleteand settings writes returnEditError::Refused: the files are the truth. fetch_textprints a folder store’s definitions from the file manifest’s span (header# read-only: from <path> bytes a..b), before falling back tosource-spanfacts.text_source(key)returns the file, source id and span a definition’s text came from; the caller compares the source id withsophia_store::folder::source_record(path, current text).idto tell whether the store is stale for that file.- Tested in
tests/acceptance.rs(folder_store_is_read_only_and_prints_files, four backends).
Store-truth folders and generated files (2026-10-28, SPEC-010)
- Laid-out names: see edit. Tested by
store_truth_folder_routes_laid_out_names_through_files, four backends. export(namespace): the sophialang definitions named directly in a namespace, as one file in name order, printed in the reader’s profile for that namespace. Each name has a span. A recursive group prints once, at its first name, and its names share that span. Two names of one record (identical content) each print under their own name: the first version shared spans by component and printedancestor_sigunderparent_sig, and the test caught it. Laid-out names and foreign records are left out, because their files are Moonkale’s. The token is a hash over each exported name and its text token, and Moonkale keeps it ingenerated.jsonwith the file’s source id. Tested bya_namespace_exports_as_one_file: every span is the name’sfetch_text, the export is deterministic, saving a span’s text back writes nothing, and an edit changes the token.- Codex review (2026-10-28), two fixes. (1) The names, and their tokens, were read in one transaction and printed in others, so an edit in between could give old text with the new token. Now the names, components and token are read in one transaction (
export_snapshot), the names are printed, and the snapshot is read again; if the token changed, the export runs again (at most 8 times). (2) A second name of a recursive group’s member (also_evenforeven) got the group’s span, though the text defined the group under the other name. sophialang has no alias syntax, so the text cannot define both. The second name in name order is now listed inaliasesas (name, the exported name it is identical to), without a span. Group members print under a name in the same namespace when they have one. The test addssample.also_evenand fails without the fix. - Second Codex review (2026-10-29). The retry compared only the export token, but the text also depends on the names of what the exported definitions reference, and on the profile. Now the export runs again unless the log position is the same before and after printing: no write at all happened. No test, as for the first race.
- Relation signatures with a name of their own (B-003, 2026-11-17). A record of kind
relationthat is a relation’s current signature prints asrelation <name>at the relation’s own name. Printed again under its own name (sample.ancestor_sig), it parsed back as a new relationancestor_sig, so the export did not round-trip throughpropose_text(SPEC-010 acceptance 3), which raisedNameExists. Such a name is now listed inaliases, paired with the smallest name in the namespace of a relation that holds the record. Both sample signatures areNat --> Nat --> Prop, one record in both relations, soparent_sigpairs withsample.ancestor. The token still covers these names. A signature record that no relation of the namespace holds still prints under its own name and does not round-trip.fetch_textandwrite_texton such a name still print and re-parse it as a new relation (not fixed here). Tested bya_namespace_exports_as_one_file(the aliases) andan_exported_file_proposes_back(the round trip, which failed before the change).
- Codex review (2026-10-28), two fixes. (1) The names, and their tokens, were read in one transaction and printed in others, so an edit in between could give old text with the new token. Now the names, components and token are read in one transaction (
propose_text(namespace, text, token): an outside edit of an exported file as a proposal, see propose.fetch_textreads the token before printing (2026-10-28, found while fixing the export). Before, it printed and then read the token, so an edit in between gave old text with the new token, and saving it would overwrite the edit silently. Now an edit after the token makes the token stale, and saving conflicts. A text token covers the name, its target and its group’s names, not the names the text references (second Codex review). Iflib.gmoves to other content, savingapp.f’s unchanged text takes the newlib.g. Daniel chose to keep the token and warn on save (SR-150, option C, 2026-10-29); see edit.
The web client’s read path (2026-10-24, SPEC-009 §4)
Interface::open_read_only(backend, reader) opens without writing (no built-in profile rows; missing ones read as Julia’s). With Sqlite::from_bytes it is what Moonkale’s web client would use. tests/wasm_read.rs runs on wasm32-unknown-unknown under Node, against tests/fixtures/web.sophia (108 KB, written by the ignored native test in tests/wasm_fixture.rs): it browses (children, neighbours, all), prints a sophialang definition and a folder-store file’s text, gets text_source, runs a recursive query, and checks that writes are refused. 4 of 4 pass. Not run in a browser yet (the runner needs geckodriver or chromedriver, not installed).
cargo install wasm-bindgen-cli --version 0.2.129 --locked --root $SOPHIA_CORPORA/tools # must match the wasm-bindgen crate
PATH=$SOPHIA_CORPORA/tools/bin:$PATH CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
cargo test -p sophia-interface --no-default-features --features sqlite --target wasm32-unknown-unknown --test wasm_readtests/wasm_deep.rs (2026-10-24): the read path (normalise and encode, decode with display names, equality, clone, hash, print in two profiles, drop) on sum, lambda and let chains 100 000 deep, in wasm: 3 of 3 pass under Node and in headless Firefox (geckodriver 0.37.1). Before the explicit stacks, 10 000 levels already raised RangeError: Maximum call stack size exceeded under Node. tests/wasm_read.rs also passes in Firefox:
WASM_BINDGEN_USE_BROWSER=1 GECKODRIVER=$(which geckodriver) PATH=$SOPHIA_CORPORA/tools/bin:$PATH \
CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
cargo test -p sophia-interface --no-default-features --features sqlite --target wasm32-unknown-unknown --test wasm_deepSince the parser and elaborator use explicit stacks too (2026-10-24, later), tests/wasm_deep.rs also checks Debug and parses the printed text back to the same id in both profiles: 3 of 3 under Node (22 s, debug build) and in Firefox (19 s).
Source text by full name (Gemini’s review, 2026-11-07 (Review 2026-11-07 Gemini (external)); fixed 2026-11-08, S-BUG-05). text_source(key) and fetch_text pass the full store path to Store::text_source, so a span is taken from the file that owns the name (folder). The source-span fact path in foreign_text matches the name’s word (folder::name_word) as a whole word, not as a substring.
Wasm, rerun 2026-11-08: wasm_read 4 of 4 and wasm_deep 3 of 3 under Node. They need the system toolchain (rustup with wasm32-unknown-unknown, CC_wasm32_unknown_unknown=clang), because nix develop’s Rust has no lld. Run without -q, which wasm-bindgen-test-runner refuses:
PATH=$SOPHIA_CORPORA/tools/bin:$PATH CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
CC_wasm32_unknown_unknown=clang cargo test -p sophia-interface --no-default-features --features sqlite \
--target wasm32-unknown-unknown --test wasm_readIn CI (2026-11-18, B-012). The wasm job of .github/workflows/ci.yml runs wasm_read and wasm_deep under Node on every CI run, beside the native job. It installs wasm-bindgen-test-runner at the version Cargo.lock gives the wasm-bindgen crate, and uses the runner image’s clang. The browser runs (geckodriver) are still by hand.