Time as a Base

design open-problem

Is there a clean MTK × Lenticulum extension — one that assumes a time dimension?

Yes. And the reason it is clean is that it is a base change, not a redesign: nothing about polarity, combine, the Bethe form or the hypergraph structure has to change. What changes is what a variable carries.

The catch is in §8, and it is real: the clean design is a division of labour, not a merge.

Nothing here is implemented. This is a design note.

Sources: Willems, The Behavioral Approach to Open and Interconnected Systems, IEEE CSM 2007; Anderson, Barfoot, Tong, Särkkä, Batch Nonlinear Continuous-Time Trajectory Estimation as Exactly Sparse Gaussian Process Regression, RSS 2014 / Autonomous Robots 39(3):221–238, arXiv:1412.0630; Särkkä, Solin, Hartikainen, Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing, IEEE Signal Processing Magazine 2013; Dong, Mukadam, Dellaert, Boots, Motion Planning as Probabilistic Inference using Gaussian Processes and Factor Graphs, RSS 2016; Rauch, Tung, Striebel, 1965; code: beliefs.jl

Theory (CT-ML wiki): Hypergraph Category · Frobenius Monoid · Bayesian Inversion · Variational Free Energy

1. The one-line change

Today a variable node is a wire carrying a value in $X$. Make it carry a trajectory:

$$\text{now: }x\in X⟹\text{then: }x\in X^{\mathbb{T}},\text{or }\Gamma (E\to \mathbb{T})$$

That is Willems exactly — a system is a subset of ${\mathbb{W}}^{\mathbb{T}}$, and ModelingToolkit as an Acausal Relation §3 already establishes that Willems’ interconnection (variable sharing) is Mycelium’s shared variable node. The temporal extension keeps the whole graph structure and changes the value category from spaces to trajectory spaces.

Everything downstream follows from that one substitution, which is why it deserves to be called clean.

2. Three kinds of temporal factor

Once variables carry trajectories, factors sort into three kinds — and the taxonomy is the useful part of this note.

kind	constrains	example	today
(a) pointwise	values at one instant: $0=g(x(t),y(t))$	Ohm’s law, a measurement model	every current Lenticulum factor
(b) differential	the jet at an instant: $0=f(x(t),\dot{x}(t),t)$	MTK’s D(x) ~ ...	none — needs the derivation
(c) two-point	two times: $x(t)={\Phi }_{s\to t}(x(s))$	a flow, a Markov transition	NeuralODEFactor, GaussianFactor on a chain

Two observations that make the taxonomy earn its place:

A Lenticulum factor is a temporal factor supported at one instant. Kind (a) is what the whole current library does, applied fibrewise over $\mathbb{T}$. So the extension is conservative: existing factors keep working, lifted pointwise.

Kinds (b) and (c) are related by integration. A differential factor integrates to a two-point factor over any interval. That is exactly the discretisation move — MTK’s continuous equation becoming a chain of transition factors — and it is the same transformation Särkkä’s LTI-SDE ↔ discrete-time state-space duality performs. So (b) is the primitive and (c) is what a solver produces from it.

NeuralODEFactor is instructive here: it is a kind-(c) factor produced by integrating a kind-(b) one, with the integration sealed inside. The Structural Gap to ModelingToolkit §2 is about that sealing.

3. The derivation is structure, not a factor

$D$ relates a variable to itself at infinitesimally nearby times. A bipartite factor graph cannot express that: there is no edge from a variable to itself.

The move MTK itself makes — and every index-reduction algorithm makes — is to introduce $\dot{x}$ as a separate variable with the relation $\dot{x}=D(x)$. But that relation is not an arbitrary factor. It is fixed, universal, non-learnable, and the solver must know what it means rather than merely evaluating it.

So in the extension, $D$ is part of the signature of the variable category, alongside the base $\mathbb{T}$. A Channel would carry:

• a base $\mathbb{T}$ (the independent variable),
• a fibre (the space the values live in),
• and a jet order — how many derivatives this channel exposes.

That last field is where the index problem enters, and §7 is about it.

4. What a belief over a trajectory is — and the self-similarity

This is where the extension stops being bookkeeping and starts paying.

A belief over a trajectory is a stochastic process. For it to be tractable it must be Gaussian and Markov — i.e. the solution of a linear SDE. And then the payoff is a theorem:

A Gauss–Markov process, evaluated at times $t_1<\cdots <t_N$, has a joint precision matrix that is exactly block-tridiagonal.

That is Anderson, Barfoot, Tong & Särkkä’s result (arXiv:1412.0630): a GP prior generated by an LTI SDE has an exactly sparse inverse kernel. And a block-tridiagonal precision is precisely the precision of a chain factor graph — each off-diagonal block is one pairwise factor.

The extension is self-similar

A belief over a trajectory is itself a factor graph. Specifically, it is The Linear Gaussian Chain — which is why that note is the discrete shadow of this one and why implementing the odometry chain first was the right order.

So the temporal extension needs no new belief type. It needs GaussianBelief with a structured (block-tridiagonal) precision, and message passing on it is message passing on the chain. The canonical form of beliefs.jl already adds precisions; block-tridiagonal plus block-tridiagonal is block-tridiagonal, so combine still works.

And this connects two gaps into one: The Structural Gap to ModelingToolkit §4 says subsystems are not first-class, and §2 says time is sealed. Both are the same missing capability — a variable whose belief is a subgraph. Build that once and you get hierarchy and continuous time together.

5. The operation this buys: interpolation

The reason to do any of this is one operation:

$\text{query }x(t^{\ast })\text{ at a time no factor mentions}$

For a Gauss–Markov prior this is GP interpolation, and the same sparsity that makes the prior a chain makes interpolation $O(1)$ — it is a local computation between the two bracketing states, not a global solve. Anderson et al. call it out explicitly alongside the regression.

Concretely it fixes three things:

1. The $z(0.5)$ gap. You can attach a factor to the trajectory at any time, not just at the endpoints a flow was integrated between.
2. Asynchronous measurements. Sensors do not tick together. This is the actual reason continuous-time SLAM exists, and the discrete chain of The Linear Gaussian Chain handles it only by inventing a variable per measurement time.
3. Solver-independent state. The discretisation is a property of the query, not of the model, so refining it does not change the graph.

6. What transports unchanged — the case for “clean”

The claim that this is a base change rather than a redesign is checkable. Going through the machinery:

	transports?	why
Polarity, Observed/Unobserved/Latent	yes	a per-channel labelling; indifferent to what the channel carries
combine as canonical addition	yes	precisions of Gauss–Markov processes add, and stay block-tridiagonal
the exclusion principle	yes	a statement about graph structure, not values
assemble / polarity resolution	yes	unchanged
the Frobenius spider	yes	a shared trajectory is still a junction of equal wires
Bethe free energy	mostly	entropies become entropy rates; finite for Gauss–Markov, and the counting correction is unchanged
schedules	yes, and it gets a name	forward–backward on the chain is the RTS smoother

That last row is satisfying: Schedules’s forward_backward_schedule on a temporal chain is exactly Rauch–Tung–Striebel smoothing, and The Linear Gaussian Chain §3’s “these are smoothed marginals, not filtered ones” is the same observation one level down.

7. What does not transport

The index problem. A kind-(b) factor of differentiation index $\ge 2$ has hidden constraints — consequences of the equations invisible without differentiating. A naive temporal factor graph would happily pass messages that violate them, and the failure is silent drift rather than an error. Differential Algebra and DAE Factors is the note on this, and its verdict stands: index reduction is a compile-time, exact-arithmetic, small-system job. It has to happen before any belief is propagated.

Consistent initialization. Same problem as MTK’s, and the same one as the loopy bootstrap deadlock of constraint.md §4.2.

Non-Gaussian trajectory beliefs. Dropping Markov-Gaussian costs the sparsity, and with it everything in §4 and §5. Particle beliefs over trajectories are particle smoothing, whose cost grows with the horizon — the vault’s general combine gap (messages §1) in its most painful form.

Loops. A temporal chain is a tree, so message passing is exact. A physical model with algebraic loops — which is most of them, per ModelingToolkit as an Acausal Relation §6 — is not, and everything in Loopy Message Passing applies with the extra cost that each variable is now a whole trajectory.

8. The catch: this needs the symbolic layer

Here is the honest problem, and it is the reason this note ends in a division of labour rather than a merged library.

Everything in §7 that must happen before inference — index reduction, alias elimination, tearing, consistent initialization — is symbolic. It requires looking inside a factor and rewriting it. And The Structural Gap to ModelingToolkit §1 argues that Lenticulum structurally cannot do that, because its factors are opaque objects and making them terms would cost the thing the design exists for (a factor’s internals being an arbitrary network).

So the clean extension has a hard dependency on the one gap that is not addable.

The design that survives this

MTK owns compile time. Lenticulum owns run time.

• MTK takes the acausal, hierarchical, unit-checked, possibly high-index model and compiles it: flattening, alias elimination, index reduction, tearing.
• The interface between them is the compiled, index-1, semi-explicit system — a flat set of equations in a known set of states, with the hidden constraints made explicit.
• Lenticulum takes that and does what MTK cannot: puts beliefs on the states, noise on the equations, priors on the parameters, and a free energy on the whole thing.

This is §8 of ModelingToolkit as an Acausal Relation — the MTKFactor whose inversion is a solver — taken seriously and given a reason. It is not a compromise: it is the only arrangement in which each library does the part it is structurally capable of.

9. What it would actually be for

Not “MTK with error bars”. The product is probabilistic acausal modelling, and the target problems are the ones neither library reaches alone:

• Bayesian system identification on a physical model. Parameters of an acausal circuit or mechanism, with posteriors rather than point estimates, from noisy asynchronous data. MTK calibrates; it does not give you a posterior. GTSAM gives posteriors; it has no continuous physical model.
• Stochastic DAEs. MTK’s equation plus process noise on the constraint — a physical law that holds approximately, which is what LinearConstraintFactor’s $Q$ already expresses pointwise and what this would express over time.
• Grey-box models. Some factors are physical equations from MTK; others are learned (DEQFactor, NeuralODEFactor, a diffusion prior). The graph does not care which, which is the entire point of the factor interface, and this is where the three families of Implicit Learners would meet a real physical model.
• Model misfit as a number. The free energy of the graph measures how badly the physical equations fit the data, per factor, via the graded energy. That is a diagnostic MTK has no vocabulary for.

10. The smallest honest first step

Not the full extension. The one thing that would make the rest real:

A trajectory belief type — a Gauss–Markov process represented by its block-tridiagonal precision — together with combine and interpolate on it.

Everything else in this note is scaffolding around that object. And it is blocked on the same thing three factor packages are already blocked on: GaussianBelief lives in the top-level Lenticulum package where no lib/ package can reach it (The Equilibrium Family §5). Moving it to LenticulumCore is the prerequisite for this note and for three others.

Sources

• Willems, The Behavioral Approach to Open and Interconnected Systems, IEEE CSM 2007 — a system is its behaviour; interconnection is variable sharing.
• Anderson, Barfoot, Tong, Särkkä, Batch Nonlinear Continuous-Time Trajectory Estimation as Exactly Sparse Gaussian Process Regression, RSS 2014 / Autonomous Robots 39(3):221–238, arXiv:1412.0630 — the block-tridiagonal inverse kernel and $O(1)$ interpolation. The load-bearing citation for §4 and §5.
• Särkkä, Solin, Hartikainen, Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing, IEEE Signal Processing Magazine 2013 — the GP ↔ LTI-SDE duality that turns a covariance function into a Markov process.
• Dong, Mukadam, Dellaert, Boots, Motion Planning as Probabilistic Inference using Gaussian Processes and Factor Graphs, RSS 2016 — the same construction used as a working factor graph.
• Rauch, Tung, Striebel, 1965 — the smoother that §6’s schedule row is.

The other way to make feedback acyclic

Unrolling a loop into a chain of time-indexed variables is one option; lifting a level — a meta-graph whose variables are the base graph’s state — is the other. They are alternatives rather than competitors, and The Inferencer and the Optimizer §4 takes the second road.

Related: The Inferencer and the Optimizer, The Structural Gap to ModelingToolkit, ModelingToolkit as an Acausal Relation, The Linear Gaussian Chain, Differential Algebra and DAE Factors, Acausal Composition is a Hypergraph Category, Composition is Elimination, Schedules, Loopy Message Passing, Implicit Learners