Lenticulum against RxInfer, and both against exact references, on two models: a Kalman smoother and a mixture of two linear regressions. On the smoother both are exact to and agree with each other to about . Lenticulum is within a factor 1.7 of RxInfer up to and 4.3–5.8 times slower at –, because its cost grows faster than linearly in the size of the graph. On the mixture, Lenticulum’s
fit_emis exactly maximum-likelihood EM; RxInfer’s variational estimates are slightly worse in likelihood but 9 to 10 times faster. Hand-written closed-form EM is about 2 000 and 4 900 times faster than Lenticulum and about 220 and 490 times faster than RxInfer.
Sources: Bagaev, Podusenko & de Vries, RxInfer: A Julia package for reactive real-time Bayesian inference, JOSS 2023 (the library compared, version 6.1.3). code:
bench/rxinfer_baseline.jl.Bibliography: Bagaev et al. 2023
Theory (CT-ML wiki): Variational Free Energy
Setup
Run julia -t auto --project=bench bench/rxinfer_baseline.jl (BENCH_PART=ssm or mix for one part).
Measured 2026-11-11 on one machine (8 threads, -t auto; Julia 1.13.1, RxInfer 6.1.3), both parts
in one run; times are warm (after one run), best of five. The script checks, and stops otherwise,
that infer! converged and that fit_em’s free energy at its estimate is the score below.
1. A linear-Gaussian state-space model
A constant-velocity state in , position-only measurements, steps
(Xoshiro(T) data). Both libraries run belief propagation on a tree, which is exact, so their
free energies must equal and their posteriors the conditional marginals. The
references are a Kalman filter’s prediction-error decomposition in 256-bit arithmetic for
, and the dense joint solve in information form for the marginals.
| mean error L / R | L build | L infer | L free energy | L total | R | R without | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| 10 | 3.6e-15 | 5.7e-14 | 5.3e-14 | 8.9e-15 / 3.5e-13 | 0.2 | 0.7 | 0.4 | 1.3 | 1.0 | 0.7 |
| 100 | 1.7e-13 | 2.8e-13 | 1.1e-13 | 9.9e-14 / 3.3e-13 | 2.6 | 7.8 | 4.6 | 15.3 | 9.2 | 6.7 |
| 1000 | 5.8e-11 | 3.5e-11 | 2.3e-11 | 6.4e-12 / 4.1e-12 | 127.5 | 164.6 | 167.6 | 458.3 | 106.9 | 76.2 |
| 2000 | 8.8e-11 | 3.2e-11 | 1.2e-10 | 7.3e-12 / 6.4e-12 | 470.6 | 493.7 | 599.9 | 1583.0 | 273.9 | 188.1 |
Times in ms. L build: the graph and its parameters; L infer: tree_schedule and infer!; L free
energy: scalar_free_energy. R: RxInfer’s infer with free_energy = true, model creation
included. Errors cover every marginal, to . The largest covariance error is – for Lenticulum and
for RxInfer at every .
What it shows.
- Both are exact, to in and in the means, and agree with each other to .
- Lenticulum’s cost grows faster than linearly. From to the graph build
takes 3.7 times as long, the free energy 3.6 times and
infer!3.0 times; RxInfer takes 2.6 (with ) and 2.5 times. The quadratic part is the one the Feature Backlog records for parameter keys looked up by name (LP-024: the free energy of 601, 1 201 and 2 401 factors in 7.7, 25.8 and 102 ms). Up to the two are within a factor 1.7. - RxInfer needs an option beyond . At its default run overflowed the
stack; its documented remedy,
limit_stack_depth = 100, is used there.
Getting the reference right. Two dense formulas for were wrong at while the libraries agreed with each other to . The covariance form was off by : the prior covariance of a constant-velocity state grows like . The information form was off by : it is a difference of terms of order –. The Kalman filter in BigFloat is exact.
2. A mixture of two linear regressions
300 or 3 000 points, , line with weights ,
, , , (Xoshiro(11), the data of
lib/Lenticulum.jl/examples/regime_lines.md). One start for all: , ,
, .
- EM (hand): closed-form EM for a mixture of regressions, the reference.
- Lenticulum: one
RegimeFactorper point with tied parameters,fit_em(the closed-form M-steps ofsrc/m_steps.jl), as intest/em.jl. Graph building not timed. - RxInfer: priors , , , the means
dot(X[i], θk), aNormalMixturelikelihood, mean-field VMP over all variables. Point estimates are posterior means, . Run until its free energy changes by less than per iteration.
The estimators differ: EM gives the maximum-likelihood parameters, VMP a variational posterior under priors. The common score is at each estimate, labels matched.
| method | iterations | time (ms) | ||||||
|---|---|---|---|---|---|---|---|---|
| 300 | EM (hand) | (0.9972, −0.9941) | (−0.0092, 0.4784) | (0.1934, 0.5023) | (0.519, 0.481) | 253.389349 | 11 | 0.6 |
| 300 | Lenticulum | (0.9972, −0.9941) | (−0.0092, 0.4784) | (0.1934, 0.5023) | (0.519, 0.481) | 253.389349 | 11 | 723.4 |
| 300 | L (plate) | (0.9972, −0.9941) | (−0.0092, 0.4784) | (0.1934, 0.5023) | (0.519, 0.481) | 253.389349 | 11 | 19.0 |
| 300 | RxInfer | (0.9970, −0.9940) | (−0.0092, 0.4805) | (0.2284, 0.5162) | (0.522, 0.478) | 256.859526 | 32 | 124.9 |
| 3000 | EM (hand) | (1.0003, −1.0064) | (0.0021, 0.4979) | (0.1972, 0.4915) | (0.500, 0.500) | 2 603.684235 | 11 | 5.7 |
| 3000 | Lenticulum | (1.0003, −1.0064) | (0.0021, 0.4979) | (0.1972, 0.4915) | (0.500, 0.500) | 2 603.684235 | 11 | 9 272.9 |
| 3000 | L (plate) | (1.0003, −1.0064) | (0.0021, 0.4979) | (0.1972, 0.4915) | (0.500, 0.500) | 2 603.684235 | 11 | 147.7 |
| 3000 | RxInfer | (1.0003, −1.0065) | (0.0022, 0.4978) | (0.2009, 0.4927) | (0.501, 0.499) | 2 604.168160 | 40 | 3 421.8 |
Times from the run of 2026-11-15 (BENCH_PART=mix, after I21; one machine, -t auto; log
~/.cache/lenticulum/i21/bench_mix.log). “Lenticulum” is
the graph without a plate record, “L (plate)” the same graph with its replicas recorded as a plate
(as_plate) and swept by plate_schedule, where the regime factors’ messages, free energies and EM
statistics are batched (I6), since I19 in one stacked joint for all replicas, and since I20 with the
data factors grouped and the regime factors’ messages built as blocks, since I21 also when the
outcomes’ precisions differ, with the energy rows kept. The run after I20 (2026-11-15) had 30.3 and
236.5 ms for the plate (746.5 and 9 408.5 ms unplated, 125.8 and 3 249.5 ms for RxInfer); the run
of 2026-11-13, after
I19, had 67.9 and 679.9 ms for the plate (757.6 and 9 504.9 ms unplated, 125.5 and 3 405.5 ms for
RxInfer). The run of
2026-11-12, after I6 and before I19, had 356.7 and 3 806.3 ms for the plate (801.4 and 10 076.0 ms
unplated, 126.8 and 3 149.2 ms for RxInfer). The first table of 2026-11-11 had 1 196.3 and 31 635.8 ms for the
unplated graph and 132.6 and 3 197.4 ms for RxInfer; what made the unplated graph faster since was not
measured step by step. Lenticulum’s time excludes building its graph, 2.6 and 28.7 ms; RxInfer’s includes creating its
model. RxInfer’s own free energy at convergence is 305.24 and 2 666.56: a bound under its priors,
not comparable with the column above.
What it shows.
- Lenticulum’s EM is EM: the same iterates and the same maximum to the printed digits, as
test/em.jlasserts step for step. - Unplated it is slow; as a plate it is faster than RxInfer, still far from the closed form.
Unplated, 0.72 s against 0.6 ms for the closed form at , 9.3 s against 5.7 ms at
: 5.8 and 2.7 times RxInfer. As a plate (I6, I19, I20 and I21 of the Feature Backlog)
it takes 19 ms and 0.15 s, 38 and 63 times faster than unplated, 0.15 and 0.043 times RxInfer’s
time, and ten times the data costs 7.8 times the time; it is still about 32 and 26 times the
closed form. A profile of the plate after I21 (
bench/regime_plate_profile.jl) has no single hot spot: about half offit_em’s time in the sweeps (mostly the regime factors’ stacked joint and solves), a fifth in the free energy. - RxInfer’s estimates are close but not maximum likelihood. At its is 0.228 against 0.193, which costs 3.5 nats in . At it is 0.48 nats. Partly that is the estimator: is not the maximiser, and the posterior is mean-field. Which part is which was not separated.
Caveats
- One machine, one data set per size, one start. Both EMs kept the start’s label order; RxInfer’s came out swapped and were matched by the slopes.
- RxInfer was run as its documentation describes, not tuned: no custom schedules,
limit_stack_depthonly where the default failed. Lenticulum’s graph-building time is excluded in part 2 (reported beside it) and included in part 1’s total. - Not measured: compilation time (first run), memory, streaming use, graphs with loops.
Review (2026-11-11)
Mistral (vibe) reviewed the script and this note before the first commit, given the
requirement and the diff. Its ten points:
| # | point | right? | done |
|---|---|---|---|
| 1 | rx_converged counts one iteration too many | no: a test on a constructed trace returns the first iteration whose change is below the tolerance (findfirst returns a position in 2:length(F)) | nothing |
| 2 | part 2 times RxInfer’s model creation but not Lenticulum’s graph build | yes | the build is timed and reported (2.7 and 28 ms) |
| 3, 4 | ”9–10 times” and “210–420 times” did not cover 10.3 and 424 | yes | the ranges, from the final run |
| 5 | ”agree as closely” as while is | yes | stated as measured |
| 6 | the run command did not request the 8 threads used | yes | -t auto in the command |
| 7 | RxInfer’s marginal was not compared | yes | compared |
| 8 | infer!’s convergence report was discarded | yes | checked |
| 9 | only RxInfer’s labels were matched | yes | Lenticulum’s too |
| 10 | fit_em’s free energy was not checked against the score | yes | checked |
Points 2–10 changed no result but the timing description and the ranges.