open-problem

Status: open (2026-11-05). Area: probabilistic graphical models, tropical geometry, topological data analysis · Prerequisites: Switching Constraint Factors, Why Learned Relations Do Not Collapse · Related: Feature Backlog (overlaid factors), lib/VariationalDiffusion.jl/examples/calibration.md, LP-022 Constraint factors are normalised in their residual, not their channels.

Sources: Zhang, Naitzat & Lim, Tropical Geometry of Deep Neural Networks, ICML 2018 (ReLU networks are tropical rational maps); original analysis of this repository. code: lib/Lenticulum.jl/src/switching.jl; tests: lib/Lenticulum.jl/test/rq008.jl

Bibliography: Zhang et al. 2018

Theory (CT-ML wiki): Variational Free Energy

Question

Several factors overlaid on the same variables multiply, so their energies add. Which relations can such overlays of simple factors (linear constraints, fixed nonlinear energies, switches) represent? Which of them stay tractable, and normalised in their data channels? How does the combinatorics of a switching factor’s low-noise limit (a tropical polynomial) relate to the topology of the learned relation (its branches and their connections)? The motivation is a program in which a large learned model is distilled step by step into a factor graph of small learners. There, each step needs to know how many branches a piece has and how the pieces connect.

Setting

  • Overlay. Factors on one variable set form the overlay , with energy . Each keeps its own parameters.
  • Linear constraint. , convex in .
  • Switching factor (SwitchingConstraintFactor, the location-shift case: , shared by the outcomes) with its outcome summed out:
  • Tropical polynomial (max-plus): , convex and piecewise affine. Its tropical hypersurface is the set where the maximum is attained at least twice; it partitions -space into the cells where each term wins.

What is known

  1. Convex overlays stay unimodal (elementary). A sum of convex energies is convex, so an overlay of log-concave factors (a linear constraint and a convex ) is log-concave: its superlevel sets are convex and its mode is unique if any term is strictly convex. Such an overlay cannot represent a mixture, and it cannot replace a switching factor.
  2. The summed-out switch is a soft-min, and a difference of convex functions (derived here; checked in lib/Lenticulum.jl/test/rq008.jl against the graph’s exact free energy to ). With ,

That is the linear constraint’s convex quadratic minus a convex log-sum-exp of affine functions of .

  1. Its low-noise limit is tropical (derived here; checked in lib/Lenticulum.jl/test/rq008.jl). With ,

a convex quadratic minus a tropical polynomial in . The error is , dominated by . Measured: 0.035, and at , that is –. The weights drop out of the limit; they set where the cells’ boundaries lie at finite . So a switching factor is an overlay: a linear constraint plus a fixed concave, piecewise-linear factor (minus a tropical polynomial), with learnable offsets . The tropical hypersurface separates the regions each outcome wins.

  1. ReLU networks are tropical rational maps (Zhang, Naitzat & Lim 2018), differences of two tropical polynomials. The same max-plus language describes switches and any piecewise-linear learner distilled into them.
  2. Collapse. A linear constraint with every coefficient learned collapses on data (LP-022). A plain overlay with a convex does not prevent it, because at the linear factor is still at its maximum. An overlay node that normalises its product over the data channel, , would: it makes the overlay a conditional density. For log-concave members this is a log-concave integral, so it is tractable by Laplace or quadrature in low dimension.
  3. Branch counts are measured by hand today. examples/calibration.md counts recalled branches of learned implicit relations against analytic ones.

What would count as an answer

  • A representation theorem. Characterise the energies an overlay of linear constraints and fixed piecewise-linear factors represents (all differences of convex piecewise-quadratic functions?), and which of them are normalisable in a given channel.
  • A tractability result. For which overlays are exact or provably good messages available (closed form, EP with convergence, a cluster node)? And what does the normalised overlay node cost?
  • A topology statement. Relate the tropical hypersurface of a switching factor (its cells and their adjacency) to the homology of the learned relation’s zero set at small noise. For instance, the number of branches of a query against the number of cells its line crosses.
  • A measurement protocol. Persistent homology of the root sets of implicit_roots over a grid of queries (, the components, as a branch count; , the loops), compared with the analytic relation on the calibration benchmark, and used to choose the number of outcomes of a distilled switch.

Why it matters here

  • Overlaid factors (the Feature Backlog’s “mirrored factors”): the identity in 2 says which overlays add expressive power (a concave, piecewise-linear member) and which do not (a convex one). It also suggests building the overlay as one node, exact like a cluster node, with an optional normaliser.
  • Distillation into a factor graph (the LenticulumLM moonshot, Language Models; a diffusion model distilled subgraph by subgraph): each distilled piece needs a branch count and a connectivity. The tropical form gives the pieces, and persistent homology measures the target.
  • The calibration defects (spurious and off-domain roots, merged branches) are questions about the topology of the smoothed relation against the data’s.

Leads

  • Verify 2 and 3 in more cases: outcome-dependent or (RegimeFactor). There the pieces are different quadratics, and the limit is a minimum of quadratics, not a quadratic minus a tropical polynomial.
  • The overlay node exists since 2026-11-05 for members with a cluster form (OverlayFactor, lib/Mycelium.jl/src/overlay.md), with the normaliser per joint outcome; it equals a cluster node of the members and the closed forms (lib/Lenticulum.jl/test/overlay.jl). Open: members with nonlinear energies (EP or Laplace), and the normalisation of the whole product, which gates the outcomes by .
  • Dequantisation. The passage from log-sum-exp to max as is Maslov’s dequantisation. The temperature is the switch’s noise, so annealing the noise is a path from a smooth mixture to its tropical skeleton, as annealing the field levels is for implicit relations (calibration.md).
  • Persistent homology on the calibration benchmark’s root sets: Ripser-style Vietoris–Rips over the found roots, per query line and over the whole grid.
  • Known not to work: convex members alone (1). A product of experts with log-concave experts is unimodal.