Derivation. The spectral solution of Fitting is a Nullspace Problem uses the algebraic distance. A first-order geometric correction produces a point-dependent scalarisation — and shows what the energy space is: the space the noise lives in.
Sources: original analysis; Chojnacki, Brooks, van den Hengel & Gawley (2000), On the fitting of surfaces to data with covariances, §4.1–§4.2, Fig. 1; Absil, Mahony & Sepulchre (2008), Optimization Algorithms on Matrix Manifolds, Chapter 7, Algorithms 10–11; code:
lib/LenticulumCore.jl/src/energy.jl(SampsonNorm,localise,sampson_damping),lib/LenticulumCore.jl/test/runtests.jl(circle closed form),lib/AlgebraicFactors.jl/src/polynomial.jl,polyfit.jl,lib/AlgebraicFactors.jl/test/polynomial.jl(real factor gauge/basis tests and fitting).Bibliography: Chojnacki et al. 2000 · Absil et al. 2008
Theory (CT-ML wiki): Lax Functor · Statistical Game
The two distances
Algebraic distance. . Cheap, quadratic in , hence the closed-form fit. But it is not a distance: it depends on how the variety is written. Scaling scales its squared residual. Locally, for a hypersurface near a regular point, so a large residual gradient inflates algebraic distance even at the same geometric distance.
Geometric distance. . The thing you actually mean. Independent of the equations. Also very expensive: the number of complex critical points of on is the Euclidean Distance Degree (Draisma–Horobeț–Ottaviani–Sturmfels–Thomas). For a generic hypersurface of degree in ,
Sanity check: a plane conic, , : — the classical fact that the foot of the perpendicular to a generic conic solves a quartic. For fixed degree , this formula grows exponentially in . It counts complex critical points, not real minima or a runtime bound. Exact geometric fitting can therefore require an expensive projection subproblem; this is not a universal hardness claim (affine models, for example, have closed-form projection).
The first-order correction: Sampson distance
Linearise. Near , the variety is approximately the affine subspace with . The minimum-norm displacement to that subspace, assuming has full row rank, solves subject to , whose solution is , giving
the Sampson distance (Sampson 1982; Taubin 1991). For a single equation () it reduces to the familiar .
The derivation that matters: is where the noise lives
Now read the same formula statistically rather than geometrically. Posit the honest errors-in-variables model: the true point lies on , and we observe with in the data space.
Push the noise forward through the residual. To first order,
For a Gaussian approximation in residual coordinates with invertible , the negative log-density includes a normalisation term:
The quadratic term at is half the squared Sampson distance defined
above. SampsonNorm implements this energy, not a normalised likelihood. The earlier
formula omitted as a “constant”, although varies with point
and parameters. A density in residual coordinates also changes with the residual gauge;
it is not by itself a likelihood for the observed data. Two consequences:
What the energy space is
The residual space of Scalar and Multivariate Energy is the space the observation noise lives in after pushforward, and the scalarisation is the noise precision. Choosing is choosing a noise model, not choosing a convenience.
Why the energy must stay multivariate
cannot be computed from the scalar energy . It needs and its Jacobian — i.e. the vector energy. A factor that scalarises early has irrecoverably thrown away its own noise model. This is the concrete instance of Scalar and Multivariate Energy §6’s claim that “scalarising early destroys the metric”.
The cost: three properties lost
generally depends on both and . That breaks three things that made the algebraic distance attractive:
- The objective is no longer quadratic in .
does not have the fixed-metric
nullspace solution of Fitting is a Nullspace Problem. Freezing the metrics and
resolving a quadratic subproblem omits derivatives of those metrics. Such
naive reweighting does not in general minimise the full Sampson objective.
polyfit implements its full gradient with an Armijo Riemannian solver,
the corrected FNS iteration for k=1, and Riemannian trust regions with
analytic Hessian and truncated CG.
AbstractSampsonMethod/sampson_solvemake the solver exchangeable; the default is the trust-region method. - There is no single fixed . A frozen
SquaredNorm(M)is convex and has Jensen gap . A field of metrics at different points is a different object: the fixed-map Jensen identity does not apply.jensen_gap(SampsonNorm(), ...)throws, including for empty samples. The earlier claim that the varying metric’s gap was was wrong. - The inverse fails where the pushed-forward covariance loses rank.
In particular this happens when loses row rank. Vanishing variance in a
residual direction does not mean unconstrained noise there; the nonsingular
Gaussian approximation has failed.
SquaredNormdoes validate PSD metrics; the earlier claim that it did not was stale.SampsonNormrejects singular or numerically deficient covariance rather than silently taking a pseudoinverse.
Localisation and gauge-compatible damping (implemented)
L-ADR-0007 Point-dependent scalarisations are localised before use was accepted on 2026-10-25.
Its implementation in energy adds AbstractPointScalarisation, localise and
SampsonNorm. The pointwise path localises before scalarising; free-energy scalarisation
is unchanged. Graded scalarisations containing point-dependent parts are rejected until
their parts can be given their own factor and arguments.
For and , a factor can override sampson_damping with
. For any data covariance of the right dimension,
Under , residual and covariance transform by and , so is invariant when defined. For full-row-rank , PSD and , this covariance is SPD at rank-deficient too. Default identity damping lacks this law. The implementation requires : badly conditioned gauges can fail the numerical check even when exact-arithmetic energies agree.
Core tests check the circle closed form , its ratio tending
to geometric energy , covariance scaling and singular rejection/damping.
AlgebraicFactors tests use a test-only degree-two residual with two equations in two
variables, seed 2307, 20 random points and a well-conditioned row transformation.
They check gauge and QR-representative agreement (rtol=1e-10), including damping
at a constructed singular point, and demonstrate failures
of identity damping and algebraic distance (difference above 1e-3 at some point).
Those original §2.3 tests remain. The real PolynomialFactor tests now check
its exact Jacobian, row gauges at λ=0 and 0.3, anisotropic covariance and a
singular point. Undamped monomial/Chebyshev energies agree. The coefficient
damping ΘΘᵀ is row-gauge compatible but changes under nonorthogonal polynomial
basis conversions; polynomial states that limitation.
The honest summary
| distance | fit | statistical interpretation |
|---|---|---|
| algebraic | global spectral solution under orthonormal-row constraint | depends on residual/feature scaling |
| Sampson | non-convex; full-gradient descent, RTR, or k=1 FNS | first-order geometric correction |
| geometric | non-convex projection problem | profile errors-in-variables MLE under isotropic Gaussian noise |
fit_sampson initialises at fit_nullspace by default. The seeded noisy-conic
test lowers the full Sampson cost, checks the gradient and random local
directions, and compares all three methods. RTR reaches tol=1e-7 in four iterations
on the unscaled ellipse, its X/2 variant, and the noisy two-quadric k=2 curve;
polyfit records costs, gradient norms and budgets. These checks do not
certify a global minimum.
Algebraic and Sampson energies use the same residual, with a fixed metric versus
a field of metrics. Exact geometric projection additionally needs the variety and a
projection solver; it is not in general a function of the residual vector alone.
This refines the reweighting discussion in Scalar and Multivariate Energy §6.4.
Related: Fitting is a Nullspace Problem, Scalar and Multivariate Energy, Branches and the Discriminant, The Algebraic Factor as a Statistical Game