model derivation

Derivation. The spectral solution of Fitting is a Nullspace Problem uses the algebraic distance. A first-order geometric correction produces a point-dependent scalarisation — and shows what the energy space is: the space the noise lives in.

Sources: original analysis; Chojnacki, Brooks, van den Hengel & Gawley (2000), On the fitting of surfaces to data with covariances, §4.1–§4.2, Fig. 1; Absil, Mahony & Sepulchre (2008), Optimization Algorithms on Matrix Manifolds, Chapter 7, Algorithms 10–11; code: lib/LenticulumCore.jl/src/energy.jl (SampsonNorm, localise, sampson_damping), lib/LenticulumCore.jl/test/runtests.jl (circle closed form), lib/AlgebraicFactors.jl/src/polynomial.jl, polyfit.jl, lib/AlgebraicFactors.jl/test/polynomial.jl (real factor gauge/basis tests and fitting).

Bibliography: Chojnacki et al. 2000 · Absil et al. 2008

Theory (CT-ML wiki): Lax Functor · Statistical Game

The two distances

Algebraic distance. . Cheap, quadratic in , hence the closed-form fit. But it is not a distance: it depends on how the variety is written. Scaling scales its squared residual. Locally, for a hypersurface near a regular point, so a large residual gradient inflates algebraic distance even at the same geometric distance.

Geometric distance. . The thing you actually mean. Independent of the equations. Also very expensive: the number of complex critical points of on is the Euclidean Distance Degree (Draisma–Horobeț–Ottaviani–Sturmfels–Thomas). For a generic hypersurface of degree in ,

Sanity check: a plane conic, , : — the classical fact that the foot of the perpendicular to a generic conic solves a quartic. For fixed degree , this formula grows exponentially in . It counts complex critical points, not real minima or a runtime bound. Exact geometric fitting can therefore require an expensive projection subproblem; this is not a universal hardness claim (affine models, for example, have closed-form projection).

The first-order correction: Sampson distance

Linearise. Near , the variety is approximately the affine subspace with . The minimum-norm displacement to that subspace, assuming has full row rank, solves subject to , whose solution is , giving

the Sampson distance (Sampson 1982; Taubin 1991). For a single equation () it reduces to the familiar .

The derivation that matters: is where the noise lives

Now read the same formula statistically rather than geometrically. Posit the honest errors-in-variables model: the true point lies on , and we observe with in the data space.

Push the noise forward through the residual. To first order,

For a Gaussian approximation in residual coordinates with invertible , the negative log-density includes a normalisation term:

The quadratic term at is half the squared Sampson distance defined above. SampsonNorm implements this energy, not a normalised likelihood. The earlier formula omitted as a “constant”, although varies with point and parameters. A density in residual coordinates also changes with the residual gauge; it is not by itself a likelihood for the observed data. Two consequences:

What the energy space is

The residual space of Scalar and Multivariate Energy is the space the observation noise lives in after pushforward, and the scalarisation is the noise precision. Choosing is choosing a noise model, not choosing a convenience.

Why the energy must stay multivariate

cannot be computed from the scalar energy . It needs and its Jacobian — i.e. the vector energy. A factor that scalarises early has irrecoverably thrown away its own noise model. This is the concrete instance of Scalar and Multivariate Energy §6’s claim that “scalarising early destroys the metric”.

The cost: three properties lost

generally depends on both and . That breaks three things that made the algebraic distance attractive:

  1. The objective is no longer quadratic in . does not have the fixed-metric nullspace solution of Fitting is a Nullspace Problem. Freezing the metrics and resolving a quadratic subproblem omits derivatives of those metrics. Such naive reweighting does not in general minimise the full Sampson objective. polyfit implements its full gradient with an Armijo Riemannian solver, the corrected FNS iteration for k=1, and Riemannian trust regions with analytic Hessian and truncated CG. AbstractSampsonMethod/sampson_solve make the solver exchangeable; the default is the trust-region method.
  2. There is no single fixed . A frozen SquaredNorm(M) is convex and has Jensen gap . A field of metrics at different points is a different object: the fixed-map Jensen identity does not apply. jensen_gap(SampsonNorm(), ...) throws, including for empty samples. The earlier claim that the varying metric’s gap was was wrong.
  3. The inverse fails where the pushed-forward covariance loses rank. In particular this happens when loses row rank. Vanishing variance in a residual direction does not mean unconstrained noise there; the nonsingular Gaussian approximation has failed. SquaredNorm does validate PSD metrics; the earlier claim that it did not was stale. SampsonNorm rejects singular or numerically deficient covariance rather than silently taking a pseudoinverse.

Localisation and gauge-compatible damping (implemented)

L-ADR-0007 Point-dependent scalarisations are localised before use was accepted on 2026-10-25. Its implementation in energy adds AbstractPointScalarisation, localise and SampsonNorm. The pointwise path localises before scalarising; free-energy scalarisation is unchanged. Graded scalarisations containing point-dependent parts are rejected until their parts can be given their own factor and arguments.

For and , a factor can override sampson_damping with . For any data covariance of the right dimension,

Under , residual and covariance transform by and , so is invariant when defined. For full-row-rank , PSD and , this covariance is SPD at rank-deficient too. Default identity damping lacks this law. The implementation requires : badly conditioned gauges can fail the numerical check even when exact-arithmetic energies agree.

Core tests check the circle closed form , its ratio tending to geometric energy , covariance scaling and singular rejection/damping. AlgebraicFactors tests use a test-only degree-two residual with two equations in two variables, seed 2307, 20 random points and a well-conditioned row transformation. They check gauge and QR-representative agreement (rtol=1e-10), including damping at a constructed singular point, and demonstrate failures of identity damping and algebraic distance (difference above 1e-3 at some point). Those original §2.3 tests remain. The real PolynomialFactor tests now check its exact Jacobian, row gauges at λ=0 and 0.3, anisotropic covariance and a singular point. Undamped monomial/Chebyshev energies agree. The coefficient damping ΘΘᵀ is row-gauge compatible but changes under nonorthogonal polynomial basis conversions; polynomial states that limitation.

The honest summary

distancefitstatistical interpretation
algebraic global spectral solution under orthonormal-row constraintdepends on residual/feature scaling
Sampson non-convex; full-gradient descent, RTR, or k=1 FNSfirst-order geometric correction
geometric non-convex projection problemprofile errors-in-variables MLE under isotropic Gaussian noise

fit_sampson initialises at fit_nullspace by default. The seeded noisy-conic test lowers the full Sampson cost, checks the gradient and random local directions, and compares all three methods. RTR reaches tol=1e-7 in four iterations on the unscaled ellipse, its X/2 variant, and the noisy two-quadric k=2 curve; polyfit records costs, gradient norms and budgets. These checks do not certify a global minimum. Algebraic and Sampson energies use the same residual, with a fixed metric versus a field of metrics. Exact geometric projection additionally needs the variety and a projection solver; it is not in general a function of the residual vector alone. This refines the reweighting discussion in Scalar and Multivariate Energy §6.4.

Related: Fitting is a Nullspace Problem, Scalar and Multivariate Energy, Branches and the Discriminant, The Algebraic Factor as a Statistical Game