implementation derivation

Grassmann(k, m) represents a real row space by an orthonormal row matrix. Retracted wraps Mycelium’s gradient descent and momentum with horizontal projection, positive-diagonal QR retraction and projection of the velocity.

Sources: Edelman, Arias & Smith (1998), The Geometry of Algorithms with Orthogonality Constraints; Absil, Mahony & Sepulchre (2008), Optimization Algorithms on Matrix Manifolds; implementation of L-SPEC-004 Algebraic Factors Toric and Grassmannian §2.1–§2.2; code: grassmann.jl, ../test/runtests.jl, ../../Mycelium.jl/src/factors.jl.

Bibliography: Edelman et al. 1998 · Absil et al. 2008

Theory (CT-ML wiki): Parametric Lens · Gradient Based Learning with Parametric Lenses

Representation and metric

A real -plane in is represented by with . The constructor requires . Orthogonal changes of row basis represent the same point. This is a Stiefel representative modulo , not a coordinate chart. The horizontal lift of the Grassmann tangent space and its Frobenius orthogonal projection are

The full Stiefel tangent space also contains vertical basis rotations. These are removed here. For row-space-invariant objectives, projecting the Euclidean gradient gives the Riemannian gradient for the canonical Grassmann metric. No Fisher metric for a statistical model is computed or identified with it. Parenthesising the two products avoids forming an projector: the dense projection costs .

Geometry operations check shapes but assume an orthonormal base point. ispoint(M, Θ; atol=1e-8) checks shape, finiteness and orthonormality with zero relative tolerance. It is a separate check rather than an extra matrix product inside every update. The implementation targets dense real CPU arrays; no GPU or AD contract is supplied.

QR and its sign ambiguity

For , retain thin and multiply its columns by . The corresponding diagonal of is positive. Without this fix, even the zero step can flip rows of the representative and invalidate a stored velocity’s basis convention. With the fix, an already orthonormal matrix retracts to itself to roundoff. Arbitrary displacements are accepted as row orthonormalisation; the update rule supplies horizontal ones. Only the thin Q is materialised, not a full matrix.

Non-finite inputs throw ArgumentError. A QR diagonal with any abs(R[i,i]) <= max(k,m)*eps(T)*norm(R, Inf) also throws. This is a floating-point diagonal rank criterion, not an exact rank certificate or a rank-revealing pivoted factorisation. A horizontal step from an exact point is full row rank because

rand(rng, M) uses the same sign fix on a standard Gaussian matrix and returns a dense Float64 representative with Haar-distributed row space. rand(M) uses Random’s default RNG. Tests check orthonormality, reproducibility and the QR convention, not a statistical goodness-of-fit test for the Haar distribution.

Mycelium rule and state contract

Retracted(rule, M) accepts only GradientDescent and Momentum; unsupported AbstractUpdateRules throw at construction. Nesterov’s forward look-ahead is off the manifold and needs a separate retraction/gradient-transport contract. rule_get is the identity on the representative. rule_put computes

Gradient descent’s state is passed through unchanged, usually nothing. Momentum’s state is exactly Mycelium’s plain parameter-shaped velocity matrix, not a tuple or a wrapped state. Its recurrence is (heavy-ball momentum; Mycelium had until LP-015 corrected it). After retraction, return as its new state: projection transport onto the new representative’s horizontal space. No differentiated retraction, parallel transport or isometry is claimed.

Mycelium has no rule-state initializer. Callers initialise momentum with zero(Θ) and GD with nothing, exactly as its Euclidean rules are driven. OptimiserFactor(:θ, rule) emits rule_get(rule, st.state, st.p) from state (p=Θ, state=s); optimiser_step calls rule_put separately from inference. No Mycelium change is needed.

The integration difficulties were the QR sign convention and the state layout: returning an equivalent but sign-flipped row basis breaks zero-step identity, and replacing Mycelium’s velocity by a conventional positive- recurrence would silently change the wrapped optimiser. Tests check the candidate and projected velocity against an independently formed projector, and check the factor-driven and direct trajectories at every step.

Checked by and limits

The Grassmann geometry and Retracted rules and eigenplane descent testsets in ../test/runtests.jl check:

  • rectangular, line and full-space points, zero-step identity, positive QR diagonal, horizontal projection and its idempotence, rank-deficient and non-finite failures, shape errors, invalid dimensions and Float32 retraction;
  • row-basis invariance of and covariance of its projected gradient; at step , the objective decrease agrees with within relative tolerance 2e-5;
  • a seeded symmetric matrix with spectrum , and gap . Both GD and momentum use η=0.04 < 1/(2*opnorm(S))=0.1 for 400 steps, momentum with γ=0.6. From the seeded random start, the largest principal angle to the span of the first two eigenvectors is below 1e-6 radians and the objective agrees with to absolute tolerance 2e-12;
  • the same iterates through OptimiserFactor / optimiser_step, identity emission, unchanged GD state, momentum tangency after every step, the inherited recurrence and projection transport, and rejection of Nesterov and another unsupported rule.

The step size and iteration budget are demonstrated for this fixed test problem; no global convergence theorem for the wrapped momentum is claimed. Principal angles are computed in the tests by singular values of the overlap; there is no public distance routine. §2.3’s Sampson scalarisation, exposed manifold channel spaces, polynomial factors, variable-rank models and beliefs on the Grassmannian remain outside this implementation.