definition

CategoricalBelief(p) is a distribution over a finite set of states: a vector of probabilities, stored as log-probabilities. Pooling two of them is elementwise multiplication and renormalisation: exact, associative, commutative, and always defined unless their supports are disjoint. bernoulli(p) is the two-state case.

Sources: code: open_model.jl (the type), messages.jl and free_energy.jl (its operations)

Theory (CT-ML wiki): Markov Category · Bayesian Inversion

Where it comes from

  • Discrete variables: “is this enzyme active?” (Metabolomics and Proteomics), a match outcome (win, draw, loss), a regime switch, a token at one position of a text (Language Models §3).
  • The predictive of a non-Gaussian likelihood: a Gaussian belief pushed through a count or ordinal likelihood becomes a categorical belief over the outcomes, labelled by them, with the tail of a count lumped into its top value (likelihood §3).
  • The switch of a mixture: the categorical variable that says which component of a Mixture Belief a value came from.
  • Logic factors: implication, AND, OR between Bernoulli variables are factors whose messages are categorical beliefs (Belief Algebra §2; the factors themselves are not built).

How it behaves

operationresult
combine with a categorical over the same stateselementwise product, renormalised; disjoint supports are an error
combine with a TrivialBelief / DiracBeliefthe other / the Dirac
combine with a SampleBelief of statesthe samples, reweighted by the probabilities
belief_logdensity(b, k)\log p_k; a label instead of an index if the belief has labels
variable_entropyShannon entropy -\sum_k p_k\log p_k (with 0\log 0 = 0)
belief_distancetotal variation \tfrac12\sum_k\lvert p_k - q_k\rvert
dampthe convex combination of the probability vectors

It is the easy belief type: the blocker that pooling general beliefs needs densities does not arise, because on a finite set the density is a table. The cost is size: one number per state, which is fine for a switch and large for a vocabulary.

Related: Beliefs, Belief Algebra, Mixture Belief, Dirac Belief, Sample Belief