Supervised Topology: Semantic Potentials and Metric Segmentation

Supervised Topology: Semantic Potentials and Metric Segmentation#

TLDR#

  • Supervision is not just labels; it is a geometric constraint that shapes the latent metric and partitions state space into semantic regions.

  • Class structure appears as potentials and jump structure: “belong together” becomes “live near each other” in \(\mathcal{Z}\).

  • This chapter links metric learning/contrastive ideas to the broader geometry/control story: classification is a topological segmentation problem.

  • Use the derived losses and diagnostics to detect when semantics are collapsing (merged classes) or fragmenting (over-segmentation).

  • Outputs: a principled interpretation of supervised losses as semantic field constraints, not heuristics.

Roadmap#

  1. Why classification should be read as geometry/topology.

  2. Semantic potentials, metric segmentation, and trainable losses.

  3. Diagnostics: what “good semantic structure” looks like in \(\mathcal{Z}\).

Here’s a question that sounds simple but has kept me up at night: What does it mean to classify something?

The standard story goes like this: you have a picture of a cat, you feed it to a neural network, out pops “cat.” Classification done. But wait—what actually happened in there? What does it mean for the network to “know” it’s a cat?

The usual answer is something about decision boundaries and hyperplanes separating feature vectors. And that’s fine as a computational description, but it misses something profound. Classification isn’t just about drawing lines between categories. It’s about organizing your understanding of the world so that things that behave similarly end up near each other, and things that behave differently end up far apart.

In this section, we’re going to take that intuition seriously. We’re going to see that class labels aren’t just targets for prediction—they’re geometric constraints on the shape of the agent’s internal representation. When you label a bunch of examples as “cat,” you’re not just providing training signal. You’re telling the agent: “These things belong together. They should live in the same neighborhood of your mental map.”

And here’s the beautiful part: this geometric view connects classification to everything else we’ve been building. The same metric structure that governs how the agent moves through latent space, the same potential landscapes that guide decisions, the same chart-based organization—all of it gets woven together with supervision into a unified picture.

Researcher Bridge: Metric Learning for Classification

This section recasts supervised labels as geometric constraints. It is the same idea as contrastive or metric learning, but expressed as class-conditioned potentials and separation in the latent manifold.

We rigorously define the role of discrete class labels \(\mathcal{Y}\) within the continuous latent geometry. Rather than treating classification as “predicting a target variable,” we use the Manifold Hypothesis [Carlsson, 2009] as a modeling premise: the representation is trained so that class labels can be represented by coherent regions. Under the stated smoothness and dynamical hypotheses, classification can then be read as equilibrium chart assignment under a class-conditioned flow.

This section extends the context-conditioned framework of The Context Space: Unified Definition, providing the topological constraints that make classification geometrically meaningful. The approach integrates ideas from topological data analysis [Carlsson, 2009], mixture-of-experts routing [Shazeer et al., 2017], hyperbolic embeddings [Nickel and Kiela, 2017], and Riemannian optimization [Bonnabel, 2013].

Relationship to the Context-Conditioned Framework#

Before diving into the formalism, let me give you the big picture.

We’ve already established that classification can be viewed as selecting a context \(c\) from the label space \(\mathcal{Y}\)—that’s what Section 23.6 was about. The effective potential \(\Phi_{\text{eff}} = -\log p(y|z)\) tells us how “expensive” it is to be at position \(z\) if we’re trying to be class \(y\).

But that’s just the beginning. The potential alone doesn’t tell us whether our classification system is well-organized. We could have a valid potential landscape but still have a mess—different classes all jumbled together, no clear separation, brittle boundaries.

What we need are topological constraints: rules that encourage geometric sanity. These constraints say things like: “Charts shouldn’t be confused about what class they represent,” and “Different classes should be separated in the chosen metric,” and, under the relevant invariance and barrier assumptions, “If you start in the cat region and flow downhill, you should stay in the cat region.”

Think of it like city planning. The potential landscape is like the terrain—hills and valleys. But good city planning also requires that neighborhoods be coherent (you don’t want houses interleaved with factories), that districts be separated by clear boundaries, and that traffic flows smoothly within regions. The topological constraints are the zoning laws.

Remark 24 (Extension, Not Replacement)

The Context Space: Unified Definition establishes classification as selecting a context \(c \in \mathcal{Y}\) (the label space), with cross-entropy cost \(-\log p(y|z)\) (Definition Definition 92). This section specifies the topological constraints that enforce geometric coherence of this classification:

  1. Charts should be semantically pure (one class per chart, modulo transition regions)

  2. Different classes should be metrically separated (long geodesics between class regions)

  3. Classification should be stable under dynamics (regions of attraction)

Now let’s get precise. What does it mean for the atlas of charts to be “organized” with respect to class labels?

Definition 107 (Semantic Partition)

Let \(\mathcal{Y} = \{1, \ldots, C\}\) be the set of class labels and \(\mathcal{K}\) the macro-state register (Definition 2.2.1). A labeling \(Y: \mathcal{X} \to \mathcal{Y}\) induces a soft partition of the chart atlas:

\[ \mathcal{A}_y := \{k \in \mathcal{K} : P(Y=y \mid K=k) > 1 - \epsilon_{\text{purity}}\},\]

where \(\epsilon_{\text{purity}} \in (0, 0.5)\) is the purity threshold. Define the transition-chart set

\[ \mathcal{T}:=\mathcal{K}\setminus\bigcup_{y\in\mathcal{Y}}\mathcal{A}_y = \left\{k:\max_y P(Y=y\mid K=k)\le 1-\epsilon_{\text{purity}}\right\}. \]

Interpretation: \(\mathcal{A}_y\) is the sub-atlas of charts predominantly associated with class \(y\). A chart \(k\) belongs to \(\mathcal{A}_y\) if, given that a sample routes to chart \(k\), the probability of class \(y\) exceeds \(1 - \epsilon_{\text{purity}}\).

Let me make sure this is crystal clear. We have charts—local coordinate systems that tile the latent manifold. And we have class labels—cat, dog, car, whatever. The semantic partition asks: “Which charts are ‘cat charts’? Which are ‘dog charts’?”

A chart \(k\) belongs to the cat sub-atlas if, whenever a sample ends up being handled by chart \(k\), it’s almost always a cat. The “almost always” is controlled by \(\epsilon_{\text{purity}}\)—if we set it to 0.1, then chart \(k\) is a cat chart if at least 90% of the samples routed to it are cats.

The word soft refers to the probabilistic association and the threshold, not to overlapping sub-atlases. Because \(\epsilon_{\text{purity}}<1/2\), a chart cannot pass the threshold for two classes. Charts in the complement \(\mathcal{T}\) simply fail the selected purity threshold. They may be near a decision boundary, but calling one a saddle or an unstable transition requires a separate smooth-potential and dynamical argument.

Definition 108 (Transition Chart Set)

For \(\epsilon_{\text{purity}}<1/2\), the sets \(\mathcal{A}_y\) are pairwise disjoint, because two conditional probabilities cannot both exceed \(1-\epsilon_{\text{purity}}>1/2\). The transition charts are the complement \(\mathcal{T}\) defined above. They are charts whose dominant class does not meet the selected purity threshold; high conditional entropy is an optional diagnostic, not part of the definition.

Remark (Geometric Interpretation). A transition chart may lie near a decision boundary, but this does not by itself make it a saddle or an unstable fixed point; those properties require a specified smooth potential and a dynamical analysis.

Cross-references: The Context Space: Unified Definition (Context-Conditioned Policies), Definition 2.2.1 (Macro-State Register), Tier 6: The Attentive Atlas (Permutation-Equivariant Routing) (Router Weights).

The Geography of Confusion

Think of the latent space as a landscape with mountain ranges separating different kingdoms (classes). Each chart is like a town.

  • Pure charts are towns whose routed data meet the selected class-purity threshold.

  • Transition charts are towns in the complement \(\mathcal{T}\): their empirical class distribution does not meet that threshold. They may be border towns, but the definition does not say that they sit at a saddle or that their dynamics are unstable.

The mountain-pass picture becomes valid only after a smooth potential and a dynamical analysis supply it. Hard examples may cluster near such a pass in a particular trained model, but that is an empirical observation to test, not a consequence of the partition definition.

The Semantic Potential#

We embed class labels into the dynamics via a class-conditioned potential that shapes the energy landscape.

Here’s the key idea: we already have a potential landscape \(V_{\text{base}}\) that the agent uses for navigation and decision-making. The semantic term first mixes the chart-level class scores with the soft router at the current point \(z\), and then applies the negative logarithm. The result can vary smoothly with \(z\), so its gradient can actually depend on the requested class.

Think of it like coloring a topographic map. The base potential gives you the elevation—the hills and valleys. The soft semantic term paints those valleys in different colors: blue for cat, red for dog, green for car. A cat-conditioned potential can lower the score in cat-associated regions, subject to the regularity and stability assumptions used by the dynamics.

Definition 109 (Class-Conditioned Potential)

Given a target class \(y \in \mathcal{Y}\) and differentiable soft-router weights \(w_k(z)\), define the semantic potential:

\[ V_y(z) := -\beta_{\text{class}} \log\left(\sum_{k=1}^{N_c} w_k(z)P(Y=y \mid K=k)\right) + V_{\text{base}}(z),\]

where:

  • \(P(Y=y \mid K=k) = \text{softmax}(\Theta_{k,:})_y\) with learnable parameters \(\Theta \in \mathbb{R}^{N_c \times C}\)

  • \(V_{\text{base}}(z)\) is the unconditioned critic (The HJB Correspondence (Costs as Value Updates))

  • \(\beta_{\text{class}} > 0\) is the class temperature (inverse of semantic diffusion)

  • Units: \([V_y] = \mathrm{nat}\)

Remark (Chart-to-Class Mapping). The learnable parameter \(\Theta_{k,y}\) represents the log-affinity of chart \(k\) for class \(y\). The soft mixture is differentiable in \(z\); replacing \(w_k(z)\) by a hard chart index would make the semantic term piecewise constant and remove its class-dependent continuous drift.

Remark (Alternative: Empirical Estimation). Instead of learnable parameters, one may estimate \(P(Y|K)\) empirically via exponential moving average:

\[ \hat{P}(Y=y \mid K=k) = \frac{\text{EMA}[\mathbb{I}[Y=y, K=k]]}{\text{EMA}[\mathbb{I}[K=k]]}.\]

This is non-differentiable w.r.t. chart assignment but more grounded in observations. A hybrid approach initializes learnable \(\Theta\) from empirical estimates after warmup.

Let me unpack this formula piece by piece.

The formal definition first forms the soft-router score \(\sum_k w_k(z)P(Y=y\mid K=k)\) and then takes its negative logarithm. A high score for class \(y\) lowers the semantic contribution, but a low value of the total potential is a candidate for a stable state only when the dynamics supply a stable minimum. The potential value alone does not prove stability or convergence.

The coefficient \(\beta_{\text{class}}\) controls how strongly the semantic score competes with \(V_{\text{base}}\). A larger value gives the class term more influence; a smaller value leaves more of the landscape to the base potential. Which class wins still depends on the learned base potential, router, and the hypotheses in the relevant theorem.

There are two distributions that are easy to confuse. The chart-level model scores may come from a learnable matrix \(\Theta\), with a softmax row for each chart. The purity calculation uses the empirical conditional distribution of labels given the routed chart, estimated from a batch or an EMA. These quantities can be initialized from one another, but they are not automatically the same object.

Why Logarithms?

The logarithm in \(-\log P(Y=y|K)\) isn’t arbitrary. There are deep reasons why it’s the right choice:

  1. Information-theoretic: \(-\log P\) is the surprisal or self-information. The semantic potential measures how “surprising” it is to find class \(y\) in chart \(K\). We want to minimize surprise.

  2. Additive composition: If you have independent pieces of evidence, their log-probabilities add. Potentials add. The logarithm makes evidence composition work naturally with potential composition.

  3. Scale-appropriate gradients: When the score depends on \(z\), \(\nabla(-\log P) = -\nabla P/P\) makes changes in a small probability visible. This can help expose poorly represented classes, although class imbalance still requires its own data and optimization checks.

  4. Connection to free energy: For a Gibbs law \(P(x)=e^{-\beta E(x)}/Z\), the surprisal is \(-\log P(x)=\beta(E(x)-F)\), where \(F=-\beta^{-1}\log Z\) is the free energy. The class temperature plays the role of the inverse-temperature scale in this dimensionless score.

Definition 110 (Region of Attraction)

The region of attraction for class \(y\) is:

\[ \mathcal{B}_y := \left\{z \in \mathcal{Z}: \lim_{t\to\infty}\phi_t(z)\ \text{exists and}\ K\!\left(\lim_{t\to\infty}\phi_t(z)\right)\in\mathcal{A}_y\right\},\]

where \(\phi_t\) denotes the flow of the curl-corrected system

\[ \dot{z} = \mathcal{M}_{\text{curl}}\!\left(-G^{-1}(z)\nabla V_y(z)\right), \qquad \mathcal{M}_{\text{curl}} := (I - \beta_{\text{curl}} G^{-1}\mathcal{F})^{-1} \]

(conservative case: \(\mathcal{F}=0\)). Here \(\nabla V_y\) is the ordinary differential of the smooth semantic potential. A non-conservative reward component can be included through the separately defined curl mobility; the class-conditioning statement below is made in the conservative case \(\mathcal{F}=0\).

Interpretation: \(\mathcal{B}_y\) is the set of initial conditions from which the deterministic gradient flow on \(V_y\) converges to the class-\(y\) region.

This is one of my favorite definitions because it turns classification into physics.

Imagine you drop a ball somewhere on the potential landscape \(V_y\). In the deterministic idealization the ball rolls according to the specified metric and force field. The region of attraction \(\mathcal{B}_y\) is the set of starting points whose limiting point, when it exists, maps to the class-\(y\) region. Reaching a single local minimum is an additional property of the potential, not part of the picture by itself.

Notice the Riemannian metric \(G^{-1}\) in the dynamics. The gradient isn’t in Euclidean space—it respects the curved geometry of the latent manifold. If you’re in a region where the metric is “stretched” (high \(G\)), you move more slowly. The geometry regulates the flow.

This is why I keep saying “classification can be relaxation.” You encode an input into latent space, then let the specified dynamics evolve it. In a deterministic regime, the eventual chart assignment can be read as a relaxed classification. With noise or a non-conservative field, the theorem must say what is controlled—for example a finite-horizon retention probability or an omega-limit set.

Proposition 32 (Conditional Classification Relaxation)

Under the conservative, deterministic overdamped dynamics (The Overdamped Limit) with the smooth potential \(V_y\):

\[ dz = -G^{-1}(z)\nabla V_y(z)\,ds, \qquad T_c=0.\]

The limiting chart assignment satisfies, whenever the trajectory converges:

\[ K\!\left(\lim_{s \to \infty} z(s)\right) \in \mathcal{A}_y,\]

provided:

  1. \(z(0) \in \mathcal{B}_y\) (initial condition in the basin)

  2. the trajectory remains in the domain of the smooth router and converges to a local minimum of \(V_y\)

  3. that limiting minimum lies in \(K^{-1}(\mathcal{A}_y)\).

Proof sketch. Use \(L(z):=V_y(z)\). Under the conservative overdamped dynamics,

\[ \frac{dL}{ds} = dV_y(\dot z) = -\nabla V_y^{\mathsf T}G^{-1}\nabla V_y \le 0.\]

This proves descent until a limiting critical set; it does not imply global convergence, positive-temperature almost-sure convergence, or that a class-\(y\) minimum exists without the stated hypotheses. \(\square\)

There’s something deeply satisfying about this result, provided we keep its quantifiers straight. At zero temperature, starting in \(\mathcal{B}_y\) gives the deterministic basin statement encoded by that definition. At positive temperature, this theorem says nothing by itself; a finite-horizon exit estimate would need its own barrier and diffusion hypotheses, and it would not make a noisy trajectory settle permanently with probability one.

The assumptions are worth thinking about:

  1. Starting in the basin: This is the job of the encoder. The encoder must map inputs into latent positions covered by the theorem’s basin condition. The relaxation dynamics then refine this initial guess.

  2. Noise and horizon: Temperature \(T_c\) controls the noise level. A separate finite-horizon result may make an exit probability small when \(T_c\) is small, but any positive noise can eventually produce an exit unless additional structure rules it out.

  3. Finite barriers and regularity: A barrier estimate needs a specified potential, invariant region, and regularity assumptions. It controls the stated probability or limit set; it does not by itself establish a unique global class minimum.

The Lyapunov and La Salle tools describe the deterministic conservative part under their hypotheses. The stochastic argument then adds a finite-horizon estimate. This is the useful statement: a controlled retention probability on a chosen time scale, rather than an unconditional almost-sure convergence claim.

Corollary 10 (Inference via Relaxation)

Classification inference proceeds as:

  1. Encode: \(z_0 = \text{Enc}(x)\)

  2. Relax under neutral potential \(V_{\text{base}}\) (no class conditioning) to equilibrium \(z^*\)

  3. Read out: \(\hat{y} = \arg\max_y P(Y=y \mid K(z^*))\)

Remark (Fast Path). In practice, we often skip the relaxation and use direct readout: \(\hat{y} = \arg\max_y \sum_k w_k(x) \cdot P(Y=y \mid K=k)\), where \(w_k(x)\) are the router weights (Tier 6: The Attentive Atlas (Permutation-Equivariant Routing) (Router Weights)). This is an operational classifier; identifying it with a zero-temperature, infinite-time limit requires additional convergence and calibration assumptions.

Cross-references: The Overdamped Limit (Overdamped Limit), Definition Definition 72, The HJB Correspondence (Costs as Value Updates) (Critic).

The Two Paths to Classification

Let me contrast the relaxation path described by the corollary with the fast path:

The Relaxation Path (Theoretically Clean):

Input x → Encode to z₀ → Evolve the specified flow → Read K(z*) when the limit exists → Predict argmax P(Y|K)

This path uses the metric and potential in the formal dynamics. Its limit and class assignment are conditional on the stated basin, smoothness, and convergence hypotheses. It requires running a dynamical system, which takes time.

The Fast Path (Practically Useful):

Input x → Compute router weights w(x) → Predict argmax Σₖ wₖ P(Y|K=k)

This skips the relaxation entirely. You just use the soft router weights to aggregate class predictions from each chart.

Why can the fast path work? It evaluates the router-weighted class scores at the encoded point and is therefore cheap. It agrees with the relaxed readout only under an additional condition, such as no meaningful router change or chart-boundary crossing along the relaxation. The limits \(T_c \to 0\) and \(s \to \infty\) do not themselves move the weights that were evaluated at the initial point.

In practice, compare the two paths on validation data. The relaxation picture is useful for training and for edge cases near basin boundaries, while the fast path is an operational approximation whose calibration must be checked.

Connection to RL #21: Imitation Learning as a Supervised Topology Analogy

The General Law (Fragile Agent): Class labels define Semantic Partitions with Class-Conditioned Potentials:

\[ V_y(z) = -\beta_{\text{class}} \log\left(\sum_k w_k(z)P(Y=y\mid K=k)\right) + V_{\text{base}}(z)\]

Under the conditional deterministic hypotheses above, trajectories can relax into class-specific basins via gradient flow on the learned metric.

The Degenerate Limit: Set \(V_{\text{base}} = 0\). Interpret labels as expert actions. Use Euclidean minimization.

The Special Case (Standard RL):

\[ \mathcal{L}_{\text{BC}} = \mathbb{E}_{(s,a^*) \sim \mathcal{D}_{\text{expert}}}[-\log \pi(a^*|s)]\]

This recovers Behavioral Cloning and Imitation Learning [Pomerleau, 1991].

What the generalization offers:

  • Metric segmentation: Classes are metrically separated on the learned manifold

  • Potential landscape: \(V_y\) creates basins of attraction, not just classification boundaries

  • Jump modulation: Cross-class transitions suppressed by separation penalty \(\gamma_{\text{sep}}\)

  • Relaxation dynamics: Classification emerges from physics, not discrete argmax

Metric Segmentation via Jump Rate Modulation#

Now we come to one of the cleverest ideas in this framework: encouraging class separation not by directly pushing embeddings apart, but by making it hard to jump between class regions in the declared transition dynamics.

Think about it like this. You could try to separate cats from dogs by pushing all the cat embeddings in one direction and all the dog embeddings in another. That’s what contrastive learning does, and it works. But there’s another approach: make the metric very “expensive” to traverse between cat territory and dog territory. You don’t push them apart—you build a wall between them.

The wall isn’t literal. It’s implemented through the jump rates between charts. Remember, our latent space is tiled with charts, and transitions between charts happen via “jumps.” The key insight is: we can modulate how fast those declared jumps happen based on whether the source and destination charts belong to the same class.

If both charts are cat charts, leave their base rate alone. If you’re trying to jump from a cat chart to a dog chart, suppress that edge exponentially. This creates an operational barrier in the rate-weighted transition graph without claiming that the underlying continuous latent manifold has been cut apart.

Definition 111 (Class-Consistent Jump Rate)

For the WFR reaction term (Definition Definition 53), modulate the inter-chart transition rate:

\[ \lambda_{i \to j}^{\text{sup}} := \lambda_{i \to j}^{(0)} \cdot \exp\left(-\gamma_{\text{sep}} \cdot D_{\text{class}}(i, j)\right),\]

where:

  • \(\lambda^{(0)}_{i \to j}\) is the base transition rate from the GKSL master equation (Definition 43, [Gorini et al., 1976, Lindblad, 1976], Connection to GKSL / Master Equation (Section 12.5)), derived from the overlap consistency of jump operators (Section 7.13)

  • \(\gamma_{\text{sep}} \geq 0\) is the separation strength (hyperparameter)

  • \(D_{\text{class}}(i, j) = \mathbb{I}[\text{Class}(i) \neq \text{Class}(j)]\) is the class disagreement indicator

  • \(\text{Class}(k) := \arg\max_y P(Y=y \mid K=k)\) is the dominant class of chart \(k\)

Remark (Rate vs Operator). Factorized Jump Operators: Efficient Chart Transitions defines the transition function \(L_{i \to j}\) (the coordinate change map). The transition rate \(\lambda_{i \to j}\) is a separate quantity from the GKSL/master equation framework (Connection to GKSL / Master Equation (Section 12.5), Equation 20.5.2) that governs how often jumps occur, not where they go. The rate is typically derived from the overlap structure: \(\lambda_{i \to j}^{(0)} \propto \mathbb{E}_{x}[w_i(x) w_j(x)]\), measuring how much probability mass lies in the overlap \(U_i \cap U_j\).

Interpretation: Transitions between charts of the same class proceed at the base rate \(\lambda^{(0)}\). Transitions between charts of different classes are exponentially suppressed by factor \(e^{-\gamma_{\text{sep}}}\).

Let me walk through the formula. The base rate \(\lambda^{(0)}_{i \to j}\) is determined by geometry—roughly, how much the charts overlap. Charts that share a lot of territory have high transition rates; charts that barely touch have low rates. This is the unsupervised part, determined by the manifold structure.

The class-modulation factor is applied on the candidate transition set. Within that set, same-class transitions form one subset and cross-class transitions are its complement. Thus the class disagreement is zero on the first subset and one on the second:

  • Same-class transition: the factor is \(e^0 = 1\), so the base rate is unchanged.

  • Cross-class transition: the factor is \(e^{-\gamma_{\text{sep}}}\), so the edge is suppressed.

Pairs outside the candidate transition set are simply not declared as edges by this rate law; they should not be silently interpreted as cross-class jumps.

The hyperparameter \(\gamma_{\text{sep}}\) controls how strong this suppression is. At \(\gamma_{\text{sep}} = 0\), there’s no suppression. At \(\gamma_{\text{sep}} = 10\), cross-class jumps are \(e^{-10} \approx 0.00005\) times as frequent as same-class jumps. That’s serious suppression.

A Toll Road Between Cities

Here’s an analogy that might help. Imagine the latent space as a country with cities (charts) and roads between them. The base transition rates \(\lambda^{(0)}\) are like the natural road connectivity—nearby cities have fast highways, distant cities require longer journeys.

Now suppose we want to separate two regions—say, Northern Territory (cats) and Southern Territory (dogs). We could physically move the cities apart, but that’s disruptive. Instead, we build toll booths on every road crossing the border. The toll is \(\gamma_{\text{sep}}\), and since time is money, the “effective distance” for crossing the border becomes much larger.

From a traveler’s perspective, cities within the same territory are still easy to reach. But getting to the other territory requires paying the toll, which makes the corresponding edge rare.

The exponential form \(e^{-\gamma_{\text{sep}}}\) comes from thinking about rates: the toll changes the waiting time or conductance of that graph edge. It is a useful kinetic analogy; it does not by itself define, or make divergent, the continuous WFR distance on the latent manifold.

Remark 25 (Transition-Rate Suppression Diagnostic)

Increasing \(\gamma_{\text{sep}}\) suppresses the prescribed cross-class jump rates:

\[ \frac{\lambda_{i\to j}^{\text{sup}}}{\lambda_{i\to j}^{(0)}}=e^{-\gamma_{\text{sep}}} \quad\text{when }\operatorname{Class}(i)\ne\operatorname{Class}(j).\]

This is a rate-suppression diagnostic. It does not imply that the WFR distance diverges: WFR paths include reaction controls, and a pure-reaction path can have finite Hellinger-type cost independently of \(\gamma_{\text{sep}}\). A geometric disconnection claim would require a different metric whose admissible paths explicitly exclude that reaction channel.

Read this remark carefully: it gives a quantitative suppression ratio for each prescribed cross-class edge. Increasing \(\gamma_{\text{sep}}\) makes that edge less conductive. This is a statement about the declared discrete jump channel, not yet a theorem about a path metric.

It is not a claim that the continuous WFR distance from Definition 53 becomes infinite. That functional has its own transport and reaction controls and contains no \(\gamma_{\text{sep}}\). Nor does a suppressed edge prove that the continuous latent manifold has different connected components. In the ideal limit \(\gamma_{\text{sep}}\to\infty\), an edge may disappear from the discrete graph; that graph limit still needs to be distinguished from a geometric impossibility of moving continuously.

This is still useful beyond a bare decision boundary: it assigns a measurable kinetic penalty to a declared class edge. Turning that penalty into a finite lower bound for a discrete metric would require the extra graph and positivity hypotheses stated by such a result; the rate formula alone does not supply them.

Remark 26 (Tunneling as Anomaly Detection)

Cross-class transitions are not forbidden, merely exponentially suppressed. A detected cross-class jump indicates:

  1. Anomaly: The sample lies in a transition region not well-covered by training

  2. Distribution shift: The test distribution differs from training

  3. Adversarial input: Deliberate perturbation to cross class boundaries

This provides a natural out-of-distribution detection mechanism: monitor the rate of cross-class transitions.

This is a useful diagnostic, but it is not an anomaly theorem. A cross-class jump records that the model’s dynamics used a declared edge. That can indicate an ambiguous example, distribution shift, an adversarial perturbation, or simply an imperfect class partition.

Monitoring the pressure for such jumps can therefore provide a warning signal. Interpret it together with purity, chart coverage, and calibration: a rise in cross-class traffic is evidence of tension in the learned representation, not proof of an out-of-distribution input.

The same caution applies to the word stability. A sample may leave a class region because of noise or a legitimate class overlap. The rate monitor measures the event; the diagnosis requires the surrounding hypotheses and data.

Definition 112 (Class-Modulated Jump Operator)

Modify the jump operator (Definition Definition 23) to incorporate class consistency:

def class_modulated_jump_rate(
    lambda_base: torch.Tensor,    # [N_c, N_c] base jump rates
    chart_to_class: torch.Tensor, # [N_c, C] learnable logits
    gamma_sep: float = 5.0,       # Separation strength
) -> torch.Tensor:
    """
    Compute class-modulated jump rates.

    Cross-ref:
        - Definition 25.3.1 (Class-Consistent Jump Rate)
        - Definition 7.13.1 (Jump Operator)
    """
    # Get dominant class per chart
    p_y_given_k = F.softmax(chart_to_class, dim=1)  # [N_c, C]
    dominant_class = p_y_given_k.argmax(dim=1)       # [N_c]

    # Compute class disagreement matrix
    class_match = (dominant_class.unsqueeze(1) == dominant_class.unsqueeze(0)).float()  # [N_c, N_c]
    D_class = 1.0 - class_match  # 1 if classes differ, 0 if same

    # Modulate rates
    lambda_sup = lambda_base * torch.exp(-gamma_sep * D_class)

    return lambda_sup

Cross-references: The WFR Metric (Benamou-Brenier Formulation) (WFR Metric), Definition Definition 23, Connection to GKSL / Master Equation (Section 12.5) (GKSL Connection).

The Supervised Topology Loss#

We define training losses that encourage the geometric structure described above.

So far we’ve described what we want: pure charts, separated classes, consistent routing. Now we need to turn those desires into loss functions that we can actually minimize.

The Supervised Topology Loss has four components, each penalizing a different failure mode. Think of it like a multi-objective fitness function: we want the representation to be good at predicting labels (route alignment), to have clean chart-class associations (purity), to use its capacity (balance), and to separate different classes according to the selected metric (metric separation). These penalties are training objectives; low loss alone is not a proof of a topological property.

Let me take you through each one.

Chart Purity Loss (Conditional Entropy)#

Definition 113 (Purity Loss)

The purity loss measures how well charts separate classes:

\[ \mathcal{L}_{\text{purity}} = \sum_{k=1}^{N_c} P(K=k) \cdot H(Y \mid K=k),\]

where:

  • \(P(K=k) = \mathbb{E}_{x \sim \mathcal{D}}[w_k(x)]\) is the marginal chart probability

  • \(H(Y \mid K=k) = -\sum_y P(Y=y \mid K=k) \log P(Y=y \mid K=k)\) is the class entropy within chart \(k\)

Interpretation: \(\mathcal{L}_{\text{purity}} = H(Y \mid K)\), the conditional entropy of class given chart. Minimizing this encourages each chart to be associated with a single class.

This loss asks: “If I tell you which chart a sample was routed through, how much uncertainty remains about its class?”

In the implementation, estimate the routed label conditional from the labelled batch or its EMA, for example \(\widehat p_{y|k}^{\mathrm{data}}\) from the router weights and one-hot labels. The purity term is the corresponding weighted empirical entropy. It is a batch/EMA estimate of \(H_{\mathrm{data}}(Y|K)\), not the model’s \(\Theta\)-row entropy and not automatically the population conditional entropy.

If the empirical routed distribution is perfectly pure (each used chart contains only one class), the estimated within-chart entropy is zero. If a chart is maximally mixed in the observed data, its estimated entropy is \(\log C\). These statements describe the selected sample or EMA distribution; they do not assert that an unseen data distribution has the same conditional law.

The weighting by the empirical \(\widehat P(K=k)\) is important. A rarely-used chart contributes less to the measured purity than a frequently-used one. The separate metric term uses the bounded router-overlap proxy in the definition, so neither term should be read as a direct WFR or jump distance without an additional estimator.

Proposition 33 (Purity-Information Duality)

Minimizing \(\mathcal{L}_{\text{purity}}\) is equivalent to maximizing the mutual information \(I(K; Y)\):

\[ \mathcal{L}_{\text{purity}} = H(Y) - I(K; Y).\]

Since \(H(Y)\) is fixed by the data, \(\min \mathcal{L}_{\text{purity}} \Leftrightarrow \max I(K; Y)\).

Why Mutual Information?

This proposition connects purity to information theory in a beautiful way, with one essential bookkeeping point.

The identity \(H_{\mathrm{data}}(Y) = H_{\mathrm{data}}(Y|K) + I_{\mathrm{data}}(K;Y)\) uses the empirical joint distribution of labels and routed charts. When the conditional entropy is computed from batch or EMA label conditionals, minimizing this empirical estimate is equivalent to maximizing the corresponding empirical mutual information for the fixed sample or tracked distribution.

Entropy of the learnable chart-score rows measures model confidence for the route predictor. It can make every row sharp without making chart assignment informative about the observed labels, so it is not a substitute for the routed-label statistic. The code and the statistic must therefore be checked together.

Load Balance Loss (Uniform Coverage)#

[Shazeer et al., 2017]

Definition 114 (Balance Loss)

Prevent degenerate solutions where all samples route to few charts:

\[ \mathcal{L}_{\text{balance}} = D_{\text{KL}}\left(\bar{w} \;\|\; \text{Uniform}(N_c)\right),\]

where \(\bar{w} = \mathbb{E}_{x \sim \mathcal{D}}[w(x)]\) is the average router weight vector.

Interpretation: Encourages all charts to be used, preventing “dead charts” and ensuring the atlas covers the label space.

This is the “anti-collapse” loss. Without it, the system might find that it’s easiest to just use one or two charts for everything. That’s technically a valid solution, but it wastes all the representational capacity you built into your model.

The balance loss measures how far the average routing distribution is from uniform. If every chart gets used equally often, \(\bar{w} = (1/N_c, \ldots, 1/N_c)\) and the KL divergence is zero. If all mass concentrates on one chart, the penalty is positive (in this convention it is \(\log N_c\), not infinite). The term discourages collapse; it cannot by itself guarantee that every chart is useful.

This is a well-known trick from the mixture-of-experts literature. Shazeer et al. found that without load balancing, expert networks collapse to using just a handful of experts. The same principle applies here: charts are like experts, and we need to encourage the system to use all of them.

The Expert Collapse Problem

Why does collapse happen without balancing? Here’s the intuition:

Early in training, some charts will be randomly better than others for some subset of the data. The router learns to send that data to those charts. But now those charts see more data and get even better. Meanwhile, the neglected charts see less data and stagnate. This positive feedback loop leads to “rich get richer” dynamics where a few charts dominate and the rest become useless.

The balance loss breaks this feedback. It says: “Yes, you can route to whichever chart works best, but I’m going to penalize you for being too uneven.” It encourages the system to use all charts, while the task loss and data determine whether that use is productive.

The trade-off controlled by \(\lambda_{\text{bal}}\) (typically small, like 0.01) is between routing quality and routing diversity. Too much balancing can force bad routings; too little can permit collapse.

Metric Contrastive Loss (Geometric Separation)#

[Khosla et al., 2020, Schroff et al., 2015]

Definition 115 (Contrastive Loss)

Enforce that different-class samples have low router overlap, using the following bounded proxy for separation:

\[ \mathcal{L}_{\text{metric}} = \frac{1}{|\mathcal{P}|} \sum_{(i,j) \in \mathcal{P}: y_i \neq y_j} (w_i^\top w_j)\,\max\!\left(0, m - \left(1-w_i^\top w_j\right)\right)^2,\]

where:

  • \(\mathcal{P}\) is the set of sample pairs in the batch

  • \(w_i, w_j\) are router weight vectors

  • \(m \in (0,1)\) is the margin for the bounded proxy

  • \(1-w_i^\top w_j\) is a router-overlap proxy, not a WFR or jump distance

Interpretation: If two samples have different labels but high router overlap (\(w_i^\top w_j\) large), the bounded proxy is below the margin and the loss penalizes the configuration. A genuine geometric distance would require a separately defined jump-cost estimator.

This loss says: “Different classes should be separated according to the distance or overlap proxy actually defined in the formula.”

The structure is classic contrastive learning, but with a twist. The term \(w_i^\top w_j\) measures how much two samples share routing. If they route through completely different charts, \(w_i^\top w_j \approx 0\) and there’s no penalty regardless of their distance. But if they route through similar charts (high overlap), then we demand that they be far apart if they have different labels.

When the formal distance \(d\) is used, the hinge form \(\max(0, m - d)^2\) is a margin loss: distances greater than \(m\) contribute zero loss, while distances smaller than \(m\) contribute a penalty that grows quadratically as you get closer. If code uses an overlap proxy instead, read this as the corresponding proxy objective rather than as a theorem about a geometric distance.

The margin \(m\) is a hyperparameter that sets a desired training target. Set it too high and you’re asking for more separation than the geometry can provide. Set it too low and classes can still be confusably close.

The Role of Router Overlap

The factor \(w_i^\top w_j\) is subtle but important. Why weight the contrastive penalty by routing similarity?

The answer is efficiency. Most pairs of different-class samples are already separated just by routing to different charts. A cat that routes 90% to chart 3 and a dog that routes 90% to chart 7 have low overlap (\(w_i^\top w_j \approx 0.1\)) and don’t need explicit pushing apart—the routing already separates them.

The pairs we care about are those with high routing overlap despite being different classes. These are the boundary cases, the hard examples, the potential confusions. By weighting by overlap, the loss focuses its effort on exactly the pairs that need work.

The weighting changes which pairs contribute most, but evaluating every pair is still \(O(N^2)\) unless the implementation samples or sparsifies pairs explicitly. The overlap factor is a prioritization signal, not a complexity guarantee.

Route Alignment Loss (Prediction Consistency)#

Definition 116 (Route Alignment Loss)

The primary classification loss:

\[ \mathcal{L}_{\text{route}} = \mathbb{E}_{x, y_{\text{true}}}\left[\text{CE}\left(\sum_k w_k(x) \cdot P(Y=\cdot \mid K=k), \; y_{\text{true}}\right)\right],\]

where \(\text{CE}\) denotes cross-entropy.

Interpretation: The predicted class distribution is the router-weighted average of per-chart class distributions. This must match the true label.

This is the “make correct predictions” loss—the supervised learning objective we’re all familiar with, just written in our chart-based language.

Here’s how prediction works: each chart \(k\) has a class distribution supplied by the model or by the empirical estimate specified in the formal definitions. The router gives us weights \(w_k(x)\) saying how much we trust each chart for this particular input. The final prediction is the weighted average: \(\sum_k w_k(x) \cdot P(Y|K=k)\).

The route alignment loss is just cross-entropy between this prediction and the true label. If the prediction is confident and correct, loss is low. If it’s confident and wrong, or uncertain about the right answer, loss is high.

This is the direct label-prediction loss. The other losses (purity, balance, metric) shape routing and geometry, but none of them guarantees correct generalization without data and optimization assumptions.

Combined Supervised Topology Loss#

Definition 117 (Total Loss)

The full supervised topology loss:

\[ \mathcal{L}_{\text{sup-topo}} = \mathcal{L}_{\text{route}} + \lambda_{\text{pur}} \mathcal{L}_{\text{purity}} + \lambda_{\text{bal}} \mathcal{L}_{\text{balance}} + \lambda_{\text{met}} \mathcal{L}_{\text{metric}}.\]

Typical hyperparameters: \(\lambda_{\text{pur}} = 0.1\), \(\lambda_{\text{bal}} = 0.01\), \(\lambda_{\text{met}} = 0.01\).

Algorithm 25.4.7 (SupervisedTopologyLoss Implementation).

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Dict


class SupervisedTopologyLoss(nn.Module):
    """
    Supervised topology loss enforcing chart purity, balance, and separation.

    Cross-ref:
        - Definition 25.4.6 (Total Loss)
        - {ref}`sec-tier-the-attentive-atlas` (Router Weights)
    """

    def __init__(
        self,
        num_charts: int,
        num_classes: int,
        lambda_purity: float = 0.1,
        lambda_balance: float = 0.01,
        lambda_metric: float = 0.01,
        margin: float = 0.5,
        temperature: float = 1.0,
    ):
        super().__init__()
        self.num_charts = num_charts
        self.num_classes = num_classes
        self.lambda_purity = lambda_purity
        self.lambda_balance = lambda_balance
        self.lambda_metric = lambda_metric
        self.margin = margin

        # Learnable chart-to-class mapping (Definition 25.2.1)
        self.chart_to_class = nn.Parameter(
            torch.randn(num_charts, num_classes) * 0.01
        )
        self.temperature = temperature

    @property
    def p_y_given_k(self) -> torch.Tensor:
        """P(Y|K) distribution [N_c, C]."""
        return F.softmax(self.chart_to_class / self.temperature, dim=1)

    def forward(
        self,
        router_weights: torch.Tensor,  # [B, N_c]
        y_true: torch.Tensor,          # [B] class labels
        z_latent: torch.Tensor = None, # [B, D] optional for metric loss
    ) -> Dict[str, torch.Tensor]:
        """
        Compute supervised topology losses.

        Returns dict with individual losses and total.
        """
        B = router_weights.shape[0]
        p_y_k = self.p_y_given_k  # [N_c, C]

        # === Route Alignment Loss (Definition 25.4.5) ===
        # P(Y|x) = sum_k w_k(x) * P(Y|K=k)
        p_y_x = torch.matmul(router_weights, p_y_k)  # [B, C]
        loss_route = F.cross_entropy(
            torch.log(p_y_x + 1e-8), y_true
        )

        # === Purity Loss (Definition 25.4.1) ===
        # Estimate P(Y|K) from the labelled batch, rather than from the
        # learnable chart-to-class prior used by the route predictor.
        p_k = router_weights.mean(dim=0)  # [N_c]
        y_one_hot = F.one_hot(y_true, num_classes=self.num_classes).to(router_weights.dtype)
        p_y_given_k_emp = torch.matmul(router_weights.t(), y_one_hot)
        p_y_given_k_emp = p_y_given_k_emp / (p_k.unsqueeze(1) + 1e-8)
        entropy_per_chart = -(p_y_given_k_emp * torch.log(p_y_given_k_emp + 1e-8)).sum(dim=1)
        # L_purity = sum_k P(K=k) * H(Y|K=k)
        loss_purity = (p_k * entropy_per_chart).sum()

        # === Balance Loss (Definition 25.4.3) ===
        # KL(p_k || Uniform) = sum_k p_k * log(p_k / (1/N_c)) = sum_k p_k * (log(p_k) + log(N_c))
        uniform = torch.ones_like(p_k) / self.num_charts
        # Manual KL computation: KL(P||Q) = sum P * log(P/Q)
        loss_balance = (p_k * (torch.log(p_k + 1e-8) - torch.log(uniform))).sum()

        # === Metric Contrastive Loss (Definition 25.4.4) ===
        loss_metric = torch.tensor(0.0, device=router_weights.device)
        if self.lambda_metric > 0 and B > 1:
            # Router overlap as proxy for proximity
            # w_i^T w_j measures routing similarity
            overlap = torch.matmul(router_weights, router_weights.t())  # [B, B]

            # Class disagreement mask
            y_match = (y_true.unsqueeze(1) == y_true.unsqueeze(0)).float()
            y_diff = 1.0 - y_match  # 1 if different classes

            # Penalize high overlap for different-class pairs
            # Use bounded router-overlap proxy; this is not a WFR distance.
            pseudo_dist = 1.0 - overlap
            hinge = F.relu(self.margin - pseudo_dist)
            loss_metric = (y_diff * overlap * hinge ** 2).sum() / (y_diff.sum() + 1e-8)

        # === Total Loss ===
        loss_total = (
            loss_route
            + self.lambda_purity * loss_purity
            + self.lambda_balance * loss_balance
            + self.lambda_metric * loss_metric
        )

        return {
            'loss_total': loss_total,
            'loss_route': loss_route,
            'loss_purity': loss_purity,
            'loss_balance': loss_balance,
            'loss_metric': loss_metric,
        }

Cross-references: Tier 6: The Attentive Atlas (Permutation-Equivariant Routing) (Router Weights), Section 7.13 (Jump Operators), Diagnostics: Stability Checks (Monitors) (Diagnostic Nodes).

The total loss is just a weighted sum of the four components. The weights \(\lambda\) control the relative importance of each objective; they do not turn a soft penalty into a hard constraint.

Let me give you some intuition for the typical values:

  • \(\lambda_{\text{pur}} = 0.1\): Purity is important but not dominant. We want charts to specialize, but not at the expense of prediction accuracy. If you set this too high, charts become so specialized they can’t generalize.

  • \(\lambda_{\text{bal}} = 0.01\): Balance is a soft constraint. We want to avoid collapse but not force unnatural uniformity. A small weight nudges toward balance without fighting the natural structure of the data.

  • \(\lambda_{\text{met}} = 0.01\): The metric loss is supplementary. It penalizes the selected overlap or distance proxy for edge cases where different-class samples remain coupled in the router.

In practice, you should tune these on a validation set and check chart coverage. The displayed values are starting points, not universal guarantees.

Anatomy of the Implementation

Let me walk through the code step by step, matching each piece to the math.

The learnable chart-to-class mapping:

self.chart_to_class = nn.Parameter(torch.randn(num_charts, num_classes) * 0.01)

This is \(\Theta \in \mathbb{R}^{N_c \times C}\) from Definition Definition 109. Each entry \(\Theta_{k,y}\) is the “affinity” of chart \(k\) for class \(y\). We initialize it near zero (small random values) so all charts start with roughly equal preference for all classes.

Computing the chart-level class scores:

return F.softmax(self.chart_to_class / self.temperature, dim=1)

Apply softmax along the class dimension. Temperature controls sharpness: low temperature gives more peaked distributions (charts commit strongly to one class), high temperature gives flatter distributions (charts remain agnostic). The semantic potential then mixes these rows with the input-dependent soft router before taking its logarithm.

Route alignment loss:

p_y_x = torch.matmul(router_weights, p_y_k)  # [B, C]
loss_route = F.cross_entropy(torch.log(p_y_x + 1e-8), y_true)

Matrix multiply router weights \([B, N_c]\) with class distributions \([N_c, C]\) to get batch predictions \([B, C]\). Then compute cross-entropy against true labels.

Purity loss (empirical routed labels):

entropy_per_chart = -(p_y_k * torch.log(p_y_k + 1e-8)).sum(dim=1)
p_k = router_weights.mean(dim=0)
loss_purity = (p_k * entropy_per_chart).sum()

The purity statistic uses the routed labels in the current batch, or the corresponding EMA, to form empirical conditionals \(\widehat p_{y|k}^{\mathrm{data}}\) before taking the chart-weighted entropy. The chart-score rows \(\Theta\) belong to the route-prediction and semantic-potential path; they are not the purity observation. Thus the result is an empirical batch/EMA proxy for \(H_{\mathrm{data}}(Y|K)\), with sampling and smoothing error.

Balance loss:

loss_balance = (p_k * (torch.log(p_k + 1e-8) - torch.log(uniform))).sum()

This is \(D_{KL}(\bar{w} \| \text{Uniform})\), computed directly from the definition of KL divergence.

Metric loss: The implementation uses router overlap as the bounded proxy specified by the definition. It weights pairs that share routing (high overlap) and have different classes; it does not compute a WFR or jump distance unless a separate estimator is added. Keep this route/metric proxy distinct from the empirical routed-label conditionals used by purity.

Thermodynamics: Conditioned Generation#

This framework relates classification to the generative law (Radial Generation: Entropic Drift and Policy Control).

Now here’s where things get really elegant. We’ve been treating class labels as things we predict—given an input, which class? But we can also treat class labels as things we condition on—given a class, generate an input.

The semantic potential \(V_y\) can be used in both settings, but the conclusions are different. For classification, the relaxation result concerns the stated basin or finite-horizon event. For generation, the same potential defines a sampling objective whose invariant law exists only under the corresponding Langevin and confinement hypotheses.

The useful connection is that both tasks use the same score and router. It does not mean that a classification basin, or a generated sample, is correct by construction; those claims still require their respective checks.

Remark 27 (Connection to Mobius Re-centering)

The Möbius re-centering \(\phi_c\) (Definition Definition 75) can be used to center a conditioned generation run at the class centroid defined below. It is a coordinate change; it does not alter the potential or prove that generated samples have the requested label:

\[ c_y := \arg\min_{c\in\mathbb D} \sum_{x:Y(x)=y}d_{\mathbb D}(c,\operatorname{Enc}(x))^2,\]

i.e., the Fréchet mean of the encoded class-\(y\) samples. The associated coordinate map is \(\phi_{c_y}(z):=(-c_y)\oplus z\); conditioned generation may initialize at \(c_y\) and express subsequent coordinates relative to it.

The class centroid is the Fréchet mean of the encoded samples with label \(y\), measured in the Poincaré metric. It is a point chosen by a distance minimization, not an ordinary coordinate average. If a Möbius recentering is introduced, it moves that chosen point to the origin; the map is a coordinate operation, while the potential and its dynamics remain separate objects.

Think of it like this: if you want to generate a cat, you may initialize near the hyperbolic center of the cat samples and then add variation around it. This is a useful initialization or conditioning convention, not a proof that every generated sample has class \(y\).

Remark 28 (Class-Conditioned Langevin Model)

The conservative overdamped Langevin equation from Theorem Theorem 10 with class conditioning is, in computation time,

\[ dz^k = \left[-G^{k\ell}\partial_\ell V_y -T_cG^{ij}\Gamma^k_{ij}\right]ds +\sqrt{2T_c}\,(G^{-1/2})^{kj}\,dW_s^j,\]

where \(V_y\) is the class-conditioned potential (Definition Definition 109). This is a formal conditional model; an invariant Gibbs law requires compatible drift, boundary conditions, and volume corrections for the chosen manifold convention.

Interpretation: To generate a sample of class \(y\), we run Langevin dynamics with the \(V_y\) potential. The semantic term \(-\beta_{\text{class}} \log P(Y=y \mid K)\) biases the flow toward class-\(y\) charts.

Langevin dynamics is one of the most beautiful tools in all of computational physics. You have a potential landscape, you start somewhere, and you let the system roll downhill while being buffeted by random noise. Under the required regularity, confinement, and invariant-measure conditions, the long-run law can be proportional to \(e^{-V/T}\).

For class-conditioned generation, we use \(V_y\) as the potential. This potential has two parts: the base potential \(V_{\text{base}}\) (which is intended to encode the data preference) and the soft-router semantic term (which favors a high class-\(y\) score).

The result, when the invariant law exists, is a preference for regions where \(V_y\) is low. This may combine realistic base-potential regions with high class-\(y\) score, but the correspondence is a modeling objective and must be checked rather than assumed.

Remark 29 (Label as a Conditional Symmetry-Breaking Field [Ho and Salimans, 2022])

Assume the conservative model, a differentiable router, and

\[ dV_{\mathrm{base}}(0)=0,\qquad A(0)=0, \]

so that the origin is a critical point of the unconditioned potential. Put \(q_y(z):=\sum_k w_k(z)P(Y=y\mid K=k)\) and assume \(q_y(0)>0\). The class label breaks the rotational symmetry at the origin precisely when \(dq_y(0)\ne0\):

\[ dV_y(0) = -\beta_{\mathrm{class}}\,\frac{dq_y(0)}{q_y(0)}. \]

With \(G(0)\) positive definite, the corresponding metric gradient is nonzero if and only if \(dq_y(0)\ne0\); only under that non-orthogonality condition does the initial deterministic drift acquire a class-dependent direction.

This is physics language, but the idea is simple. An isotropic metric or entropy field at the origin does not automatically make the learned base potential symmetric. To claim an unconditioned critical point, the corresponding hypothesis on \(V_{\text{base}}\) and the reward one-form must be stated.

With the soft router, the class term can have a nonzero gradient at the origin only when the router derivative has a component along the class-affinity differences. In a symmetric router that component may vanish. The class label therefore can break the symmetry, but it does not do so for free.

This is analogous to classifier-free guidance in diffusion models: a conditioned score can bias an unconditioned one. The analogy identifies the role of the extra score; it does not replace the hypotheses of the Langevin equation or prove that the flow reaches a class basin.

The strength of this guidance is controlled by \(\beta_{\text{class}}\). High \(\beta_{\text{class}}\) gives the semantic term more weight; low \(\beta_{\text{class}}\) leaves more room for the base potential and noise.

Definition 118 (Class Centroid in Poincare Disk)

For the Poincare disk embedding [Ganea et al., 2018, Nickel and Kiela, 2017], define the class centroid using the Fréchet mean [Lou et al., 2020]:

\[ c_y := \arg\min_{c \in \mathbb{D}} \sum_{x: Y(x)=y} d_{\mathbb{D}}(c, \text{Enc}(x))^2.\]

Under the usual finite second-moment assumption, the complete negatively curved disk has a unique Fréchet mean. This \(c_y\) is the centroid used by the Möbius map in Definition Definition 75.

Cross-references: Policy Control Field (Langevin Dynamics), Definition Definition 75, and Definition Proposition 15.

Why Hyperbolic Centroids Are Special

In Euclidean space, the centroid (mean) of a set of points is straightforward: just average the coordinates. In hyperbolic space, like the Poincare disk, it’s more subtle.

The Frechet mean is the generalization of the centroid to curved spaces: it’s the point that minimizes the sum of squared distances to all the data points. In Euclidean space, this gives you the ordinary mean. In hyperbolic space, it gives you something different.

Why does this matter? Hyperbolic space has the property that volume grows exponentially with radius, so Euclidean averaging gives the wrong geometry. A Fréchet mean need not be closer to the origin in every data configuration; its position is determined by the hyperbolic distances and the sample distribution.

With the usual completeness and finite-moment assumptions, the negative-curvature geometry gives the Fréchet objective a unique mean. The uniqueness is a property of those hypotheses, not a promise that an arbitrary finite or ill-conditioned sample has a well-behaved estimate.

Remark 30 (Integration with TopologicalDecoder)

The TopologicalDecoder (TopoEncoder Architecture Overview (Current Implementation)) receives the geometric content \(z_{\text{geo}} = e_K + z_n\) and routes through chart-specific projectors. For class-conditioned generation:

  1. Class determines charts: The class label \(y\) biases chart selection toward \(\mathcal{A}_y\) via the semantic potential \(V_y\)

  2. Decoder routing: The TopologicalDecoder’s inverse router (Topological Decoder Module) can either:

    • Accept an explicit chart index \(K\) (from the generative flow)

    • Infer routing from \(z_{\text{geo}}\) (autonomous mode)

  3. Consistency constraint: The decoder’s inferred routing should agree with the encoder’s class-conditioned routing:

    \[ \mathcal{L}_{\text{route-consistency}} = \mathbb{E}_{x,y}\left[\text{CE}\left(w_{\text{dec}}(z_{\text{geo}}), w_{\text{enc}}(x)\right)\right]\]

    where \(w_{\text{dec}}\) are the decoder’s soft router weights and \(w_{\text{enc}}\) are the encoder’s.

This ensures that class-conditioned generation produces samples that the encoder would classify correctly—a form of cycle consistency between encoding and decoding under the semantic topology.

This remark connects our classification framework to the decoder. The key insight is cycle consistency.

Here’s the scenario: we generate a sample conditioned on class \(y\). The generative flow uses the semantic score and may visit class-associated charts. Now we take that generated sample and encode it back. Does the encoder agree that it’s class \(y\)?

The answer is an empirical consistency question. The route-consistency loss encourages the decoder’s routing decisions to agree with the encoder’s; it does not make the generated sample class-\(y\) by construction, and it does not make encoding and decoding exact inverses.

This is why the cycle check is useful: it tests whether the two directions remain aligned after training. A mismatch can come from the generator, the encoder, the router, or the data support, and the loss gives a signal for correcting it.

Hierarchical Classification via Scale Decomposition#

Real-world categories are hierarchical (e.g., Animal -> Dog -> Terrier). The stacked TopoEncoder (Stacked TopoEncoders: Deep Renormalization Group Flow) can be used to reflect this.

Most classification problems have implicit hierarchy. A Terrier is a Dog is an Animal is a Living Thing. ImageNet has 1000 leaf classes but they cluster into broader categories (vehicles, animals, furniture, etc.).

Standard classification treats all classes as equally unrelated. But that’s clearly wrong—misclassifying a Terrier as a Poodle is less bad than misclassifying it as a Truck. The semantic structure matters.

Our framework can encode this design: stacked layers operate at multiple scales, and one may align coarse labels with bulk features and fine labels with boundary features. The alignment is an inductive choice that must be validated on the data; it is not automatic evidence that a layer has the required semantic resolution.

Definition 119 (Hierarchical Labels)

A label hierarchy is a sequence of label spaces:

\[ \mathcal{Y}_L \xrightarrow{\,\pi_L\,}\mathcal{Y}_{L-1} \xrightarrow{\,\pi_{L-1}\,}\cdots \xrightarrow{\,\pi_1\,}\mathcal{Y}_0,\]

where each \(\pi_\ell:\mathcal{Y}_\ell\twoheadrightarrow\mathcal{Y}_{\ell-1}\) is a surjective coarsening map. \(\mathcal{Y}_0\) contains coarse labels (super-categories), while \(\mathcal{Y}_L\) contains fine labels (leaf categories).

Example: \(\mathcal{Y}_0 = \{\text{Animal}, \text{Vehicle}\}\), \(\mathcal{Y}_1 = \{\text{Dog}, \text{Cat}, \text{Car}, \text{Bike}\}\), \(\mathcal{Y}_2 = \{\text{Terrier}, \text{Poodle}, \ldots\}\).

The surjection arrows \(\twoheadrightarrow\) mean “can be coarsened to.” Each fine label maps to exactly one coarser label. Terrier maps to Dog maps to Animal. This forms a tree structure.

The label hierarchy tells us how to group classes at different levels of abstraction. At the coarsest level (level 0), you just distinguish Animal from Vehicle. At the finest level (level \(L\)), you distinguish all the leaf categories.

Proposition 34 (Scale-Label Alignment)

In the stacked TopoEncoder (Stacked TopoEncoders: Deep Renormalization Group Flow), enforce purity at each scale:

  • Layer 0 (Bulk/Slow): Charts at level 0 correspond to coarse classes. Enforce:

    \[ \mathcal{L}_{\text{purity}}^{(0)} = H(\mathcal{Y}_0 \mid K^{(0)})\]

  • Layer \(\ell\) (Intermediate): Charts at level \(\ell\) correspond to level-\(\ell\) classes. Enforce:

    \[ \mathcal{L}_{\text{purity}}^{(\ell)} = H(\mathcal{Y}_\ell \mid K^{(\ell)})\]

  • Layer \(L\) (Boundary/Fast): Charts at level \(L\) correspond to fine classes. Enforce:

    \[ \mathcal{L}_{\text{purity}}^{(L)} = H(\mathcal{Y}_L \mid K^{(L)})\]

The idea is beautiful in its simplicity: match the scale of the label to the scale of the representation.

At the bulk (deep, slow) layers of the network, we don’t expect to distinguish Terriers from Poodles. These fine distinctions require subtle features that only emerge at the boundary (shallow, fast) layers. But bulk layers can distinguish Animals from Vehicles—that’s a coarse distinction that shows up even in low-resolution features.

So we can encourage a hierarchy: bulk charts are trained toward purity for coarse labels, boundary charts toward purity for fine labels, and intermediate layers toward intermediate labels. The loss expresses this preference; it does not guarantee purity at every scale.

This alignment can be computationally useful. It asks each layer to solve a task suited to its intended scale, while leaving the actual representational capacity and the quality of the hierarchy to validation.

Remark 31 (Renormalization Group Interpretation)

The semantic hierarchy matches the physical renormalization scale:

Scale

Latent Structure

Semantic Structure

Bulk (Layer 0)

Slow modes, large wavelengths

Super-categories

Intermediate

Medium modes

Categories

Boundary (Layer \(L\))

Fast modes, fine details

Sub-categories

This is the semantic RG flow: coarse-graining in the label space corresponds to flowing toward the bulk in latent space.

What is the Renormalization Group?

The Renormalization Group (RG) is one of the deepest ideas in physics. It describes how physical systems look different at different scales.

Imagine zooming out from a picture. At high resolution, you see fine details: individual pixels, textures, edges. As you zoom out, details blur together. What remains are the large-scale structures: shapes, colors, overall composition.

The RG says this isn’t just losing information—it’s systematically losing the right information. At each scale, certain patterns dominate and others become irrelevant. The “flow” from fine to coarse follows predictable rules.

In our semantic context: fine categories (Terrier, Poodle) are “high-resolution” features that blur into coarser categories (Dog, Cat) as you zoom out. At the coarsest scale, you just see “Animal.” This semantic coarse-graining is the same mathematical structure as physical coarse-graining—that’s the RG connection.

The stacked TopoEncoder literally implements this: bulk layers see coarse features, boundary layers see fine features, and the flow from boundary to bulk is the semantic RG flow.

Definition 120 (Hierarchical Supervised Loss)

The total hierarchical loss:

\[ \mathcal{L}_{\text{hier}} = \sum_{\ell=0}^{L} \alpha_\ell \left(\mathcal{L}_{\text{route}}^{(\ell)} + \lambda_{\text{pur}} \mathcal{L}_{\text{purity}}^{(\ell)}\right),\]

where \(\alpha_\ell\) weights the contribution of each scale (typically \(\alpha_\ell = 1\) or decaying with \(\ell\)).

Cross-references: Stacked TopoEncoders: Deep Renormalization Group Flow (Stacked TopoEncoder), Definition Definition 20, Multiresolution Interpretation (RG Analogy) (RG Interpretation).

The hierarchical loss sums the route alignment and purity losses across all scales, weighted by \(\alpha_\ell\).

Why might you use decaying weights \(\alpha_\ell\)? One reason: coarse decisions may be more important than fine ones. Getting Animal vs Vehicle right may be more useful than getting Terrier vs Poodle right. Decaying weights (higher \(\alpha\) for coarse scales) encode that chosen priority.

Another reason: fine distinctions are harder to learn. Giving them equal weight can lead to the network spending all its effort on subtle distinctions while ignoring obvious ones. Decaying weights ensure the basics are learned first.

In practice, the weights are hyperparameters to validate. Equal weights (\(\alpha_\ell = 1\)) are a baseline, not a guarantee that the hierarchy will be learned.

Summary and Diagnostic Nodes#

Let’s take stock of what we’ve built. We started with a simple question—what does classification mean geometrically?—and constructed a complete framework:

  1. Semantic Partitions: Class labels induce a thresholded, probabilistic partition of the chart atlas; the transition set is its complement.

  2. Class-Conditioned Potentials: Each class \(y\) has a soft-router potential whose class-dependent drift and basins are interpreted under the stated smoothness and dynamical hypotheses.

  3. Jump Rate Modulation: Cross-class edges in the declared transition graph are suppressed, producing a measurable kinetic separation; this does not make the continuous WFR geometry divergent.

  4. Multi-scale Losses: Purity, balance, route alignment, and metric separation work together as soft training objectives for a geometrically coherent classifier.

  5. Hierarchical Extension: The framework naturally extends to hierarchical labels via the stacked encoder’s scale decomposition.

The result is a geometric interpretation of classification—classes are learned regions, boundaries can act as barriers, and prediction may be implemented by relaxation. Each statement keeps the hypotheses of its metric, potential, or diagnostic.

Table 25.7.1 (Summary of Supervised Topology Laws).

Aspect

Formula

Units

Reference

Semantic Partition

\(\mathcal{A}_y = \{k: P(Y=y\mid K=k) > 1-\epsilon\}\), \(\mathcal{T}=\mathcal{K}\setminus\bigcup_y\mathcal{A}_y\)

—

Def Definition 107

Class Potential

\(V_y = -\beta_{\text{class}} \log\!\left(\sum_k w_k(z)P(Y=y\mid K=k)\right) + V_{\text{base}}(z)\)

nat

Def Definition 109

Jump Modulation

\(\lambda_{i\to j}^{\text{sup}}/\lambda_{i\to j}^{(0)} = e^{-\gamma_{\text{sep}}}\) on cross-class edges

step\(^{-1}\)

Def Definition 111

Purity Loss

\(\sum_k P(K=k) H_{\mathrm{data}}(Y\mid K=k)\)

nat

Def Definition 113

Route Alignment

\(\text{CE}(\sum_k w_k P(Y\mid K=k), y_{\text{true}})\)

nat

Def Definition 116

Supervised PurityCheck

Following the diagnostic node convention (Theory: Thin Interfaces), we define checks that monitor the health of the supervised topology.

#

Name

Component

Type

Interpretation

Proxy

Cost

S-P

PurityCheck

Router

Semantic Clustering

Are charts class-pure?

\(H_{\mathrm{data}}(Y \mid K)\)

\(O(BN_c + N_cC)\)

Trigger conditions:

  • High PurityCheck: Charts contain mixed classes; classification boundaries fall within charts.

  • Remedy: Increase purity loss weight \(\lambda_{\text{pur}}\); increase number of charts; check for insufficient training data per class.

Interpreting Purity Diagnostics

What does it mean when purity is high (bad)?

Diagnosis 1: Too few charts. If you have 10 classes and only 5 charts, at least some charts must handle multiple classes. Solution: add more charts.

Diagnosis 2: Classes genuinely overlap. Some classification problems have classes that are intrinsically hard to separate. In this case, high purity might be unavoidable, and you should focus on good routing rather than chart purity.

Diagnosis 3: Training issues. Maybe you have enough charts, but training hasn’t converged. Check learning curves. The purity loss should decrease over training.

Diagnosis 4: Class imbalance. If one class dominates, all charts might end up associated with it. Check your class distribution and consider balancing.

Supervised ClassSeparationCheck

#

Name

Component

Type

Interpretation

Proxy

Cost

S-C

ClassSeparationCheck

Jump Op

Class Separation

Are cross-class jump rates sufficiently suppressed?

\(\max_{i,j:\,\operatorname{Class}(i)\ne\operatorname{Class}(j)} \lambda_{ij}^{\text{sup}}/\lambda_{ij}^{(0)}\)

\(O(C^2 N_c)\)

Trigger conditions:

  • Low ClassSeparationCheck: Different classes are metrically close; cross-class transitions are too frequent.

  • Remedy: Increase separation strength \(\gamma_{\text{sep}}\); add metric contrastive loss; check for class imbalance.

The separation check monitors the largest cross-class rate ratio. A high value means that at least one cross-class edge remains close to its base rate; a low value means that the prescribed suppression is active. This is a warning about the learned transition channel, not a verdict about the continuous WFR distance.

The proxy can be expensive to compute exactly, so in practice you might use a cheaper approximation based on router overlap statistics. Record which proxy was used, because a router overlap, a discrete edge metric, and the continuous WFR action are different objects.

Cross-references: Diagnostics: Stability Checks (Monitors) (Sieve Diagnostic Nodes), Section 24.7 (Scalar Field Diagnostics).

And that’s supervised topology. We’ve seen how class labels can be represented geometrically, how the potential and jump channels play different roles, and how the training diagnostics fit with the rest of the framework.

The key insight, the thing I want you to take away, is this: classification is not about drawing decision boundaries. It’s about organizing the internal representation so that similar things are near and different things are far. The decision boundary is a consequence of this organization, not the primary object.

When you think about it that way, a lot of things make sense. Why can representation learning help classification? Because it organizes the space. Why can contrastive losses help? Because they penalize selected forms of overlap. Why do we care about the metric? Because the chosen distance or rate-weighted cost tells us what “near” and “far” mean in that model.

The math in this section—the potentials, the jump rates, the losses—makes these intuitions precise and trainable. The core idea is simple: classification can be studied as geometry, provided each conclusion stays attached to its object and hypotheses.