Distribution Matching Distillation
without Fake Score Network

Flow Matching models learn a velocity field from an interpolant that transports particles between a prior distribution and the target data distribution. Let \(p_\mathrm{real}\) denote a data distribution. For a tractable prior distribution \(p_Z\), previous work defines the linear interpolation \(x_t = (1-t)x + tz\), where \(x\sim p_\mathrm{real}\), \(z\sim p_Z\), and \(t\in [0, 1]\).

The conditional velocity along this path is \(v_t(x_t|x) = (x_t - x) / t\), and the marginal velocity is given by \(v^*_t(x_t) = \mathbb E_{x|x_t}[v_t(x_t|x)]\). Generation follows the probability-flow ODE (PF-ODE) induced by \(v^*_t\), namely \(dx_t = v^*_t(x_t)\,dt\). A good approximation requires multiple sampling steps, which motivates a direct few-step generator.

Flow Maps. To construct a few-step generator from the underlying flow, a flow map (Boffi et al., 2025; Kim et al., 2024) defines a mapping between two points \(x_t\) and \(x_s\) on the same trajectory:

Following MeanFlow-style flow maps, we parameterize the flow-map network \(X_\theta\) with the pseudo-velocity network \(u_\theta\), given by \(X_\theta(x_t; t, s) = x_t + (s-t)u_\theta(x_t; t, s)\), and the corresponding consistency objective is:

where \(\operatorname{sg}[\cdot]\) is a stop-gradient operator and \(w_{t,s}\) is a scalar weight. In particular, \(u_\theta(x_t; t, s)\) represents the average velocity needed to move \(x_t\) to time \(s\), and \(u_\theta(x_t;t,t)\) recovers the instantaneous velocity \(v^*_t(x_t)\) in the \(s\to t\) limit.

Distribution Matching Distillation (DMD) (Yin et al., 2024b) matches the real and generative distributions through the difference between their perturbed scores to train a few-step generator \(G_\theta\). Let \(p_{\mathrm{fake}} = (G_\theta)_\# p_Z\) be the push-forward generative distribution. DMD minimizes the Kullback-Leibler (KL) divergence between the noised distributions:

for \(\hat x_t = (1-t)G_\theta(z) + tz'\) and \(z'\sim p_Z\). In this case, we assume the teacher-score network \(s_\Phi(x_t; t)\approx\nabla_x\log p_\mathrm{real}(x_t; t)\). As the fake score changes with \(G_\theta\), standard DMD introduces an auxiliary network \(s_\psi\) to estimate the score of the current generative distribution:

for \(\gamma_t = (1-t)t^{-1}\). The key point is that the fake score is not fixed: it changes as \(G_\theta\) updates. Therefore, DMD uses a separate fake-score tracker, represented here by \(s_\psi\) or \(v_\psi\), which introduces additional computational overhead.

Our goal is estimating the fake velocity without introducing an additional network.

Motivating Observation. This becomes tractable when the generator has the flow-map structure in Eq. 2. Starting from the flow-map setting, the auxiliary fake velocity \(v_\psi\) would be trained by

where \(\hat x = G_\theta(z) = X_\theta(z; 1, 0)\). If the perturbation reuses the generator input, \(z'=z\), the fake velocity becomes a conditional expectation over the pseudo-velocity, \(\mathbb E_{z|\hat x_t}[u_\theta(z; 1, 0)]\). However, the conditional expectation is still not directly accessible since \(p(z|\hat x_t)\) is nontrivial in general.

Deriving the Surrogate. A further simplification appears under a PF-ODE-based noising process, which defines an inverse map \(\hat x_t = X(\hat x; 0,t)\). By injectivity and invertibility of the flow map, the posterior collapses to a point mass \(p(\hat x|\hat x_t) = \delta(X(\hat x; 0, t) - \hat x_t)\), yielding the corresponding fake velocity \(v_\mathrm{fake}^\mathrm{ODE}\) (see App. A.3):

Therefore, under the PF-ODE coupling, the pseudo-velocity \(u_\theta(\hat x_t; t,0)\) approximates the fake velocity \(v_\mathrm{fake}^\mathrm{ODE}(\hat x_t; t)\) without introducing a separate network or training loop.

We use it as a tractable surrogate for the fake velocity, replacing the explicit fake-velocity estimator:

Implementation. The implemented FSF-DMD objective inserts this surrogate into a stop-gradient objective, given by:

where \(w_t\) is a scalar. Since the surrogate is derived under the flow-map generator, we use consistency distillation (Sabour et al., 2026), \(\mathcal L_\mathrm{CD}\), to align the generator with the flow-map structure. The full objective combines this term with Eq. 11, weighted by a scalar \(\lambda\):

In the few-step generation setting, except for the first step, the generator input is derived from the previously generated sample. This induces a training-inference mismatch when we directly use perturbed data samples during training, and may affect generation performance. To minimize this mismatch, DMD2 introduces backward simulation. However, unlike DMD2, the flow-map generator has a different generation pipeline:

for \(\hat x = \hat x_0\), \(\hat x_1 = z\), and \(0\le i < N\). We found that backward simulation has room for improvement in our setting, since it does not fully reflect the flow-map generation process. We therefore make it consistent with the flow-map generator. We use \(N\) for inference-time generation steps and \(M\) for backward-simulation steps. Following Huang et al. (2026), applying stop-gradient to intermediate samples gives:

Incorporating Eq. 16 into the single-step objective in Eq. 11 gives our final few-step objective:

This objective preserves the flow-map generation process and coincides with the initial version in Eq. 11 when \(M=1\). We observe that it gives better performance in our setting, as shown in Table 5.

We can even extend FSF-DMD to the training-from-scratch setting without a teacher network. Since the MeanFlow-style parameterization gives \(u_\theta(x_t; t, t)\approx v^*_t(x_t)\), we can substitute the teacher network \(v_\Phi(x_t; t)\) with the network approximation itself:

For this setting, we replace consistency distillation with consistency training in Eq. 3. This objective provides a DMD-style correction for generated samples even in the training-from-scratch stage. It requires neither a teacher network nor an additional fake-velocity objective in generator training.