Stabilizing Consistency Training:
A Flow Map Analysis and Self-Distillation

In this section, we analyze instability and reproducibility in consistency models from a flow map perspective, focusing on how different design choices affect convergence properties and training dynamics. Based on this analysis, we explain why training from scratch may deviate from the desired flow map in practical settings.

To enable a systematic analysis of training instability in consistency models, we propose a flow map representation for commonly used interpolations, expressed as a one-step Euler solution with a network \(F_\theta\):

Most consistency training approaches are guided by a conditional velocity. However, this setup can contribute to training instability. To examine its effect, we consider Eulerian distillation with a conditional velocity field. Following Boffi et al. (2025), we refer to this as direct training.

Since the global optimum of Eulerian distillation corresponds to the flow map, we show that direct training does not guarantee convergence to this solution by comparing their stationary conditions.

Thus, the discrepancy is not only an objective-level mismatch: through the term \(C\), it can actually shift the stationary point and admit degenerate flow maps. This term arises from the gap \(\Delta v\) between the marginal velocity \(v^*_t(x_t)\) and the per-sample conditional velocity \(v_t(x_t|x)\).

Compared to Eulerian distillation, which successfully learns the ground-truth distribution, direct training converges to a degenerate distribution, consistent with the proposition.

When a stop-gradient operation is applied to \(\mathcal L_\mathrm{DT}\), it coincides with the consistency training objective \(\mathcal L_\mathrm{CT}\). This objective can recover solutions satisfying the Eulerian equation, even when guided by conditional velocity. However, this holds only at the expectation level, and a small batch size can induce a residual gap.

Figure 3 shows drift toward a degenerate solution as the batch size decreases, which is consistent with the finite-batch residual in Proposition 4.3. We also measure \(\mathcal L_\mathrm{ED}\) for the toy dataset, which serves as a proxy for estimating the optimality of flow maps. While it does not fully reflect generation performance, consistency training with small batch sizes and direct training yield higher \(\mathcal L_\mathrm{ED}\) values, which can be interpreted as a signal of a degenerate solution.

From this perspective, both the bias term in Proposition 4.2 and the finite-batch residual in Proposition 4.3 originate from the same conditional-marginal velocity mismatch. We therefore consider incorporating the marginal velocity as the underlying flow field to reduce this gap in the next section.

We further observe that consistency models with \(s=0\) are sensitive to weight initialization, which can affect reproducibility. Recent approaches rely on initialization with pretrained flow models, where the network is already approximately aligned with target flows. This reduces certain early-stage instability, and is often referred to as a preconditioner (Geng et al., 2025). We find that performance varies with the preconditioner, and reproducing results without known preconditioners can be challenging.

We evaluated an open-source consistency model (Sun et al., 2025) on ImageNet-1K \(256\!\times\!256\). With the baseline preconditioner, the model achieves reasonable results but remains above the reported FID. However, with other pretrained flow models, the FIDs become worse.

The resulting few-step FIDs are inconsistent: stronger preconditioners do not necessarily yield better consistency models. Additionally, models diverge when initialized randomly, suggesting that the optimum is inaccessible in this case.

As a complementary hypothesis, this can be understood from the stop-gradient (fixed-point) updates of consistency models: initialization matters as training is driven toward nearby fixed points rather than the global optimum. The result suggests the possible existence of attracting basins under different initialization weights.

Some studies (Geng et al., 2025; Frans et al., 2025) enable training from scratch without a preconditioner. The key difference is that they allow \(s<t\), while others fix \(s=0\). Intuitively, training long-range mappings is more challenging than short-range ones. Under the flow map representation, Eulerian distillation can be written as

We observe that \(s\to t\) amplifies the flow matching term \(v^*_t - F_\theta\) since \(A''_{t,t}=A_{t,t}=0\), while \(s\to 0\) amplifies the linearization term involving the JVP \(dF_\theta/dt\). As the JVP is structurally more complex, we hypothesize that it may give rise to more undesired fixed points. This suggests that fixing \(s=0\) makes optimization less stable, while relaxing to \(s<t\) balances the terms and mitigates instability.

To support our hypothesis, we examine the loss landscapes of each objective, using their variance and number of spikes as proxies for training stability.

Time relaxation reduces loss variance and spikes, which can be interpreted as reduced risk of training divergence, supporting our claim. We also note that our method in the next section, iSD, produces even smoother landscapes.

To evaluate our method on image generation, we conduct experiments on the ImageNet-1K dataset. We use downsampled \(32\!\times\!32\!\times\!4\) latent variables from \(256\!\times\!256\) images encoded by a VAE, and adopt a DiT in BF16 precision. For unconditional settings, we validate our method on the CelebA-HQ dataset. We further evaluate our method on diffusion-based policy learning to assess the applicability beyond image generation.

We conduct our ablation study on ImageNet-1K using DiT-B/4. The model is trained for 400K steps. We set \(\omega = 5.0\) for \(\mathcal L_\text{iSD-T}\) and use conditional velocity guidance for \(\mathcal L_\text{CT}\). All weights are randomly initialized. We measure the variance of quantitative results across five runs with different random initializations, and report mean values.

We begin with consistency training \(\mathcal L_\mathrm{CT}\) using linear interpolation. Joint training with flow matching improves results. Relaxing the time condition further improves results and reduces variance, which is consistent with our linearization cost hypothesis.

We observe that self-distillation alone underperforms, while incorporating Pre-CFG \(\mathcal L_\text{iSD-T}\) accelerates convergence, improves performance and reproducibility. One possible explanation is that vanilla iSD may trade the finite-batch residual for network approximation error, which can propagate to flow map training. Pre-CFG amplifies the class component and the flow map is trained along this sharpened field, whose task-relevant signal could become more prominent relative to the propagated approximation error.

Interpolation. Linear interpolation tends to perform better under conditional velocity guidance, whereas trigonometric interpolation achieves lower FIDs under iSD-T. We observe varying performance across settings and tasks, leaving further analysis to future work.

JVP Operation. We observe that JVP approximation often yields better empirical results than exact computation. Under BF16 precision, it tends to be more robust, as the exact one is sensitive to numerical precision. The VRAM requirement of the training decreases from 25GB to 18GB on iSD-T, and the training speed improves from 1.48 steps/sec to 4.04 steps/sec on an A100 GPU.

Classifier-free Guidance. Incorporating Pre-CFG improves FIDs compared to the vanilla \(\mathcal L_\text{iSD}\). It also outperforms Post-CFG, which deviates from the intended CFG field, whereas Pre-CFG aligns the flow map with the CFG velocity during training. Combining both often degrades performance, as it results in applying the guidance twice, overly skewing the trajectory.

Reproducibility. We examine the variance of quantitative results to assess reproducibility. Our iSD-T exhibits lower variance across runs compared to the baselines. From a performance perspective, we observe an FID gap with MeanFlow. Thus, a main advantage of iSD-T is more reproducible training outcomes with comparable sample quality.

Unconditional Generation. We compare our method with prior approaches on CelebA-HQ \(256\!\times\!256\) under a class-unconditional setting, using vanilla iSD without CFG. iSD achieves comparable performance in both few-step and multistep generation.

To examine scalability, we evaluate iSD-T with DiT-XL. iSD-T maintains stable training behavior and comparable sample quality at this scale. We note that achieving strong FID scores at this scale requires incorporating performance-oriented conditioning techniques, such as conditioning on a guidance scale or variations in latent representation. These choices are orthogonal to our analysis and are not required for stability.

To assess the applicability of iSD beyond image generation, we adopt it as a diffusion policy objective in robotic control tasks. Experimental settings follow prior work on diffusion-based policies (Chi et al., 2023) for transformer and state-based architectures. This is intended to examine the applicability rather than to establish state-of-the-art results.

iSD with linear interpolation yields performance comparable to other few-step methods, under the same evaluation protocol. With 2-step sampling, iSD achieves success rates comparable to those of diffusion policies that require more sampling steps. These results indicate that iSD can serve as a few-step alternative to diffusion-based objectives in policy learning.