Diffusion | DDIM的边界情况DDPM推导

DDPM中建模的$q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)$满足正态分布，
$$\begin{gathered} q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)=\mathcal{N}({\mathbf{x}}_{t-1};\tilde{\mathit{\mu }}({\mathbf{x}}_t,{\mathbf{x}}_0),{\tilde{\beta }}_t\mathbf{I}) \\ {\tilde{\beta }}_t=\frac{1-{\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t}\cdot {\beta }_t \end{gathered}$$
DDIM中建模的$q_{\sigma }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)$如下，第一个等式二三步用到了重参数技巧和多个独立高斯分布的等价形式，
$$\begin{array}{rl}{\mathbf{x}}_{t-1}& =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}}{\mathit{ϵ}}_{t-1}\\ & =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-{\sigma }_t^2}{\mathit{ϵ}}_t+{\sigma }_t\mathit{ϵ}\\ & =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_t-\sqrt{{\overline{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\overline{\alpha }}_t}}+{\sigma }_t\mathit{ϵ}\\ q_{\sigma }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)& =\mathcal{N}({\mathbf{x}}_{t-1};\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_t-\sqrt{{\overline{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\overline{\alpha }}_t}},{\sigma }_t^2\mathbf{I})\end{array}$$

DDIM中的的${\sigma }_t$与DDPM中的$\widetilde{{\beta }_t}$保持一致，并且添加了一个可学习参数控制方差，
$${\sigma }_t^2=\eta \cdot {\tilde{\beta }}_t$$
当$\eta =0$时，采样过程是确定的；当$\eta =1$时，退化成DDPM的形式，以下给出推导，
$$\begin{array}{rl}{\mu }_{{\mathbf{x}}_{t-1}}& =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_t-\sqrt{{\overline{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-{\tilde{\beta }}_t}\frac{{\mathbf{x}}_t-\sqrt{{\overline{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =\sqrt{{\overline{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-\frac{(1-{\overline{\alpha }}_{t-1})\cdot (1-{\alpha }_t)}{1-{\overline{\alpha }}_t}}\frac{{\mathbf{x}}_t-\sqrt{{\overline{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}+\sqrt{1-{\overline{\alpha }}_{t-1}-\frac{(1-{\overline{\alpha }}_{t-1})\cdot (1-{\alpha }_t)}{1-{\overline{\alpha }}_t}}\cdot \frac{-\sqrt{{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}})\cdot {\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}-\frac{(1-{\overline{\alpha }}_{t-1})\cdot (1-{\alpha }_t)}{1-{\overline{\alpha }}_t}}\cdot \frac{{\mathbf{x}}_t}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}+\sqrt{(1-{\overline{\alpha }}_{t-1})\cdot (1-\frac{1-{\alpha }_t}{1-{\overline{\alpha }}_t})}\cdot \frac{-\sqrt{{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}})\cdot {\mathbf{x}}_0+\sqrt{(1-{\overline{\alpha }}_{t-1})\cdot (1-\frac{1-{\alpha }_t}{1-{\overline{\alpha }}_t})}\cdot \frac{{\mathbf{x}}_t}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}+\sqrt{(1-{\overline{\alpha }}_{t-1})\cdot \frac{{\alpha }_t-{\overline{\alpha }}_t}{1-{\overline{\alpha }}_t}}\cdot \frac{-\sqrt{{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}})\cdot {\mathbf{x}}_0+\sqrt{(1-{\overline{\alpha }}_{t-1})\frac{{\alpha }_t-{\overline{\alpha }}_t}{1-{\overline{\alpha }}_t}}\cdot \frac{{\mathbf{x}}_t}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}+\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \frac{\sqrt{{\alpha }_t-{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}}\cdot \frac{-\sqrt{{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}})\cdot {\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \frac{\sqrt{{\alpha }_t-{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}}\cdot \frac{{\mathbf{x}}_t}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}-\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \frac{\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \sqrt{{\alpha }_t}}{\sqrt{1-{\overline{\alpha }}_t}}\cdot \frac{\sqrt{{\overline{\alpha }}_t}}{\sqrt{1-{\overline{\alpha }}_t}})\cdot {\mathbf{x}}_0+\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \frac{\sqrt{1-{\overline{\alpha }}_{t-1}}\cdot \sqrt{{\alpha }_t}}{\sqrt{1-{\overline{\alpha }}_t}}\cdot \frac{{\mathbf{x}}_t}{\sqrt{1-{\overline{\alpha }}_t}}\\ & =(\sqrt{{\overline{\alpha }}_{t-1}}-\frac{\sqrt{{\alpha }_t}\cdot \sqrt{{\overline{\alpha }}_t}\cdot (1-{\overline{\alpha }}_{t-1})}{1-{\overline{\alpha }}_t})\cdot {\mathbf{x}}_0+\frac{\sqrt{{\alpha }_t}\cdot (1-{\overline{\alpha }}_{t-1})\cdot {\mathbf{x}}_t}{1-{\overline{\alpha }}_t}\\ & =(\frac{\sqrt{{\overline{\alpha }}_{t-1}}-{\overline{\alpha }}_t\cdot \sqrt{{\overline{\alpha }}_{t-1}}-\sqrt{{\alpha }_t}\cdot \sqrt{{\overline{\alpha }}_t}\cdot +\sqrt{{\alpha }_t}\cdot \sqrt{{\overline{\alpha }}_t}\cdot {\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t})\cdot {\mathbf{x}}_0+\frac{\sqrt{{\alpha }_t}\cdot (1-{\overline{\alpha }}_{t-1})\cdot {\mathbf{x}}_t}{1-{\overline{\alpha }}_t}\\ & =(\frac{\sqrt{{\overline{\alpha }}_{t-1}}-{\overline{\alpha }}_t\cdot \sqrt{{\overline{\alpha }}_{t-1}}-\sqrt{{\alpha }_{t-1}}\cdot {\overline{\alpha }}_t+\sqrt{{\overline{\alpha }}_{t-1}}\cdot {\overline{\alpha }}_t}{1-{\overline{\alpha }}_t})\cdot {\mathbf{x}}_0+\frac{\sqrt{{\alpha }_t}\cdot (1-{\overline{\alpha }}_{t-1})\cdot {\mathbf{x}}_t}{1-{\overline{\alpha }}_t}\\ & =(\frac{\sqrt{{\overline{\alpha }}_{t-1}}-\sqrt{{\alpha }_{t-1}}\cdot {\overline{\alpha }}_t}{1-{\overline{\alpha }}_t})\cdot {\mathbf{x}}_0+\frac{\sqrt{{\alpha }_t}\cdot (1-{\overline{\alpha }}_{t-1})\cdot {\mathbf{x}}_t}{1-{\overline{\alpha }}_t}\\ & ={\mu }_{{\mathbf{x}}_{t-1}}^{DDPM}\end{array}$$