Overview
This posting series is a study note that records the process of learning the book "Understanding Deep Learning".
This time, it covers Chapter 18, Diffusion models.
Marginalization
In Problem 18.7, the following equation:
is transformed into the equation below, but I couldn't understand how this transformation works.
To understand this, we need to first understand Marginalization.
Let's start with the simplest case.
As shown above, we can obtain the distribution by integrating over another variable from the joint distribution.
Now, returning to the original equation:
We can separate the integrals as shown above. Since doesn't contain , we can factor it out.
For the inner integral part, we can apply marginalization. Since we integrate over everything except , only remains.
As a result, we obtain the equation above, which finally becomes the following equation according to the definition of expectation.
Reflections
Unlike other chapters, this chapter had simple concepts but was very difficult in terms of deriving and understanding mathematical equations. The concept itself is really quite simple: the predetermined encoder only adds noise from a multivariate normal distribution when converting input data x to latent variable z, and the decoder is trained to reverse-track that noise. However, the proofs and equation derivations about how to probabilistically determine the loss function and how to skip steps and approximate were challenging.
Reference
[1] Prince, S. J. D. (2023). Understanding Deep Learning. The MIT Press. Retrieved from http://udlbook.com