- latent variables are hidden causes z we never observe directly but infer from the observed data x, central to generative machine learning
- example: for handwritten digits, z could be stroke width, slant and which digit it is; the pixels x are what you actually see
- they’re the coordinates of a Latent Space, and finding good ones is basically Representation learning
- generative story: sample z from a prior p(z), then x from p(x∣z), so the data likelihood marginalizes the latent out
p(x)=∫p(x∣z)p(z)dz
- inference difficult: posterior p(z∣x) usually intractable ( Bayesian Inference ), we approximate it
LELBO=Eq(z∣x)[logp(x∣z)]−KL(q(z∣x)∥p(z))
- first term is reconstruction, second is a [[Kullback-Leibler Divergence|KL]] penalty keeping the encoder close to the prior $p(z)$
- the gap between the ELBO and $\log p(x)$ is exactly $\text{KL}(q(z|x) \parallel p(z|x))$, how wrong the approximate posterior is
- GANs keep only the generator direction: a noise vector z ( a few hundred dims, still far smaller than the image ) is mapped to data, with no encoder and no likelihood
- classic latent variable models: probabilistic PCA ( the linear gaussian case ), mixture models, HMMs ( discrete latent states over time )
- for neural data this is the whole game: a few latent dynamical factors driving many recorded neurons, e.g. LFADS
- diffusion models are latent variable models too, the progressively noised images are the latents