• latent variables are hidden causes we never observe directly but infer from the observed data , central to generative machine learning
    • example: for handwritten digits, could be stroke width, slant and which digit it is; the pixels are what you actually see
    • they’re the coordinates of a Latent Space, and finding good ones is basically Representation learning
  • generative story: sample from a prior , then from , so the data likelihood marginalizes the latent out
- first term is reconstruction, second is a [[Kullback-Leibler Divergence|KL]] penalty keeping the encoder close to the prior $p(z)$
- the gap between the ELBO and $\log p(x)$ is exactly $\text{KL}(q(z|x) \parallel p(z|x))$, how wrong the approximate posterior is
  • GANs keep only the generator direction: a noise vector ( a few hundred dims, still far smaller than the image ) is mapped to data, with no encoder and no likelihood
  • classic latent variable models: probabilistic PCA ( the linear gaussian case ), mixture models, HMMs ( discrete latent states over time )
  • for neural data this is the whole game: a few latent dynamical factors driving many recorded neurons, e.g. LFADS
  • diffusion models are latent variable models too, the progressively noised images are the latents