Graph View

Supervised Fine Tuning

1 min read

PageRank 516th of 880

  • where Pretraining builds a Large Language Models foundational knowledge of language through massive quantities of unlabeled data ( a sort of Self Supervised Learning approach ), Supervised Fine Tuning follows by adapting that pretrain to a response model through structured input-output pairs
  • teaches the more desired response structure “You are a helpful assistant, your task is {tast}, and the users request is {request}” style training.
  • fundamentally is still the same technique as pretraining, but usually at a lower learning rate or using some kind of Parameter Efficient Fine Tuning method like Low Rank Adaptation for encoding behavior and response structure rather than language understanding.
    • main difference here is purely data, it is labeled and processed in a way to encourage mimicry

Advancements ( 10-06-2026 )

  • new paper fine tuning with sampling had some breakthrough advancements that makes SFT rival current prevailing postraining methods, where they generalize better and forget less than Reinforcement Learning and OPSD
    • main idea is that RL learning is on policy, which in the past has made it generalize better
    • in this case, SFT can be made “on policy” by using sampling!