All Topics
Scaling, RLHF & Reasoning
What scaling laws say about model size and data, how RLHF shapes a model's behaviour, and where today's reasoning models come from, risks included.
AI FoundationsAI
Videos (2)
Podcasts (1)
Slide decks (1)
Infographics (1)
Sources (11)
- web page Prompt Engineering | Lil'Log
- web page Reward Hacking in Reinforcement Learning | Lil'Log
- web page [2001.08361] Scaling Laws for Neural Language Models
- web page [2005.14165] Language Models are Few-Shot Learners
- web page [2201.11903] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- web page [2203.02155] Training language models to follow instructions with human feedback
- web page [2203.15556] Training Compute-Optimal Large Language Models
- web page [2212.08073] Constitutional AI: Harmlessness from AI Feedback
- web page [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- web page [2307.09288] Llama 2: Open Foundation and Fine-Tuned Chat Models
- web page [2501.12948] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning