Curated materials to help you prepare and get the most out of your workshop experience
Welcome! We've carefully selected a collection of high-quality learning materials to help you prepare for the SCIPE Workshop on LLMs. Whether you're new to the field or looking to deepen your understanding, these resources will provide a solid foundation for the workshop.
Take your time exploring these materials at your own pace. You don't need to complete everything before the workshop, but familiarizing yourself with the basics will help you get more value from the hands-on sessions.
Start here to build a strong foundation in LLMs
An excellent starting point that covers the fundamentals of LLM architecture, training, and applications. Andrej Karpathy's clear explanations make complex concepts accessible to newcomers while providing valuable insights for experienced practitioners.
Watch VideoPractical insights on leveraging LLMs in real-world workflows. Learn how an expert integrates these tools into daily work and discovers what's possible with current technology.
Watch VideoA comprehensive playlist that covers LLM fundamentals and chatbot development. These videos break down complex topics into digestible segments, perfect for building understanding step by step.
View PlaylistAn in-depth walkthrough of implementing GPT-2 from scratch. This is for those who want to understand the intricate details of transformer architecture and training. Highly rewarding for research-oriented participants.
Watch VideoPython AI tutorial on building retrieval-augmented generation systems. This practical guide shows you how to enhance LLMs with external knowledge, a technique we'll explore in the workshop.
Watch TutorialBeautiful visual explanations of neural network concepts. If you're looking to understand the mathematical foundations with intuitive animations, this series is unmatched. Essential for grasping how neural networks actually work.
View PlaylistComprehensive university courses on LLMs and transformers
A comprehensive course dedicated to transformer architectures and large language models. Covers both theoretical foundations and practical implementations, ideal for understanding the complete picture of modern LLMs.
View Playlist | Course PageBuild language models from the ground up. This course provides deep understanding of how transformers and language models work at a fundamental level.
View Playlist | Course PageExplore deep reinforcement learning techniques that are increasingly important for training and fine-tuning LLMs with human feedback.
First Lecture | Course PageDive deeper into NLP with this advanced course. Features video lectures covering cutting-edge topics in natural language processing. Great for those who want to explore beyond the basics.
View Playlist | Course PageFocused course on inference optimization techniques and test-time scaling. Perfect for understanding how to make LLMs faster and more efficient, a topic we'll cover extensively in the workshop.
First Lecture | Course PageBuild a strong foundation in deep learning concepts and applications. This course covers the fundamentals that underpin all modern LLMs. Recommended if you're new to deep learning.
First Lecture | Course PageExplore diffusion models, another important class of generative AI models. Understanding these techniques provides broader context for generative AI beyond language models.
View Playlist | Course PageDeep dive into flow and diffusion models with mathematical foundations. Advanced course for those interested in the theoretical aspects of generative models.
View Playlist | Course PageMathematical foundations for understanding LLMs
Intuitive visual explanations of linear algebra concepts. Perfect for understanding the mathematical foundations behind neural networks and transformers.
View PlaylistComprehensive introduction to linear algebra from MIT. Covers vectors, matrices, eigenvalues, and more - all essential for understanding deep learning.
View Playlist | Course PageAdvanced matrix methods with direct applications to machine learning. Covers singular value decomposition, principal component analysis, and optimization techniques.
View Playlist | Course PageVisual intuition for calculus concepts. Essential for understanding gradients, derivatives, and the optimization algorithms used in training neural networks.
View PlaylistSpecialized course on matrix calculus with focus on machine learning applications. Covers gradients, Jacobians, and automatic differentiation.
View Playlist | Course PageEssential probability concepts for computer science and machine learning. Covers distributions, random variables, and statistical inference.
View Playlist | Course PageComprehensive introduction to probability theory. In-depth coverage of probability concepts with clear explanations and examples.
View Playlist | Course PagePriority: Start with Andrej Karpathy's 1-hour introduction, then watch the 3Blue1Brown neural networks series. These two resources will give you a solid conceptual foundation.
Optional: Browse the IBM Technology playlist for specific topics that interest you.
Priority: Review "How I Use LLMs" and the RAG tutorial to see practical applications. Look at the Stanford CME 295 syllabus to identify areas where you want to deepen your knowledge.
Optional: Explore the inference optimization course if you're interested in deployment efficiency.
Priority: Review the advanced NLP course and inference algorithms course. Consider watching the GPT-2 reproduction video to understand implementation details.
Optional: Dive into the CS 230 materials if you want to refresh your deep learning fundamentals.
6+ weeks before: Start with the foundational videos and courses that interest you most.
2-4 weeks before: Focus on areas relevant to your goals (practitioner tools or research topics).
1 week before: Review key concepts and make sure you have your development environment set up.
Remember: These are suggestions to enhance your workshop experience, not requirements. Come with curiosity and a willingness to learn!
Registration is open from November 13 to December 25, 2025
Register for the Workshop