Generative AI Model Selection Fundamentals

Meet Professor Jesse Spencer-Smith, an experienced practitioner in the field of artificial intelligence. Learn about the course structure and its significance in today's AI-driven world. This lesson sets the foundation for understanding the critical role of model selection in AI implementation and introduces the key concepts you'll master throughout the course.

Dive into the vast and diverse world of AI models. You'll explore the basic architecture of generative AI, understanding key components like tokenization, semantic spaces, and the decoder stack. This lesson covers the variations in model capabilities, from multimodal processing to long-context understanding. You'll also examine different deployment options, including web access, APIs, and local hosting, understanding the trade-offs in terms of security, cost, and customizability.

Learn how to effectively evaluate and compare AI models for your specific needs. This lesson introduces you to benchmarking techniques, including industry-standard leaderboards and their limitations. You'll discover how to address challenges like the ceiling effect and contamination in model evaluation. Importantly, you'll learn to create your own benchmarks tailored to your unique tasks, ensuring you can accurately assess model performance for your particular use case.

Sometimes, off-the-shelf models don't meet all your requirements. In this final lesson, you'll explore strategies for enhancing model performance. Learn about prompt engineering, in-context learning, and data augmentation techniques. Dive deep into Retrieval Augmented Generation (RAG) and understand its advantages and limitations compared to long-context models. By the end of this lesson, you'll have a toolkit of strategies to optimize AI model performance for your specific needs.