AI Data Collection via Crowdsourcing
This course is part of Data Skills for Artificial Intelligence.
This comprehensive course explores how crowdsourcing can be effectively used to gather high-quality data for AI and machine learning systems. Students learn to design and implement crowdsourcing tasks, understand quality control mechanisms, and evaluate machine learning models with human input. The curriculum covers cognitive biases, active learning strategies, and data quality optimization. Through practical examples and case studies, participants develop skills to create robust, unbiased datasets that enhance AI system performance and reliability.
Instructors:
English
Arabic, German, English, 8 more
Powered by
What you'll learn
Understand crowdsourcing principles for gathering AI training data
Implement quality control mechanisms in crowdsourcing tasks
Analyze factors affecting data quality and worker performance
Design effective data creation and model evaluation processes
Apply active learning techniques to optimize data collection
Evaluate and debug machine learning models with human input
Skills you'll gain
This course includes:
PreRecorded video
Graded assignments, exams
Access on Mobile, Tablet, Desktop
Limited Access access
Shareable certificate
Closed caption
Get a Completion Certificate
Share your certificate with prospective employers and your professional network on LinkedIn.
Created by
Provided by
Top companies offer this course to their employees
Top companies provide this course to enhance their employees' skills, ensuring they excel in handling complex projects and drive organizational success.
There are 6 modules in this course
This course provides a comprehensive introduction to creating and collecting high-quality data for AI systems through crowdsourcing. It covers essential aspects of leveraging human intelligence at scale, including task design, quality control mechanisms, and bias mitigation. Students learn about the ImageNet story, worker behavior analysis, incentive structures, and active learning approaches. The curriculum emphasizes practical applications in data collection, annotation, and model evaluation, with special focus on reducing worker effort while maintaining data quality.
Crowdsourcing for High-quality Data Collection and The ImageNet Story
Module 1
Quality Control Mechanisms for Crowdsourcing
Module 2
Factors Affecting Quality in Crowdsourcing
Module 3
Human Input for Data Creation and Model Evaluation in AI
Module 4
Reducing Worker Effort: Active Learning
Module 5
Interpreting, Evaluating, and Debugging ML models
Module 6
Fee Structure
Individual course purchase is not available - to enroll in this course with a certificate, you need to purchase the complete Professional Certificate Course. For enrollment and detailed fee structure, visit the following: Data Skills for Artificial Intelligence
Instructors
3 Courses
Pioneer in Human-Centered AI and Crowd Computing Innovation
Ujwal Gadiraju serves as an Assistant Professor at TU Delft's Web Information Systems group in the Faculty of Electrical Engineering, Mathematics and Computer Science, where he directs the Delft AI "Design@Scale" Lab and co-leads the Kappa research line on Crowd Computing and Human-Centered AI. His academic journey includes a PhD in Computer Science with summa cum laude from Leibniz University of Hannover and an MSc from TU Delft, followed by postdoctoral research at the L3S Research Center. As a Distinguished Speaker of the ACM and board member of CHI Netherlands, he has established himself as a leading voice in human-computer interaction research. His work spans the intersection of Human-Computer Interaction, Artificial Intelligence, and Information Retrieval, with particular emphasis on Crowd Computing. His recent notable publications include groundbreaking research on human-AI bargaining behavior, metaphorical representation in conversational crowdsourcing, and innovative approaches to knowledge elicitation through gaming. He currently serves as co-editor for two Frontiers in AI journals, associate editor for the Behavior and Information Technology journal, and co-editor-in-chief of the Human Computation Journal. His research has garnered significant recognition, with over 200 peer-reviewed publications and 10 paper awards at top-tier HCI and AI conferences. Through his work, he aims to create novel methods, interfaces, and tools to enhance human-AI interaction and build more effective, inclusive AI systems that people can rely on appropriately.
3 Courses
Pioneer in Human-Centered Machine Learning and Crowd Computing
Jie Yang serves as an Assistant Professor in the Web Information Systems group at TU Delft, where he co-leads the Kappa research line on Crowd Computing and Human-Centered AI and manages the GENIUS lab focusing on Generative AI development. His academic journey includes a PhD from TU Delft (2017), an MSc from TU Eindhoven (2013), and a BEng from Zhejiang University (2011), followed by industry experience as a Machine Learning Scientist at Amazon's Alexa Shopping and a Senior Researcher at the University of Fribourg's eXascale Infolab. His research centers on human-in-the-loop approaches for trustworthy machine learning, developing methods and tools that involve stakeholders throughout the ML lifecycle. His work has garnered significant recognition, with notable publications on unknown unknowns characterization in image recognition, diverse knowledge elicitation through gaming, explainability methods for bug identification in computer vision models, and neuro-symbolic systems for label noise reduction. As co-director of the TU Delft AI Lab Design@Scale, he focuses on transforming machine learning into an engineering discipline that ensures human control over AI systems. His expertise spans human-centered AI, crowd computing, natural language processing, and information retrieval, with particular emphasis on developing approaches that align AI systems with human values and needs. Through his research and leadership, Yang contributes to making AI systems more reliable, transparent, and effectively integrated into various real-world contexts.
Testimonials
Testimonials and success stories are a testament to the quality of this program and its impact on your career and learning journey. Be the first to help others make an informed decision by sharing your review of the course.
Frequently asked questions
Below are some of the most commonly asked questions about this course. We aim to provide clear and concise answers to help you better understand the course content, structure, and any other relevant information. If you have any additional questions or if your question is not listed here, please don't hesitate to reach out to our support team for further assistance.