Data Pipeline Development with ETL Tools
This course is part of multiple programs. Learn more.
This comprehensive course focuses on building and managing data pipelines and ETL processes essential for Business Intelligence platforms. Students learn to create automated workflows using shell scripts, Apache Airflow, and Apache Kafka. The curriculum covers both batch and streaming data pipelines, with hands-on practice in extracting, transforming, and loading data into staging areas. Through real-world projects, participants gain practical experience in implementing ETL and ELT processes using various technologies. The course emphasizes the importance of well-designed data workflows for successful data warehousing and analysis.
4.5
(6 ratings)
8,294 already enrolled
English
English
Powered by
What you'll learn
Create and manage ETL and ELT processes effectively
Build automated data pipelines using Bash shell scripts
Develop batch processing workflows with Apache Airflow
Implement streaming data solutions using Apache Kafka
Design efficient data transformation and loading processes
Skills you'll gain
This course includes:
PreRecorded video
Graded assignments, exams
Access on Mobile, Tablet, Desktop
Limited Access access
Shareable certificate
Closed caption
Top companies offer this course to their employees
Top companies provide this course to enhance their employees' skills, ensuring they excel in handling complex projects and drive organizational success.
Module Description
This course provides comprehensive training in building and managing data pipelines for business intelligence applications. The curriculum covers ETL and ELT processes, data pipeline components, and various tools and technologies. Students learn to create automated workflows using Bash shell scripts, develop batch processing with Apache Airflow, and implement streaming solutions with Apache Kafka. The course includes hands-on labs and a real-world project, giving students practical experience in data engineering solutions.
Fee Structure
Individual course purchase is not available - to enroll in this course with a certificate, you need to purchase the complete Professional Certificate Course. For enrollment and detailed fee structure, visit the following: Data Engineering, Data Warehouse Engineering, SQL, ETL and BI Fundamentals
Instructors
AI and Machine Learning Expert at IBM Canada
Yan Luo serves as a Data Scientist and Developer at IBM Canada, where he applies his expertise in machine learning and artificial intelligence to develop innovative cognitive applications across diverse domains including software repository mining, personalized health management, wireless networks, and digital banking. After earning his Ph.D. in Machine Learning from the University of Western Ontario, he has contributed significantly to technical education through developing and teaching multiple data science courses, including Applied Data Science Capstone, Machine Learning Capstone, and Introduction to R Programming for Data Science. His work focuses on practical applications of AI and cognitive computing, bridging the gap between theoretical machine learning concepts and real-world business solutions.
Technology Education and Skills Development Leader at IBM
Rav Ahuja serves as the Chief Content Officer and Global Program Director at IBM Skills Network, where he leads curriculum creation, growth strategy, and partner programs. After earning his B.Eng. from McGill University and MBA from the University of Western Ontario, he co-founded Cognitive Class, an IBM initiative focused on democratizing access to in-demand technology skills. Based at the IBM Canada Lab in Toronto, he specializes in developing instructional solutions for AI, Data Science, Cloud Computing, and Cybersecurity. His impact on technology education is evidenced through his role as architect of numerous IBM Professional Certificates and instructor for over 35 online courses, including popular offerings like "What is Data Science?", "Introduction to Cloud Computing," and "Introduction to Artificial Intelligence (AI)." His courses have reached hundreds of thousands of learners worldwide, with "What is Data Science?" alone enrolling over 638,000 students. His recent work includes developing new Generative AI courses and career guidance content for IBM's Professional Certificate programs, demonstrating his ongoing commitment to preparing learners for emerging technology careers
Testimonials
Testimonials and success stories are a testament to the quality of this program and its impact on your career and learning journey. Be the first to help others make an informed decision by sharing your review of the course.
Frequently asked questions
Below are some of the most commonly asked questions about this course. We aim to provide clear and concise answers to help you better understand the course content, structure, and any other relevant information. If you have any additional questions or if your question is not listed here, please don't hesitate to reach out to our support team for further assistance.