Machine Learning With Big Data

An introduction to the course covering the welcome message and a summary of the previous course on Big Data Integration and Processing.

This module provides an overview of machine learning concepts, categorizes different machine learning techniques, explains the machine learning process including CRISP-DM, discusses scaling machine learning algorithms, and introduces the tools used in the course.

This module covers data terminology, exploration techniques including summary statistics and visualization plots, and demonstrates how to explore data using KNIME and Spark. It introduces the Daily Weather Dataset and provides hands-on experience with data exploration tools.

This module focuses on data preparation techniques for machine learning, covering data quality issues and how to address them, feature selection and transformation, dimensionality reduction, and handling missing values in both KNIME and Spark environments.

This module introduces classification in machine learning, including how to build and apply classification models. It covers various classification algorithms such as k-Nearest Neighbors, Decision Trees, and Naïve Bayes, with practical implementation using Decision Trees in KNIME and classification in Spark.

This module focuses on model evaluation, covering generalization and overfitting concepts, particularly in decision trees. It explains validation sets, metrics for evaluating model performance, confusion matrices, and demonstrates practical evaluation of decision trees in both KNIME and Spark environments.

This final module covers three additional machine learning techniques: regression (with focus on linear regression), cluster analysis (particularly k-means clustering), and association analysis. It includes practical implementation of cluster analysis in Spark using the Minute Weather Dataset and concludes the course.