Review

Please take this Python mini-quiz before the course and take this Python mini-course if you need to learn Python or refresh your Python knowledge.

8/27

Lecture 1: Introduction

• Big Data applications
• Technologies for handling big data
• Apache Hadoop and Spark overview

8/27

8/29

Lecture 2: Hadoop Fundamentals

• Hadoop architecture
• HDFS and the MapReduce paradigm
• Hadoop ecosystem: Mahout, Pig, Hive, HBase, Spark

9/3

Lecture 3: Introduction to Apache Spark

• Big data and hardware trends
• History of Apache Spark
• Spark's Resilient Distributed Datasets (RDDs)
• Transformations and actions

9/10

Lecture 4: Machine Learning Overview

• Basic machine learning concepts
• Steps of typical supervised learning pipelines
• Linear algebra review
• Computational complexity / Big O notation review

9/12

9/17

Lecture 5: Linear Regression and Distributed ML Principles

• Linear regression
  • formulation and closed-form solution
  • gradient descent
  • grid search
• Distributed machine learning principles
• computation, storage, and communication

9/19

9/24

Lecture 6: Logistic Regression and Click-through Rate Prediction

• Online advertising
• Linear classification
• Logistic regression
• working with probabilistic predictions
• categorical data and one-hot-encoding
• feature hashing for dimensionality reduction

9/26

10/1

Lecture 7: Principal Component Analysis and Neuroimaging

• Exploratory data analysis
• Principal Component Analysis (PCA)
• Formulations and solution
• Distributed PCA

10/3

Lecture 8: Big Data ML with MLlib

• k-means Clustering
• Decision Trees and Random Forests
• Recommenders

10/8

Lecture 9: Introduction to SparkSQL

• Working with tables in Spark
• Higher-level declarative programming

10/10

Lecture 10: Analyzing Networks with GraphX

• Understanding network structure
• Computing graph statistics

See here

Final Exam