95-865: Unstructured Data Analytics (Spring 2021 Mini 4)

Lecture 1: Course overview
[slides]

Lecture 2: Basic text analysis, co-occurrence analysis
[slides]

For the basic text analysis demo to work, please install Anaconda Python 3, Jupyter, and spaCy first
[Jupyter notebook (basic text analysis)]

Recitation: Basic Python review
[Jupyter notebook]

Lecture 3: Finding possibly related entities
[slides]
[Jupyter notebook (co-occurrence analysis)]

What is the maximum phi-squared/chi-squared value? (technical)
[stack exchange answer]

Lecture 4: Visualizing high-dimensional data with PCA
[slides]
[Jupyter notebook (PCA)]

Causality additional reading:
[Computational and Inferential Thinking, "Causality and Experiments" chapter]

PCA additional reading (technical):
[Abdi and Williams's PCA review]

Recitation slot — Lecture 5: Manifold learning with Isomap
[slides]
[Jupyter notebook (manifold learning)]

Python examples for dimensionality reduction:
[scikit-learn example (PCA, Isomap, t-SNE, and many other methods)]

No class (CMU break day)

Lecture 6: Wrap up manifold learning (t-SNE), a first look at analyzing images, and an introduction to clustering phenomena
[slides]
[Jupyter notebook (manifold learning); same demo as previous lecture]
[required reading: "How to Use t-SNE Effectively" (Wattenberg et al, Distill 2016)]
For the demo below to work (on t-SNE with images), you will need to install some packages:
pip install torch torchvision
[Jupyter notebook (t-SNE with images)]
[slides with some technical details for t-SNE]

HW1 due 11:59pm Pittsburgh time

See supplementary materials from the previous lecture; in addition, here's some reading for t-SNE (technical):
[Simon Carbonnelle's t-SNE slides]
[t-SNE webpage]

Recitation: More on PCA, practice with argsort
[Jupyter notebook]

Lecture 7: Distance and similarity functions, clustering (k-means, GMMs)
[slides]
[Jupyter notebook (PCA, t-SNE, clustering with drug data)]

Clustering additional reading (technical):
[see Section 14.3 of the book "Elements of Statistical Learning" on clustering]

Lecture 8: More on clustering (interpreting clustering results, automatically choosing the number of clusters for GMM-related models)
[slides]
[Jupyter notebook (PCA, t-SNE, clustering on UCI drug consumption data); same demo as previous lecture]

Reading on DP-means, DP mixture models for which a DP-GMM is a special case (technical):
[Revisiting k-means: New Algorithms via Bayesian Nonparametrics]

Lecture 9: Topic modeling
[slides]
[Jupyter notebook (topic modeling with LDA)]

Topic modeling reading:
[David Blei's general intro to topic modeling]
[(technical) Topic Modelling Meets Deep Neural Networks: A Survey]

Lecture 10: Wrap up topic modeling; wrap up clustering; a glimpse of predictive data analytics
[slides]
[Jupyter notebook (topic modeling with LDA); same demo as previous lecture]
[Jupyter notebook (PCA, t-SNE, clustering on UCI drug consumption data); same demo as previous lecture]

HW2 due 11:59pm Pittsburgh time

Quiz 1:

• Recitation slot (3:10pm-4:30pm Pittsburgh time) for Sections A4/B4
• 6:30pm-7:50pm Pittsburgh time for Section Z4

Lecture 11: Intro to predictive data analytics
[slides]

Some nuanced details on cross-validation (technical):
[Andrew Ng's article Preventing "Overfitting" of Cross-Validation Data]
[Braga-Neto and Dougherty's article Is cross-validation valid for small-sample microarray classification? (this article applies more generally rather than only to microarray data from biology)]
[Bias and variance as we change the number of folds in k-fold cross-validation]

Lecture 12: Wrap up basic prediction concepts
[slides]
[Jupyter notebook (prediction and model validation; same demo as last time)]

Recitation: More practice on model evaluation
[Jupyter notebook]

Lecture 13: Intro to neural nets and deep learning
[slides]
For the neural net demo below to work, you will need to install some packages:
pip install torch torchvision torchaudio
pip install torchsummaryX
python -m spacy download en
pip install pytorch-nlp
[Jupyter notebook (handwritten digit recognition with neural nets; be sure to scroll to the bottom to download UDA_pytorch_utils.py)]

PyTorch tutorial (at the very least, go over the first page of this tutorial to familiarize yourself with going between NumPy arrays and PyTorch tensors, and also understand the basic explanation of how tensors can reside on either the CPU or a GPU):
[PyTorch tutorial]

Additional reading:
[Chapter 1 "Using neural nets to recognize handwritten digits" of the book Neural Networks and Deep Learning]

Video introduction on neural nets:
["But what *is* a neural network? | Chapter 1, deep learning" by 3Blue1Brown]

Mike Jordan's Medium article on where AI is at (April 2018):
["Artificial Intelligence - The Revolution Hasn't Happened Yet"]

Lecture 14: Image analysis with convolutional neural nets
[slides]
[Jupyter notebook (handwritten digit recognition with neural nets; same demo as previous lecture)]

Additional reading:
[Stanford CS231n Convolutional Neural Networks for Visual Recognition]
[(technical) Richard Zhang's fix for max pooling (presented at ICML 2019)]

Recitation slot: Lecture 15 on time series analysis with recurrent neural nets; more deep learning topics; course wrap-up
[slides]

The demo will not be covered during the recitation slot and is instead covered in the April 30 Section K4 recitation Zoom recording by your TA Erick (check Canvas Zoom recordings)
[slides]
For the demo below to work, be sure to install the prerequisite packages as mentioned for the lecture 13 demo.
[Jupyter notebook (sentiment analysis with IMDb reviews)]

Additional reading:
[Christopher Olah's "Understanding LSTM Networks"]
[A tutorial on word2vec word embeddings]
[A tutorial on BERT word embeddings]

HW3 due 11:59pm Pittsburgh time

Quiz 2:

• 1pm-2:20pm Pittsburgh time for Sections A4/B4
• 5:30pm-6:50pm Pittsburgh time for students in A4/B4 taking the alternate Z4 time
• Students officially in Section Z4 have been sent separate instructions for their Quiz 2 (check Canvas)