Digital Resources for Chapter 9: Deep Learning - MLP Neural Networks Explained

Neural Network Video from StatQuest

A video from StatQuest by Josh Starmer. This video is Part 1 from a series of eight videos about Neural Networks. While most of the videos exceed the scope of this book, Part 1 complements topics in the book.

How Two ReLU Functions Can Form a Complete Step Function

This is a blog post by Carsten Lange that shows how two Rectified Linear 
Unit (ReLU) activation functions can be combined into a complete step function (squashing function). The post includes a  simulation that allows for changing the beta values of two ReLU functions with the goal of creating a complete step function.

R Script for Tuning and Training a Neural Network to Predict Diamond Prices

Here you can download an R script to tune and train a neural network to predict diamond prices. The R script contains similar code as the related interactive section of the book. In contrast to the interactive section, the code runs in RStudio instead of a browser, and the maximum runtime is not restricted. 

The R script loads the complete set of observations from the diamonds dataset,  and all hyper-parameters can be tuned, The maximum number of epochs is set to 1000 and can be changed.

It is recommended that you experiment with tuning settings in the R script (see Step 5) and try to improve the cross-validation results. At the very end of the R script, you find sample tuning settings that produce relatively good results but require between one to several hours of computing time, depending on your computer. Use the sample tuning setting only after you have experimented with the tuning, and then see if your results can beat the sample tuning results.

Neural Networks Explained in 5 minutes

This video provides an introduction to Neural Networks. The video is part of an IBM series for Machine Learning. It is easy to understand and one of the most viewed videos about neural networks on YouTube.

Build a Convolutional Neural Network with R by DataCamp

Using Keras and Tensor Flow to predict images.

How Deep Neural Networks Work

A basic introduction to neural networks by Brandon Rohrer. The video uses a four pixel image example to introduce the principles of neural networks in a little more than 10 minutes. The video has more than a million viewers.

R Script: Demonstrate How Closely a Neural Network can Approximate Data

This R script shows how well a neural network can approximate non-linear data, given that it has enough hidden neurons. The script includes code to visualize the approximation in 3D (the related packages have to be installed before).

The R script also shows that an extremely good approximation of the training data usually leads to the problem of overlearning.

But what is a neural network? Chapter 1, Deep learning by 3Blue1Brown

This is the first video of a series of videos about Deep Learning by 3Blue1Brown. This video series is of very high quality and extremely successful on YouTube. The video series is more advanced than the level of the book but well worth exploring.

Why Data should be Normalized before Training a Neural Network

A Visual Explanation of Gradient Descent Methods (Momentum, AdaGrad, RMSProp, Adam) by Lili Jiang

Neural Network Video from StatQuest - ReLu Activation Function

This is a StatQuest video about Neural Networks from a series of eight videos. This video covers Rectified Linear Unit activation function.

Hyper-parameter Tuning Techniques in Deep Learning

The process of setting the hyper-parameters requires expertise and extensive trial and error. There are no simple and easy ways to set hyper-parameters - specifically, learning rate, batch size, momentum, and weight decay.

What is a Neural Network? Article by Ethan Huang in Analytics Vidhya.

This article provides an intuitive and broad overview about neural networks.