Hands-on course on deep learning training and inference for engineers and programmers using Python and Keras (TensorFlow)

**PLEASE NOTE: **This is a LIVE INSTRUCTOR-LED training event delivered ONLINE. It covers the same scope and content as a scheduled in-person class and delivers comparable learning outcomes.

**Practical Deep Learning** is designed to meet the needs of competent professionals, already working as engineers or computer programmers, who are looking for a solid introduction to the subject of deep learning *training* and *inference* combined with sufficient practical, hands-on training to enable them to start implementing their own deep learning systems.

Many vendors now offer deep neural network models off-the-shelf for you to use in your own applications. What the vendors might not tell you is that the level of expertise required to collect and curate a training dataset and then adapt and re-train a pre-existing model for your own application is not so different from the level of expertise you would need to create such a model from scratch. This 5-day class will teach you all you need to get started with confidence, reducing your learning curve from months to days.

The course is based on the Python programming language and makes extensive use of the Keras neural network API, the approved high-level API of the TensorFlow machine learning framework, as well as Numpy, Matplotlib, Pandas, Scikit-learn, and TensorBoard. Although based on Keras, the principles and concepts taught in this training course would be equally applicable in any deep learning library or framework.

This live ONLINE training is delivered over five consecutive days with a 6-hour interactive session each day (including time during the session for breaks). Each session includes live tuition from a Doulos subject matter expert. Each session also includes hands-on-labs using a lab platform accessed over the internet. Each attendee will require an internet connection, a web browser, and appropriate software and firewall settings. Individual (private) instructor support is available to each attendee during each session. Attendees have the option of completing some of the labs outside of the 6-hour sessions. Additional lab material is available in some topics for completion outside of the sessions.

In addition to the live ONLINE sessions, certain training materials are provided for self study after the course proper. These additional materials take the form of online videos complemented by written notes. Materials provided for self study include the more advanced or supplemental topics (which would be taught on the final day of a 5-day face-to-face Practical Deep Learning training course).

There are plenty of materials and training options out there for machine learning and deep learning, everything from e-books and blogs through online tutorials and MooCs to the formal courses offered by universities and corporations. Some of these are excellent, and they each have their place. The challenge is knowing where to start and having the time to research and study what you need to know. As a specialist provider of technical training, Doulos has been able to condense the essential knowledge and skills you need to get started with deep learning into this one 5-day training course.

Engineers, programmers, or other people with a technical or mathematical background who want a comprehensive, hands-on introduction to the subject of deep learning.

- The basics of Python programming using Jupyter Notebook
- The principles and practices of supervised learning and deep learning
- How to use neural networks to solve regression and classification problems
- How to use unsupervised learning for visualization and dimensionality reduction
- How to use convolutional neural networks for image classification
- How to use Keras and TensorBoard
- How to perform inference using pre-built neural network models
- How to take advantage of pre-trained neural network models using transfer learning
- How to prepare and curate datasets for deep learning
- Deep learning concepts and techniques in current use such as gradient descent algorithms, learning curves, regularization, dropout, batch normalization, the Inception architecture, residual networks, pruning and quantization, the MobileNet architecture, word embeddings, and recurrent neural networks
- An introduction to generative adversarial networks and object detection
- The principles behind the growing number of vendor-specific flows now available for deploying neural network models for inference in the cloud and in embedded devices

This course is not a high-level overview of deep learning for managers, business developers, or end users of machine learning technology, although attendees will certainly gain an excellent overview of deep learning by attending this course. This is a detailed, hands-on course.

This is not a course in machine learning or artificial intelligence as such, but in deep learning. Deep learning (neural networks) is one specific branch of machine learning, which is a branch of artificial intelligence.

This is not a course in mathematics, statistics, or data science. This course assumes you already have the necessary mathematical background (see prerequisites below).

This course is not for professional mathematicians or machine learning researchers. It is for programmers and implementers.

Attendees should be experienced and competent in at least one object-oriented programming language (e.g. Python, Ruby, C++, C#, Java, or SystemVerilog). Prior programming experience with Python would be useful but is not required.

Attendees should be familiar with the following mathematical concepts:

- Continuous functions of one or more variable, linear and non-linear functions, exponential functions
- Very basic differential calculus - derivatives and partial derivatives
- Very basic statistics - mean, standard deviation, variance, probability, histograms, normal distribution
- Basic linear algebra - vectors, matrices, summation, dot product

Attendees do not need a university degree in mathematics. The emphasis of this course is on practical computer programming, not on mathematical theory. This course does not require the attendee to write or solve mathematical equations, nor does it require the attendee to read or understand any mathematical proofs. However, attendees do need a willingness to immerse themselves in what is essentially a mathematical topic.

Doulos training materials are renowned for being the most comprehensive and user friendly available. Their style, content and coverage is unique, and has made them sought after resources in their own right. You get to keep the following materials at the end of the course:

- A fully indexed set of class notes that form a complete reference manual
- Jupyter Notebooks containing complete working code for all of the neural networks presented during the training class, which you can use after the class for revision or as the basis for your own networks.

**ONLINE SESSION DAY 1**

**Introduction to Deep Learning**

AI versus ML versus Deep Learning • "Classical" Machine Learning • Deep Learning • Supervised Learning • Unsupervised Learning • Neural Networks • Cloud versus Edge Computing • Applications • Libraries/Frameworks for Training • Cloud Platforms for Training and Inference • Vendor Platforms for Deploying ML / DL

**Jupyter Notebook**

AWS Deep Learning AMI • Connect to Remote Machine using SSH • Using Jupyter Notebook • Basic Markdown • Output and Evaluating Expressions • Expanding, Collapsing, Hiding Output • Menus and Tool Bar

**Python Basics**

Functions, Variables, and Values • Control Statements • Operators • Imports • Instance Objects • Kwd arguments • Range • Format • Tuples and Lists • Functions returning Functions • Numpy Array • Multi-dimensional Numpy array • Reshape • Slices • Elementwise and Scalar Operations • One-Hot Encoding • Reduction Operations • matplotlib.pyplot • List Operations

**Linear Regression**

Regression Task • Defining a Model • Cost / Loss / Error Function • Cost as a Function of Trainable Parameters • Mathematical Optimization • Contour Plot of Cost Function • Gradient Descent • Stochastic Gradient Descent

**ONLINE SESSION DAY 2**

**Keras**

TensorFlow and Keras • Versions of Keras • A Keras Sequential Model • Datasets • Gradient Descent in Keras • History Object • Plotting Progress • Model, Optimizer, and Weights

**Logistic Regression**

Classification Task • One-Hot Labels • The Hypothesis or Model • Calculating the Cost Function • Converting Scores to Probabilities • The Softmax Function • Compare using Cross-Entropy • Multinomial Logistic Regression • Plotting the Decision Boundary • Choosing the Loss Function

**Neural Networks**

Neural Networks • An Artificial Neuron • Common Activation Functions • A Deep Neural Network • Forward and Back-Propagation • Kinds of Neural Network

**Non-linear Regression**

Linear Regression • A Non-Linear Polynomial Model • The Rectified Linear Unit (ReLU) • Normalizing the Data • Exploding and Vanishing Gradients • Varying the Weight Distribution • Xavier Glorot Initialization • Non-Linear Keras Model • The Magic of Deep Neural Networks

**ONLINE SESSION DAY 3**

**Non-linear Classification**

A Non-Linear Decision Boundary • Decision Boundary and Softmax • Non-Linear Neural Network for Classification • From ReLU to Decision Boundary • Softmax

**Overfitting and Regularization**

Training versus Test Datasets • Scikit-learn • Learning Curves • Matching the Network to the Problem • How to Reduce Overfitting? • More Data? • Regularization (L2 Regularization) • Choosing Lambda • L2 versus L1 Regularization

**Stochastic Gradient Descent**

Full-Batch vs Stochastic Gradient Descent • Mini-Batches • The Landscape of the Cost Function • Stationary Points • Learning Rate • Learning Rate Decay Schedule • Momentum • Nesterov Momentum • Adaptive Per-Parameter Learning Rates • Adam Algorithm

**Splitting the Dataset**

The MNIST Dataset • A Deep Neural Network for Classification • Hyperparameters • Training, Validation, and Test Datasets • K-Fold Cross-Validation • Validatation • Choose a Single Scalar Metric • Imbalanced Classes or Rare Events • ROC Curve • Trading off Precision and Recall

**ONLINE SESSION DAY 4**

**Convolutional Neural Networks**

Convolution • Patch Size and Stride • Valid Padding versus Same Padding • Network Size • Multiple Features Maps • Pooling • Stride Versus Receptive Field • Hierarchical Feature Detection • Filter Visualization • Number of Parameters and Values • Plotting Convolution Filters

**Visualization using TensorBoard**

Visualizing a Graph in TensorBoard • Visualizing Scalars in TensorBoard • Visualizing Multiple Runs • Visualizing Weights and Activations • Highlighting a Histogram • Visualizing an Embedding in 3-D Space • Keras fit versus fit_generator • Using TensorBoard from Keras

**PCA, t-SNE, and K-Means**

Dimensionality Reduction • Principal Component Analysis • PCA for Visualization • t-SNE for Visualization • Clustering with K-Means • PCA for Dimensionality Reduction • Linear Regression Out-of-the-Box • Normalizing the Dataset

**Data Preparation**

The Deep Learning Process • CRISP-DM • Feature Engineering • Correlated Features, Missing or Bad Values • Choose Meaningful Features • Avoid Magic Values • Pandas Dataframe • Pandas Summary Statistics • Pandas Scatter Matrix • Cleaning Data with Pandas • TensorFlow Extended (TFX) • Error Analysis • Artificial Data Synthesis • Data Augmentation

**ONLINE SESSION DAY 5**

**Dropout and Batch Normalization**

Dropout • When to use Dropout? • Batch Normalization • Scale-and-Shift • Benefits of Batch Normalization • Calculating the Scaling Factors

**Inception and Residual Networks**

General Principles of Network Architecture • Evolution of CNN Architectures • Principles of the Inception Architecture • Fully-Connected versus Sparse • Inception Module • Global Average Pooling • Fully Convolutional Network • Residual Networks • Matching Dimensions • Performance of Inception and ResNet

**Transfer Learning**

Why Transfer Learning? • Re-use trained weights • Simple Transfer Learning in Keras • A Pre-trained Inception Network • Fine-Tuning Previous Layers • Saving and Loading • Make Each Class Visually Distinct • Other Tips for Image Data

**MATERIALS FOR SELF STUDY AFTER THE COURSE**

**Pruning and Quantization**

Inference Engines at the Edge • ML / DL Tool Flow for Edge Computing • Neural Network Exchange Formats • Network Pruning • Quantization • 8-Bit Quantization • TensorFlow Lite Post-Training Quantization • OpenCV • OpenVX • OpenCL

**MobileNet**

MobileNet • Depthwise-Separable Convolution • MobileNet V1 Architecture • MobileNet V2 Architecture • Hyperparameters • MobileNet Family • MobileNet Inference in TensorFlow • CNNs Compared

**Encoding and Word Embedding**

Coding Categorical Data • Binning • Text in a Neural Network • Word Embedding and Semantics • Hidden Layers and Latent Features • The Word2vec Algorithm • Negative Sampling Neural Network • Preparing the Training Dataset • Frequency Counts

**Recurrent Neural Networks**

Recurrent Neural Network (RNN) • RNN Applications • Long Short Term Memory – LSTM • LSTM Gates • LSTM Connections • Gated Recurrent Unit – GRU • Simple Character-Level RNN in Keras • LSTM trained on Linux Source Code • Bidirectional LSTM • Networks for Images, Sound, Text, Video

**GANs**

Putting Networks End-to-End • Generative Adversarial Network (GAN) • Deep Convolutional GAN Generator • Generated Images • GAN to Generate MNIST Digits

**Object Detection**

Datasets for Object Detection • Google Open Images Dataset • Networks for Object Detection • Object Detection – Faster R-CNN • Training a Faster R-CNN Network • Image Segmentation using Mask R-CNN • Pixel-Level Image Segmentation

17 Jun 2024 | ONLINE EurAsia | Enquire |

29 Jul 2024 | ONLINE Americas | Enquire |

05 Aug 2024 | ONLINE EurAsia | Enquire |

23 Sep 2024 | ONLINE Americas | Enquire |

23 Sep 2024 | ONLINE EurAsia | Enquire |

18 Nov 2024 | ONLINE Americas | Enquire |

16 Dec 2024 | ONLINE EurAsia | Enquire |

Complete an enquiry form and a Doulos representative will get back to you.

Enquiry FormPrice on request