Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure

• Comparison between biological and artificial neurons.
• The underlying model of an ANN.
• Activation functions employed in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning Mechanisms

• Review of vector and matrix algebra.
• State-space concepts.
• Principles of optimization.
• Error-correction based learning.
• Memory-driven learning.
• Hebbian learning.
• Competitive learning.

Single Layer Perceptrons

• Architecture and learning processes of perceptrons.
• Pattern classification: introduction and Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• Convergence of perceptrons.
• Inherent limitations of perceptrons.

Feedforward ANNs

• Structure of Multi-layer feedforward networks.
• The backpropagation algorithm.
• Training and convergence via backpropagation.
• Functional approximation using backpropagation.
• Practical considerations and design challenges in backpropagation learning.

Radial Basis Function Networks

• Pattern separability and interpolation techniques.
• Theory of regularization.
• Integration of regularization with RBF networks.
• Designing and training RBF networks.
• Approximation characteristics of RBFs.

Competitive Learning and Self-Organizing ANNs

• General clustering methodologies.
• Learning Vector Quantization (LVQ).
• Algorithms and architectures for competitive learning.
• Self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks

• Neuro-fuzzy systems.
• Foundations of fuzzy sets and logic.
• Designing fuzzy systems.
• Architecting fuzzy ANNs.

Applications

• Discussion of select Neural Network applications, highlighting their benefits and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets: consistent cases
  • Guarantees for finite hypothesis sets: inconsistent cases
  • General considerations
    • Deterministic vs. stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC Dimension
• The Bias-Variance tradeoff
• Regularization techniques
• Preventing Over-fitting
• Validation methods
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organization Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

Content will be presented in the context of topics explored on Days 1 and 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Network Architectures
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Practical Applications