Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure.

• Comparison of biological and artificial neurons.
• Architectural models of ANNs.
• Activation functions employed in ANNs.
• Common classes of network architectures.

Mathematical Foundations and Learning mechanisms.

• Review of vector and matrix algebra.
• Concepts of state-space representation.
• Core principles of optimization.
• Learning via error correction.
• Memory-based learning approaches.
• Hebbian learning principles.
• Competitive learning paradigms.

Single layer perceptrons.

• Perceptron structure and training process.
• Introduction to pattern classification and Bayes’ classifiers.
• Utilizing perceptrons as pattern classifiers.
• Convergence properties of perceptrons.
• Inherent limitations of perceptrons.

Feedforward ANN.

• Architecture of multi-layer feedforward networks.
• The back propagation algorithm.
• Training and convergence via back propagation.
• Functional approximation using back propagation.
• Practical considerations and design challenges in back propagation learning.

Radial Basis Function Networks.

• Pattern separability and interpolation techniques.
• Theory of regularization.
• Application of regularization in RBF networks.
• Design and training of RBF networks.
• Approximation capabilities of RBF networks.

Competitive Learning and Self organizing ANN.

• General approaches to clustering.
• Learning Vector Quantization (LVQ).
• Algorithms and architectures for competitive learning.
• Self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks.

• Integration of neuro-fuzzy systems.
• Foundations of fuzzy sets and logic.
• Design of fuzzy systems.
• Development of fuzzy ANNs.

Applications

• Discussion of specific Neural Network application cases, highlighting their benefits and potential challenges.

DAY -2 MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets in consistent scenarios.
  • Guarantees for finite hypothesis sets in inconsistent scenarios.
  • General considerations
    • Comparison of deterministic vs. stochastic scenarios.
    • Bayes error noise.
    • Distinction between estimation and approximation errors.
    • Strategies for model selection.
• Rademacher Complexity and VC-Dimension.
• The Bias-Variance tradeoff.
• Regularization techniques.
• Addressing over-fitting.
• Validation methods.
• Support Vector Machines.
• Kriging (Gaussian Process regression).
• PCA and Kernel PCA.
• Self Organisation Maps (SOM).
• Kernel induced vector space
  • Mercer Kernels and similarity metrics derived from kernels.
• Reinforcement Learning.

DAY 3 - DEEP LEARNING

This module builds upon the concepts introduced in Day 1 and Day 2.

• Logistic and Softmax Regression.
• Sparse Autoencoders.
• Vectorization, PCA, and Whitening.
• Self-Taught Learning.
• Deep Network architectures.
• Linear Decoders.
• Convolution and Pooling operations.
• Sparse Coding.
• Independent Component Analysis.
• Canonical Correlation Analysis.
• Demonstrations and real-world applications.