Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Overview and ANN Architecture.

• Comparison of biological and artificial neurons.
• Theoretical modeling of ANNs.
• Common activation functions in ANN design.
• Standard categories of network topologies.

Theoretical Basis and Learning Algorithms.

• Refresher on vector and matrix algebra.
• Introduction to state-space theory.
• Foundations of optimization techniques.
• Error-driven learning paradigms.
• Memory-centric learning approaches.
• Hebbian learning principles.
• Competitive learning mechanisms.

Single-Layer Perceptrons.

• Architecture and training of perceptrons.
• Pattern classification basics and Bayes' theorem.
• Utilizing perceptrons for pattern classification.
• Analysis of perceptron convergence.
• Constraints inherent to single-layer perceptrons.

Feedforward ANNs.

• Design of multi-layer feedforward structures.
• The backpropagation algorithm.
• Training processes and convergence in backpropagation.
• Applying backpropagation to functional approximation.
• Design considerations and practical challenges of backpropagation.

Radial Basis Function Networks.

• Pattern separation and interpolation techniques.
• Theory of regularization.
• Regularization strategies in RBF networks.
• Configuration and training of RBF systems.
• Approximation capabilities of RBFs.

Competitive Learning and Self-Organizing ANNs.

• General clustering methodologies.
• Learning Vector Quantization (LVQ).
• Competitive learning algorithms and network designs.
• Self-Organizing Feature Maps (SOFMs).
• Characteristics of feature maps.

Fuzzy Neural Networks.

• Integration of neuro-fuzzy systems.
• Foundations of fuzzy sets and logic.
• Architecture design for fuzzy systems.
• Construction of fuzzy ANNs.

Real-World Applications

• Discussion of various Neural Network use cases, highlighting their benefits and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent scenarios
  • Guarantees for finite hypothesis sets – inconsistent scenarios
  • General considerations
    • Contrasting deterministic and stochastic contexts
    • Bayes error noise
    • Distinction between estimation and approximation errors
    • Strategies for model selection
• Rademacher Complexity and VC-Dimension
• Bias-Variance tradeoff
• Regularization techniques
• Overfitting issues
• Validation procedures
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organization Maps (SOM)
• Kernel-induced vector spaces
  • Mercer Kernels and kernel-based similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

Content will be contextualized within the topics explored on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Network Architectures
• Linear Decoders
• Convolution and Pooling operations
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Practical Applications