Sec. 1 — Core Machine Learning and AI Knowledge

Section 1 of Generative AI LLM Exam Study Guide

Abstract

This section establishes the foundational machine learning and artificial intelligence principles required to understand and deploy modern generative AI systems. It bridges the gap between classical deep learning architectures, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), and the operational realities of Foundation Models and Large Language Models (LLMs). Central to this progression is the technical framework for model interoperability via Open Neural Network Exchange (ONNX), optimization strategies for inference efficiency on hardware accelerators, and the operational methodologies encapsulated by GenAIOps and LLMOps.

Key Concepts

• ONNX Interoperability and Graph Optimization: Open Neural Network Exchange (ONNX) serves as an open standard defining operations and a file format to ensure framework compatibility and interoperability across different deep learning environments. The model is represented as a directed graph where nodes express operators and their parameters, while edges indicate data flow between inputs and outputs. ONNX Runtime employs several graph optimization levels, including Basic optimizations for semantics-preserving modifications like constant folding, and Extended optimizations applicable to CPU or CUDA execution providers.
• Inference Efficiency and Memory Management: Efficient inference requires loading the full model into GPU memory or utilizing serialized models that are streamed back to the GPU when necessary. To saturate the GPU, operators increase spatial dimensions or batch sizes, while pinned memory on the host is utilized to write data to system memory before copying to device-local memory. CUDA streams facilitate parallel data preprocessing and inferencing on the GPU, ensuring that input data remains available to the model without blocking computation.
• Generative Model Architectures: Generative AI models utilize neural networks to identify patterns within existing data to generate new content, often employing unsupervised or semi-supervised learning during training. Diffusion models operate via forward diffusion to add random noise and reverse diffusion to remove it, contrasting with Variational Autoencoders (VAEs) which use an encoder-decoder structure. Transformer networks dominate text-based applications through self-attention layers that assign weights to input parts and positional encodings that represent input word order.
• Foundation Models and Enterprise Deployment: Foundation models are large AI models trained on enormous quantities of unlabeled data through self-supervised learning, adaptable to a broad range of tasks via domain-specific labeled data. Deployment scenarios vary from high-end datacenter stacks supporting uncompressed models for multiple users, to modest hardware running quantized models optimized for single-user applications. Consumer hardware deployment remains limited by resource constraints, requiring significant optimization to run even one local large model.
• Operational Lifecycles (GenAIOps, LLMOps, RAGOps): Machine Learning Operations (MLOps) provides the overarching framework for end-to-end system development, which Generative AI Operations (GenAIOps) extends to manage interaction with foundation models. Large Language Model Operations (LLMOps) focuses specifically on productionizing LLM-based solutions, while Retrieval-Augmented Generation (RAGOps) addresses the delivery of RAG architectures as a reference model for generative AI adoption. This lifecycle includes aligning models with human preferences using curated datasets before customization for enterprise use cases.
• Training Dynamics and Optimization Algorithms: Deep learning training involves an iterative process where forward passes compute predictions and backward passes adjust weights via backpropagation using the chain rule. Optimization techniques like Momentum compute an exponentially weighted moving average of gradients to prevent getting stuck in local minima, while RMSprop normalizes gradients by dividing by the square root of the weighted running mean of squared gradients. Regularization methods, including L1 and L2 penalties, reduce overfitting by constraining weight magnitudes, whereas dropout removes random units during gradient steps to emulate an ensemble of smaller networks.

Key Equations and Algorithms

• Linear Regression Model: $y=\text{intercept}+\sum c_i{x}_i+\text{Error}$. This equation defines the relationship between dependent variables and independent features, where the prediction is a floating-point value generated by coefficients $c_i$ and an intercept constant offset.
• SVM Hyperplane: $wx+b=0$. This expression represents the optimal line or hyperplane in N-dimensional space that maximizes the distance between classes in a Support Vector Machine classification task.
• Epoch and Iteration Calculation: $\text{Iterations}=N/\text{batch size}$. This relationship determines the number of steps required per epoch, where $N$ is the total number of examples in the training set processed once.
• Softmax Normalization: $\sum P({\text{class}}_k)=1.0$. The softmax function determines probabilities for each possible class in a multi-class classification model, ensuring the resulting probabilities sum exactly to one.
• Learning Rate Update: $\Delta \theta =-\text{learning rate}\times \text{gradient}$. During gradient descent, the learning rate acts as a hyperparameter that determines the magnitude of the adjustment to model parameters (weights) based on the computed gradient direction of steepest ascent.
• L2 Regularization Penalty: $\text{Penalty}\propto \sum {\text{weights}}^2$. This algorithm penalizes weights in proportion to the sum of their squares, helping to drive outlier weights closer to zero without eliminating them entirely to reduce model variance.

Key Claims and Findings

• Precision Trade-offs in Tensor Cores: Transitioning from FP32 to lower precisions like FP16 or INT8 leads to faster processing speeds and reduced memory usage on Tensor Cores, provided weights are recalibrated or quantized to maintain accuracy.
• Inference Optimization Modes: All graph optimizations in ONNX Runtime can be performed online before inference starts or offline to save the final model to disk, with offline mode reducing startup time for consecutive execution.
• SVD vs. Autoregressive Models: Although Large Language Models are termed autoregressive, they differ from classical autoregressive models because they do not rely on linear dependencies on their own previous values and are not always stationary.
• Activation Function Selection: Non-saturating activation functions like ReLU are often preferred over saturating functions like the logistic sigmoid in deep layers because they are less likely to suffer from the vanishing gradient problem during backpropagation.
• Data Augmentation Efficacy: Artificially boosting the range and number of training examples by transforming existing data through rotation, stretching, and reflection yields enough labeled data to enable excellent model training.
• Memory in Recurrent Architectures: Long Short-Term Memory (LSTM) units utilize a self-connection with a constant weight of 1.0 to preserve gradients and values indefinitely, effectively eliminating the vanishing gradient problem found in standard RNNs.

Terminology

• ONNX (Open Neural Network Exchange): An open standard defining a set of operators and a file format based on Protocol Buffers to describe deep learning models facilitating framework compatibility.
• Tensor Cores: Specialized hardware units in NVIDIA GPUs that enable Warp Matrix Multiply-Accumulate (WMMA) operations, supporting FP16-based (HMMA) and integer-based (IMMA) multiply-accumulate functions.
• GenAIOps: An extension of MLOps specifically developed to manage and operationalize generative AI solutions with distinct characteristics regarding foundation model interaction.
• Regularization: Any mechanism that reduces overfitting, such as L1 or L2 penalties, by driving weights toward zero or by removing random units during training steps.
• Autoencoder: A neural network system combining an encoder and decoder that learns to extract important information by mapping input to a lower-dimensional code and reconstructing it.
• Pinned Memory: A type of host memory used to write data into system memory efficiently before issuing a GPU command to copy it from host-visible to device-local memory.
• Convergence: A state reached when loss values change very little or not at all with each iteration, indicating that additional training will not significantly improve the model.
• Foundation Models: Large AI models trained on enormous quantities of unlabeled data through self-supervised learning, generally adapted to accomplish a broad range of downstream tasks.
• SVM (Support Vector Machine): A supervised learning algorithm that classifies data by finding an optimal hyperplane maximizing the distance between each class, using kernel functions for nonlinear separation.
• RPM (Root Mean Square Propagation): An optimization algorithm that normalizes the gradient by dividing by the magnitude of recent gradients, frequently used in recurrent neural networks to protect against vanishing and exploding gradients.