Glossary of Terms & Concepts

The course's vocabulary on one page: first the core LLM terms you will hear in almost every model card, paper and interview, grouped by topic, then an A-Z list of every key term from each module's Summary & Key Terms page.

Core LLM Vocabulary at a Glance

These six groups cover the terms that come up in nearly every discussion of how a modern LLM is built, trained, tested and served. If you can explain each one in a sentence and say why it matters, you have the core vocabulary of the field.

mindmap
  root(("🧠 Core LLM vocabulary"))
    🔍 Attention techniques
      MHA, GQA, MQA
      MLA
      SWA
      AttnRes
    🏗️ Architecture optimizations
      MoE
      RoPE, ALiBi
      LN, QK-norm
      SwiGLU
    🎛️ Model tuning
      SFT
      PEFT, LoRA, QLoRA
      DPO
    🎯 Reinforcement learning
      RLHF, RLVR
      RM
      PPO, GRPO, GSPO
    🏆 Benchmarks
      MMLU, GPQA
      AIME, HLE
      SWE-bench
    📏 Metrics
      PPL, KL
      LaaJ, Elo
      pass@k
      TTFT, TPOT

🔍 Attention techniques

Term	Meaning	Where it's taught
MHA	Multi-Head Attention: several attention heads in parallel, each with its own Q, K and V projections.	Attention Mechanisms
MQA	Multi-Query Attention: all query heads share a single K/V head, shrinking the KV cache by the head count.	Attention Mechanisms
GQA	Grouped-Query Attention: groups of query heads share a K/V head - the middle ground most open models use.	Attention Mechanisms, KV Cache
MLA	Multi-head Latent Attention: caches a compressed low-rank latent per token instead of full keys and values (DeepSeek).	Modern Architectures
SWA	Sliding Window Attention: each token attends only to the nearest W tokens, bounding compute and KV cache.	Attention Mechanisms
AttnRes	Attention Residuals: replace the fixed residual sum with learned attention over earlier layers' outputs.	Modern Architectures

🏗️ Architecture optimizations

Term	Meaning	Where it's taught
MoE	Mixture of Experts: a router sends each token to a few expert FFNs, so compute follows active parameters and memory follows total parameters.	Model Architecture Types, Modern Architectures
RoPE	Rotary Position Embeddings: rotate Q and K by position so attention scores depend on relative distance.	Transformer Architecture
ALiBi	Attention with Linear Biases: add a distance penalty to attention scores instead of position embeddings.	Transformer Architecture
LN	Layer Normalization: normalize each token's activations; modern models use Pre-LN, usually as RMSNorm.	Transformer Architecture
QK-norm	Normalize queries and keys before the dot product to stop attention logits from blowing up.	Modern Architectures, Training Stability
SwiGLU	Swish-Gated Linear Unit: the gated FFN activation used by Llama, Qwen, Mistral and most modern LLMs.	Transformer Architecture

🎛️ Model tuning

Term	Meaning	Where it's taught
SFT	Supervised Fine-Tuning: train on examples of the desired responses.	Fine-Tuning & Alignment
PEFT	Parameter-Efficient Fine-Tuning: adapt a model by training a small fraction of its parameters.	Fine-Tuning & Alignment
LoRA	Low-Rank Adaptation: freeze the base weights and train a low-rank update BA beside them.	Fine-Tuning & Alignment, LoRA & QLoRA Hands-On
QLoRA	Quantized LoRA: LoRA on a 4-bit (NF4) quantized base model, so large models fit on one GPU.	Fine-Tuning & Alignment
DPO	Direct Preference Optimization: learn from chosen/rejected pairs with a classification-style loss, no reward model or RL loop.	Preference Optimization

🎯 Reinforcement learning

Term	Meaning	Where it's taught
RLHF	RL from Human Feedback: train a reward model on human preferences, then optimize the policy against it.	RL for LLMs
RLVR	RL with Verifiable Rewards: the reward is a program (exact-match answer, unit tests) rather than a learned model.	RL for LLMs
RM	Reward Model: predicts a scalar preference score, usually trained with the Bradley-Terry pairwise loss.	Preference Optimization
PPO	Proximal Policy Optimization: clipped policy updates with a learned value critic and a KL penalty.	RL for LLMs
GRPO	Group Relative Policy Optimization: sample a group of answers and use each one's reward relative to the group as the advantage - no critic.	RL for LLMs
GSPO	Group Sequence Policy Optimization: GRPO with a sequence-level importance ratio and clipping, for stabler RL on large and MoE models.	RL for LLMs

🏆 Benchmarks

Term	Meaning	Where it's taught
MMLU	Massive Multitask Language Understanding: 57-subject multiple-choice knowledge test; saturated at the frontier.	What Benchmarks Measure
GPQA	Graduate-level Google-Proof Q&A: expert-written science questions; the Diamond subset is the usual report.	What Benchmarks Measure
AIME	American Invitational Mathematics Examination: 30 integer-answer olympiad-qualifier problems a year.	What Benchmarks Measure
HLE	Humanity's Last Exam: hard expert questions across many fields, built to stay unsaturated.	What Benchmarks Measure
SWE-bench	Resolve real GitHub issues so hidden tests pass; Verified is the 500-task human-checked subset.	What Benchmarks Measure, Agent Evaluation

📏 Metrics

Term	Meaning	Where it's taught
PPL	Perplexity: exp(cross-entropy loss) - how many tokens the model is, on average, choosing between.	Probability & Information Theory
KL	Kullback-Leibler divergence: how far one distribution is from another; the penalty that keeps a tuned policy near its reference.	Probability & Information Theory
LaaJ	LLM-as-a-Judge: a model grades outputs against a rubric; validate it against human labels before trusting it.	LLM-as-Judge
pass@k	Probability that at least one of k sampled attempts is correct.	What Benchmarks Measure, Statistics for Evaluation
Elo	Pairwise rating scale from chess; on arena leaderboards a 100-point gap ≈ 64% expected win rate.	Contamination & Leaderboards
TTFT	Time To First Token: queueing plus prefill - what makes a chat feel responsive.	Streaming & TTFT
TPOT	Time Per Output Token: the decode-step gap between tokens; dominates long answers.	Modern Serving Stack

A-Z Glossary

309 terms from the Summary & Key Terms page of every module. Where a term appears in more than one module, the meaning shown is from the first module that defines it.

A

Term	Meaning	Module
A2A	Agent2Agent: protocol for communication and task exchange between agents.	12 Agent Protocols: MCP & A2A
Ablation / baseline	Remove a component to measure its value / reference system for comparison.	13 Agentic Workflows & Multi-Agent Systems
Acceptance criteria	Observable conditions that define whether a requirement is satisfied.	20 Agent Engineering
ACL / CDC	Access Control List / Change Data Capture: permissions / source change tracking.	14 RAG Systems
Activation checkpointing	Recomputes intermediate activations during backward to save memory.	06 PyTorch Fundamentals
Active recall	Retrieve an answer from memory before checking the notes.	23 Review & Readiness
Adapter merge	Combines learned adapter updates with base weights for deployment.	09 Post-Training & Fine-Tuning
ADKAR	Awareness, Desire, Knowledge, Ability, Reinforcement: a change-management model.	21 Solutions Architecture
Adoption / capacity savings	Actual sustained use / freed time that is not automatically cash savings.	21 Solutions Architecture
ADR	Architecture Decision Record: context, decision and consequences of one choice.	21 Solutions Architecture
ADR / ROI	Architecture Decision Record / Return On Investment: design rationale / business value measure.	22 Capstones
ADR / SLO / ROI	Architecture Decision Record / Service Level Objective / Return On Investment.	23 Review & Readiness
Agent / workflow	Model-directed action selection / predefined execution path.	10 Introduction to AI Agents
Agent Card	Describes an A2A agent's capabilities and connection details.	12 Agent Protocols: MCP & A2A
AGENTS.md / SKILL.md	Repository instructions / a reusable skill's instructions and trigger metadata.	20 Agent Engineering
AIME / HLE	American Invitational Mathematics Examination problems / Humanity's Last Exam (frontier expert questions).	04 Evaluation & Benchmarks
ALiBi	Attention with Linear Biases: a distance penalty added to attention scores instead of position embeddings.	03 Foundation Models
AMP / BF16	Automatic Mixed Precision / Brain Floating Point 16: mixed dtypes / a wide-range 16-bit format.	06 PyTorch Fundamentals
ANN / HNSW	Approximate Nearest Neighbor / Hierarchical Navigable Small World: fast approximate search / a graph index.	14 RAG Systems
API / SDK	Application Programming Interface / Software Development Kit: service contract / client tooling.	01 Python & Systems
ASR	Attack Success Rate: successful attacks divided by evaluated attacks.	04 Evaluation & Benchmarks
asyncio	Python's cooperative concurrency system for overlapping I/O waits.	01 Python & Systems
AttnRes	Attention Residuals: replace the fixed residual sum with learned attention over earlier layers' outputs.	03 Foundation Models
Autoregressive generation	Sampling one token, appending it to the input and running the model again, repeatedly.	08 Training LLMs from Scratch
AWS / GCP	Amazon Web Services / Google Cloud Platform: cloud infrastructure and managed services.	19 Cloud AI Platforms

B

Term	Meaning	Module
Backpressure / jitter	Slowing intake when overloaded / randomizing retry delays.	01 Python & Systems
Batch / epoch	Examples in one step / one pass through the training data.	06 PyTorch Fundamentals
BF16 / FP8	Brain Floating Point 16 / 8-bit Floating Point: lower-precision numerical formats.	07 Pretraining at Scale
Bigram model	A language model that predicts the next token from the current token only; here a single nn.Embedding(V, V) lookup table.	08 Training LLMs from Scratch
block_size (context length)	The fixed number of tokens in each training window the model sees.	08 Training LLMs from Scratch
BM25	Lexical ranking using term frequency, document frequency and length normalization.	14 RAG Systems
Bootstrap / paired test	Resampling to estimate uncertainty / comparing systems on the same items.	02 Math for ML
BPE	Byte Pair Encoding: builds tokens by repeatedly merging frequent adjacent pairs.	03 Foundation Models

C

Term	Meaning	Module
C4	Context, Containers, Components, Code: levels for architecture diagrams.	21 Solutions Architecture
Canary / shadow rollout	Limited live release / test new behavior alongside the live system.	18 Production Engineering
Capstone	An integrated project demonstrating skills across multiple course sections.	23 Review & Readiness
Chaining / routing	Sequential dependent steps / selecting a path for a request.	13 Agentic Workflows & Multi-Agent Systems
Chat template	Formats conversation roles and separators into the model's expected token sequence.	09 Post-Training & Fine-Tuning
Checkpoint	Saved weights and run state, including optimizer, scheduler and randomness for resumption.	06 PyTorch Fundamentals
Checkpoint / session	Saved execution state / conversation-associated state.	15 Agent Frameworks
Chunk / overlap	Retrieval-sized text unit / content shared between neighboring units.	14 RAG Systems
CI / paired bootstrap	Confidence Interval / resampling matched items to estimate comparison uncertainty.	04 Evaluation & Benchmarks
CI / pass^k	Confidence Interval / success across all k repeated trials.	22 Capstones
CI / PR	Continuous Integration / Pull Request: automated checks / a reviewable change proposal.	01 Python & Systems
CI / SE	Confidence Interval / Standard Error: uncertainty range / estimated sampling variability.	02 Math for ML
Circuit breaker	Temporarily blocks calls to a repeatedly failing dependency.	16 Production Agents
Citation / abstention	Traceable source reference / declining an unsupported answer.	14 RAG Systems
Claude Agent SDK	Programmable access to an agent harness with tools and permission controls.	15 Agent Frameworks
CLM / MLM	Causal / Masked Language Modeling: predict the next token / selected hidden tokens.	07 Pretraining at Scale
Cohen's kappa	Judge agreement corrected for agreement expected by chance.	02 Math for ML
Compaction / offloading	Compress context / move details to external storage.	20 Agent Engineering
Concurrency / batch size	Requests in flight / requests processed together in a model step.	17 Inference & Serving
Confused deputy	A privileged service is tricked into acting for an unauthorized party.	12 Agent Protocols: MCP & A2A
Constrained decoding	Restricts allowed next tokens to enforce an output grammar or schema.	05 Prompt & Context Engineering
Contamination / leakage	Evaluation content in training / test information influencing development.	04 Evaluation & Benchmarks
Context block (x, y)	A window of tokens and its targets, the same window shifted one token left.	08 Training LLMs from Scratch
Context engineering	Selects and organizes instructions, evidence, history and tool results.	05 Prompt & Context Engineering
Context isolation	Keeps unrelated worker details out of other agents' contexts.	13 Agentic Workflows & Multi-Agent Systems
Continuous batching	Adds and removes requests as generation proceeds.	17 Inference & Serving
Core-domain threshold	The course target of no core readiness score below 2.	23 Review & Readiness
Cost per successful task	Total evaluated cost divided by successful task completions.	16 Production Agents
CoT / ReAct	Chain of Thought / Reasoning and Acting: intermediate reasoning / reasoning interleaved with tools.	05 Prompt & Context Engineering
CoT / test-time compute	Chain of Thought / inference work spent reasoning, searching or checking.	09 Post-Training & Fine-Tuning
CP / EP	Context / Expert Parallelism: split sequences / distribute MoE experts.	07 Pretraining at Scale
CrewAI crew / flow	Role/task-based team / explicit event-driven control flow.	15 Agent Frameworks
CUDA / kernel fusion	NVIDIA's GPU computing platform / combining operations to cut launches and memory traffic.	07 Pretraining at Scale

D

Term	Meaning	Module
Dataset / DataLoader	Sample-access interface / batching, shuffling and loading machinery.	06 PyTorch Fundamentals
DBU	Databricks Unit: a platform usage unit used in pricing.	19 Cloud AI Platforms
DDP / FSDP	Distributed Data Parallel / Fully Sharded Data Parallel: replicate / shard model state across workers.	06 PyTorch Fundamentals
Deduplication / decontamination	Remove repeated data / evaluation overlap from training data.	07 Pretraining at Scale
Deferred tool approval	Pauses a tool request until an explicit decision is supplied.	15 Agent Frameworks
Delta Lake	Table format and tooling for reliable, transactional data-lake tables.	19 Cloud AI Platforms
Dependency injection	Supplies tools with external services or configuration through a typed context.	15 Agent Frameworks
Disaggregated serving	Runs prefill and decode on separate resources.	17 Inference & Serving
Discovery / baseline	Investigating the real problem / measured starting performance.	21 Solutions Architecture
Distillation	Trains a smaller or cheaper model using a stronger model's outputs.	09 Post-Training & Fine-Tuning
Docker / Kubernetes / Helm	Container packaging / workload orchestration / templated Kubernetes releases.	18 Production Engineering
Document-level split	Assigns whole documents to train or validation so pieces of one document cannot leak across.	08 Training LLMs from Scratch
DOM / accessibility tree	Document Object Model / semantic UI representation used for automation.	20 Agent Engineering
Dot product / cosine similarity	Vector alignment / alignment after removing magnitude differences.	02 Math for ML
DP / TP / PP	Data / Tensor / Pipeline Parallelism: split batches / layer computations / model stages.	07 Pretraining at Scale
DPO	Direct Preference Optimization: learns from preferred/rejected pairs without an explicit RL rollout loop.	09 Post-Training & Fine-Tuning
DSPy	Framework for declarative model programs and metric-driven optimization.	05 Prompt & Context Engineering
Durability / idempotency	Resume after interruption / retries preserve one intended effect.	22 Capstones
Durable execution	Persists progress so a run can resume after failures or long waits.	16 Production Agents

E

Term	Meaning	Module
Elicitation	A server requests additional user input through the client.	12 Agent Protocols: MCP & A2A
Elo / Bradley-Terry	Rating scale where a 100-point gap ≈ 64% win rate / the pairwise model arenas fit to all votes.	04 Evaluation & Benchmarks
Embedding / vector search	Numeric representation of meaning / search by vector similarity.	14 RAG Systems
End-of-text token	Special token inserted between documents to mark boundaries in the token stream.	08 Training LLMs from Scratch
End-to-end project	A complete system with code, metrics, architecture and an explained outcome.	23 Review & Readiness
Environment / reset	Task world, tools and feedback / return that world to a known starting state.	20 Agent Engineering
Episodic / semantic / procedural memory	Past experiences / stored facts / reusable procedures.	11 Agent Building Blocks
Error budget / burn rate	Allowed failures / speed at which that allowance is consumed.	18 Production Engineering
Eval	A repeatable procedure for measuring system behavior or task success.	23 Review & Readiness
Eval / benchmark	A measurement procedure / a standardized task set and scoring protocol.	04 Evaluation & Benchmarks
Evaluator-optimizer	An evaluator supplies feedback to an iterative improvement step.	13 Agentic Workflows & Multi-Agent Systems
Evidence / baseline	Demonstrable work or measurements / reference used for comparison.	23 Review & Readiness
Exit criteria	Pre-agreed evidence for proceeding, iterating or stopping a pilot.	21 Solutions Architecture

F

Term	Meaning	Module
Fan-out / fan-in	Dispatching independent work / collecting and combining its results.	13 Agentic Workflows & Multi-Agent Systems
FlashAttention	Exact attention computed with less intermediate memory and data movement.	03 Foundation Models
FLOPs	Floating-Point Operations: a count of numerical work.	02 Math for ML
Freshness / lineage	How current indexed data is / its source and transformation history.	14 RAG Systems

G

Term	Meaning	Module
GKE / TPU	Google Kubernetes Engine / Tensor Processing Unit: managed Kubernetes / Google's ML accelerator.	19 Cloud AI Platforms
Google ADK	Agent toolkit with composition, runners and session management.	15 Agent Frameworks
GPT / CNN	Generative Pre-trained Transformer / Convolutional Neural Network: text decoder / convolution-based model.	06 PyTorch Fundamentals
GPTQ / AWQ	Post-training weight quantization / Activation-aware Weight Quantization methods.	17 Inference & Serving
GPU / CUDA	Graphics Processing Unit / Compute Unified Device Architecture: accelerator / NVIDIA's GPU computing platform.	01 Python & Systems
Gradient / backpropagation	Loss derivatives / efficient chain-rule computation through a model.	02 Math for ML
Gradient accumulation	Adds gradients across micro-batches before one optimizer update.	06 PyTorch Fundamentals, 08 Training LLMs from Scratch
Gradient clipping / warmup	Caps gradient size / gradually raises the initial learning rate.	07 Pretraining at Scale
GraphRAG / agentic RAG	Graph-based evidence retrieval / an agent-controlled retrieval loop.	14 RAG Systems
Grounding / faithfulness	Tying claims to evidence / consistency of the answer with that evidence.	14 RAG Systems
GRPO	Group Relative Policy Optimization: uses group-relative rewards without a separate value critic.	09 Post-Training & Fine-Tuning
GSPO	Group Sequence Policy Optimization: GRPO with a sequence-level importance ratio and clipping, for more stable RL.	09 Post-Training & Fine-Tuning
Guardrail / agent runtime	Content/action checks / infrastructure that executes agent runs.	19 Cloud AI Platforms
Guardrail / stop condition	Checks around behavior / rule that terminates execution.	10 Introduction to AI Agents

H

Term	Meaning	Module
Hallucination / sycophancy	Unsupported output / agreeing with the user over correctness.	03 Foundation Models
Handoff / agent-as-tool	Transfers control / calls an agent and returns its result.	15 Agent Frameworks
Harness	Code that runs tasks, prompts models and computes scores.	04 Evaluation & Benchmarks, 20 Agent Engineering
HBM / SRAM	High Bandwidth Memory / Static RAM: accelerator main memory / fast on-chip memory.	07 Pretraining at Scale
Held-out set	Data reserved for validation or testing rather than fitting the model.	09 Post-Training & Fine-Tuning
HF Hub	Hugging Face Hub: versioned storage and sharing for models and datasets.	09 Post-Training & Fine-Tuning
HITL	Human In The Loop: a person reviews, corrects or approves selected steps.	10 Introduction to AI Agents, 16 Production Agents
Host / client / server	Application / its protocol connector / capability provider.	12 Agent Protocols: MCP & A2A
HPA / GPU	Horizontal Pod Autoscaler / Graphics Processing Unit: replica scaling / accelerator resource.	18 Production Engineering
Hybrid retrieval / RRF	Combines lexical and semantic search / Reciprocal Rank Fusion combines ranked lists.	14 RAG Systems

I

Term	Meaning	Module
I/O / GIL	Input/Output / Global Interpreter Lock: external waiting / a constraint on Python thread execution.	01 Python & Systems
IaC / GitOps	Infrastructure as Code / reconciling deployed configuration from Git.	18 Production Engineering
IAM / RBAC	Identity and Access Management / Role-Based Access Control: identity permissions / permissions by role.	18 Production Engineering
ICL	In-Context Learning: adapting behavior from examples in the prompt without weight updates.	05 Prompt & Context Engineering
Idempotency	Repeated requests have the same intended effect as one request.	01 Python & Systems, 13 Agentic Workflows & Multi-Agent Systems
Idempotency key	Identifies an operation so retries do not duplicate its intended effect.	16 Production Agents
Instruction hierarchy	Rules for resolving conflicting instructions by source priority.	05 Prompt & Context Engineering

J

Term	Meaning	Module
JSON / SSE	JavaScript Object Notation / Server-Sent Events: structured data / one-way HTTP streaming.	01 Python & Systems
JSON Schema	Specifies the expected structure and types of JSON data.	05 Prompt & Context Engineering
JSON-RPC	JavaScript Object Notation Remote Procedure Call: structured request/response message format.	12 Agent Protocols: MCP & A2A

K

Term	Meaning	Module
KL divergence	Kullback-Leibler divergence: an asymmetric measure of distribution mismatch.	02 Math for ML
KL penalty	Kullback-Leibler divergence penalty: discourages a policy from drifting away from a reference distribution.	09 Post-Training & Fine-Tuning
KV cache	Key-Value cache: saves attention state so earlier tokens need not be recomputed.	17 Inference & Serving

L

Term	Meaning	Module
Lakehouse	Data architecture combining lake storage with warehouse-style management.	19 Cloud AI Platforms
LangChain / LangGraph	Model/tool integration layer / stateful graph orchestration.	15 Agent Frameworks
Lethal trifecta	Private data, untrusted input and outbound communication in one agent context.	16 Production Agents
LFS / DVC	Large File Storage / Data Version Control: version large artifacts / datasets.	01 Python & Systems
LLM	Large Language Model: a model that learns patterns in token sequences.	03 Foundation Models
LLM-as-judge (LaaJ)	A model grades outputs using an explicit rubric.	04 Evaluation & Benchmarks
LN / Pre-LN	Layer Normalization / normalizing before each sublayer (modern) rather than after the residual add (original Post-LN).	03 Foundation Models
Lockfile	Records resolved dependencies to reproduce an environment.	01 Python & Systems
Logits	Unnormalized next-token scores; softmax turns a row of them into probabilities.	08 Training LLMs from Scratch
Loop / graph	Repeat actions until a condition holds / explicit steps and transitions.	20 Agent Engineering
LoRA / QLoRA	Low-Rank Adaptation / Quantized LoRA: train low-rank updates / do so on a quantized base.	09 Post-Training & Fine-Tuning
Loss masking	Excludes selected tokens, such as prompts or padding, from the training objective.	09 Post-Training & Fine-Tuning
Lost in the middle	Reduced use of relevant information buried inside a long context.	03 Foundation Models

M

Term	Meaning	Module
Managed service	Provider operates a capability; the customer still owns configuration and application behavior.	19 Cloud AI Platforms
MAST	Multi-Agent System Failure Taxonomy: categories for diagnosing coordination failures.	13 Agentic Workflows & Multi-Agent Systems
MCP	Model Context Protocol: standard interface for tools and context exposed to model applications.	12 Agent Protocols: MCP & A2A
MCP / HITL	Model Context Protocol / Human In The Loop: tool integration / human review boundary.	22 Capstones
Medallion architecture	Bronze raw, silver cleaned and gold consumption-ready data layers.	19 Cloud AI Platforms
Memory poisoning	Untrusted content written to memory that steers later runs.	11 Agent Building Blocks
MFU	Model FLOPs Utilization: useful model compute relative to accelerator peak compute.	07 Pretraining at Scale, 22 Capstones
MHA / MQA / GQA	Multi-Head / Multi-Query / Grouped-Query Attention: independent / shared / grouped K/V heads.	03 Foundation Models
Microsoft Agent Framework	Agent and workflow abstractions for orchestration.	15 Agent Frameworks
Middleware / hook / callback	Extension points around model or agent execution.	15 Agent Frameworks
MinHash / LSH	Similarity sketch / Locality-Sensitive Hashing: find likely near-duplicate documents.	07 Pretraining at Scale
MLA / SSM	Multi-head Latent Attention / State-Space Model: compressed attention state / recurrent sequence state.	03 Foundation Models
MLflow / MLOps	Experiment and model lifecycle tooling / Machine Learning Operations.	19 Cloud AI Platforms
MMLU / GPQA	Massive Multitask Language Understanding (57-subject knowledge) / Graduate-level Google-Proof Q&A (expert science).	04 Evaluation & Benchmarks
Model card	Documents intended use, training, evaluation and limitations of a model.	22 Capstones
Model catalog / endpoint	Available model inventory / deployed model access interface.	19 Cloud AI Platforms
MoE	Mixture of Experts: routes tokens to a subset of expert networks.	03 Foundation Models
MQA / GQA	Multi-Query / Grouped-Query Attention: share K/V heads to shrink the KV cache.	17 Inference & Serving

N

Term	Meaning	Module
nDCG	Normalized Discounted Cumulative Gain: ranking quality with relevance and position weighting.	14 RAG Systems
NF4	NormalFloat 4: a 4-bit format used to quantize base weights in QLoRA.	09 Post-Training & Fine-Tuning
NFR	Non-Functional Requirement: quality, latency, availability, security or other operating constraint.	21 Solutions Architecture
NLL / cross-entropy	Negative Log-Likelihood / prediction loss; equivalent for observed target labels.	02 Math for ML
Node / edge / state	Execution step / transition / data carried through a graph.	15 Agent Frameworks

O

Term	Meaning	Module
OAuth / PKCE	Authorization framework / Proof Key for Code Exchange: protects authorization-code flows.	12 Agent Protocols: MCP & A2A
Observation / trajectory	Result returned from an action / sequence of steps in a run.	10 Introduction to AI Agents
OOM / VRAM	Out Of Memory / Video Random Access Memory: allocation failure / GPU memory.	06 PyTorch Fundamentals
Open-loop load test	Schedules arrivals independently of response completion to expose queueing.	22 Capstones
OpenAI Agents SDK	Agent framework with tools, handoffs, guardrails and tracing.	15 Agent Frameworks
Orchestrator-workers	A coordinator delegates scoped tasks and integrates worker results.	13 Agentic Workflows & Multi-Agent Systems
OTel	OpenTelemetry: standard instrumentation for traces, metrics and logs.	18 Production Engineering
OTel / trace / span	OpenTelemetry / end-to-end run record / timed operation within that record.	16 Production Agents
Outcome / trajectory evaluation	Checks final task success / checks the actions used to reach it.	10 Introduction to AI Agents
Over-refusal	Rejecting benign requests that should be answered.	04 Evaluation & Benchmarks
Overfit-one-batch check	Training on a single batch until loss is near zero, to prove the model and loop can learn.	08 Training LLMs from Scratch
Overfitting / quality delta	Learning training-specific patterns / measured change from the baseline.	09 Post-Training & Fine-Tuning

P

Term	Meaning	Module
p95 / p99	Latency thresholds below which 95% / 99% of observations fall.	17 Inference & Serving
PA / audit trail	Prior Authorization / record of actions, decisions and responsible identities.	16 Production Agents
PagedAttention	Stores KV state in blocks to reduce wasted memory and support sharing.	17 Inference & Serving
pass@k	Probability of at least one correct solution in k attempts.	04 Evaluation & Benchmarks
pass@k / pass^k	At least one success in k attempts / success on every one of k trials.	02 Math for ML, 10 Introduction to AI Agents
pass^k	Repeated-run reliability: success on all k trials.	16 Production Agents
PCIe / NVLink	Peripheral Component Interconnect Express / NVIDIA GPU interconnect: device communication links.	07 Pretraining at Scale
PEFT	Parameter-Efficient Fine-Tuning: adapting a model with few trainable parameters.	09 Post-Training & Fine-Tuning
Perplexity	Exponential of average token NLL in nats; depends on the tokenizer.	02 Math for ML, 08 Training LLMs from Scratch
PII / PHI	Personally Identifiable Information / Protected Health Information.	16 Production Agents, 18 Production Engineering
Plan-and-execute	Creates a plan, executes steps and revises from feedback.	11 Agent Building Blocks
PoC / pilot / production	Proof of Concept / real-user trial / dependable operated system.	21 Solutions Architecture
PPL	Perplexity: exponentiated average negative log-likelihood; unusually low PPL on test items hints at contamination.	04 Evaluation & Benchmarks
PPO	Proximal Policy Optimization: constrains policy updates during reward optimization.	09 Post-Training & Fine-Tuning
Precision / recall / F1	Correctness among positives / coverage of positives / their harmonic mean.	04 Evaluation & Benchmarks
Prefill / decode	Prompt processing / subsequent token generation.	17 Inference & Serving
Prefix caching	Reuses cached computation for an identical leading token sequence.	17 Inference & Serving
Progressive disclosure	Loads a resource's details only when they are needed.	20 Agent Engineering
Prompt caching	Reuses computation for a matching prompt prefix.	05 Prompt & Context Engineering
Prompt chaining	Passes one model step's output into the next step.	05 Prompt & Context Engineering
Prompt injection	Untrusted content attempts to redirect a model's instructions or actions.	04 Evaluation & Benchmarks, 05 Prompt & Context Engineering
Prompt injection / memory poisoning	Redirecting instructions through input / planting harmful persistent context.	16 Production Agents
Prompt optimization / overfitting	Search for better prompt behavior / tailoring it too closely to development cases.	05 Prompt & Context Engineering
Protocol version	Agreed message rules; verify compatibility rather than assuming older features remain supported.	12 Agent Protocols: MCP & A2A
PydanticAI	Typed agent framework with validated outputs and dependency injection.	15 Agent Frameworks

Q

Term	Meaning	Module
Q/K/V	Query, Key, Value: attention matches queries to keys and combines values.	03 Foundation Models
Q&A	Questions and Answers: prompts and explanations for review practice.	23 Review & Readiness
QK-norm	Normalizes queries and keys before the dot product to keep attention logits bounded in large runs.	03 Foundation Models
Quantization	Stores or computes values at lower precision to reduce resource use.	17 Inference & Serving

R

Term	Meaning	Module
RAG	Retrieval-Augmented Generation: answer using retrieved source material.	19 Cloud AI Platforms
RAG / CAG	Retrieval-Augmented / Cache-Augmented Generation: retrieved evidence / cached knowledge context.	14 RAG Systems
Rank	Number of independent directions represented by a matrix.	02 Math for ML
Rank / alpha	Adapter update size / its scaling parameter.	09 Post-Training & Fine-Tuning
ReAct	Reasoning and Acting: alternates reasoning, actions and observations.	11 Agent Building Blocks
Readiness score	0: cannot explain; 1: follow a tutorial; 2: build independently; 3: design and debug.	23 Review & Readiness
Recall@k / MRR	Relevant-item coverage in top k / Mean Reciprocal Rank of the first relevant result.	14 RAG Systems
Red-teaming / jailbreak	Adversarial testing / an attempt to bypass model safeguards.	04 Evaluation & Benchmarks
Reflection	The agent critiques its own output; helps mainly when it gets new information such as test results.	11 Agent Building Blocks
Regression	Loss of a previously working behavior or capability.	23 Review & Readiness
Regression / capability / safety suite	Protect known behavior / measure new ability / test unacceptable behavior.	16 Production Agents
Regression suite	Repeatable tests for behavior that should remain correct.	04 Evaluation & Benchmarks
Replay / checkpoint	Rebuild state from recorded events / restore a saved execution state.	16 Production Agents
Reproducibility	Enough versioned code, data, configuration and environment detail to repeat the work.	22 Capstones
Reranker	Rescores retrieved candidates for query relevance.	14 RAG Systems
Reward hacking	Exploits a scoring mechanism without achieving the intended goal.	20 Agent Engineering
Reward hacking / forgetting	Exploiting the reward without solving the task / losing earlier capabilities.	09 Post-Training & Fine-Tuning
Reward model (RM)	Predicts preference or quality to supply a training signal.	09 Post-Training & Fine-Tuning
Risk register	Tracked risks with impact, mitigations and owners.	21 Solutions Architecture
RL / RLHF	Reinforcement Learning / RL from Human Feedback: learn from rewards / human-derived preferences.	09 Post-Training & Fine-Tuning
RLVR	Reinforcement Learning with Verifiable Rewards: trains against checkable outcomes.	09 Post-Training & Fine-Tuning, 20 Agent Engineering
RMSNorm / FFN	Root Mean Square Normalization / Feed-Forward Network: normalize activations / transform each token.	03 Foundation Models
ROI / TCO	Return On Investment / Total Cost of Ownership: value relative to investment / all ownership costs.	21 Solutions Architecture
Roofline model	Estimates performance limits from compute capacity and memory bandwidth.	07 Pretraining at Scale
RoPE	Rotary Position Embeddings: rotations that encode token position in attention.	03 Foundation Models
Rubric / calibration	Scoring criteria / checking scores against trusted labels.	04 Evaluation & Benchmarks
Runbook / postmortem	Incident procedure / review of causes, impact and corrective actions.	18 Production Engineering

S

Term	Meaning	Module
Sandbox / least privilege	Restricted execution environment / minimum necessary permissions.	16 Production Agents
SBOM	Software Bill of Materials: inventory of software components and dependencies.	18 Production Engineering
Scaling law	Empirical relationship between loss, model size, data and compute.	07 Pretraining at Scale
Scope / token audience	Granted permissions / the service a token is intended for.	12 Agent Protocols: MCP & A2A
SDK / ADK	Software Development Kit / Agent Development Kit: tooling for building applications / agents.	15 Agent Frameworks
SDPA	Scaled Dot-Product Attention: the attention operation, with optimized PyTorch backends.	06 PyTorch Fundamentals
Sensitivity analysis	Tests how a conclusion changes when assumptions change.	21 Solutions Architecture
SFT	Supervised Fine-Tuning: training on examples of desired responses.	09 Post-Training & Fine-Tuning
SFT / DPO / GRPO	Supervised Fine-Tuning / Direct Preference Optimization / Group Relative Policy Optimization.	22 Capstones
SGD / AdamW	Stochastic Gradient Descent / adaptive optimizer with decoupled weight decay.	02 Math for ML
Shadow / assist / automate	Observe without acting / support a human / execute workflow actions.	21 Solutions Architecture
Shared state / ledger	Common mutable information / recorded actions and ownership.	13 Agentic Workflows & Multi-Agent Systems
SIGTERM / SIGKILL	Graceful termination request / immediate forced process termination.	01 Python & Systems
SLI / SLO / SLA	Service Level Indicator / Objective / Agreement: measurement / target / commitment.	18 Production Engineering
SLO / goodput	Service Level Objective / work completed within defined service targets.	22 Capstones
SLO / SSE	Service Level Objective / Server-Sent Events: service target / HTTP streaming format.	17 Inference & Serving
Softmax	Converts logits into probabilities that sum to one.	02 Math for ML
Spark / PySpark	Distributed data processing engine / its Python interface.	19 Cloud AI Platforms
Spec-driven development	Builds from explicit requirements, plans, tasks and verification.	20 Agent Engineering
Speculative decoding	Proposes tokens cheaply, then verifies them with the target model.	17 Inference & Serving
SSH	Secure Shell: encrypted remote terminal access.	01 Python & Systems
stdio / HTTP	Standard input/output / Hypertext Transfer Protocol: local process / network transport.	12 Agent Protocols: MCP & A2A
Stop hook	A check triggered before the harness permits a run to finish.	20 Agent Engineering
Strict schema	Constrained decoding that guarantees tool arguments match the JSON Schema.	11 Agent Building Blocks
Sub-agent / handoff	Delegated worker under a parent / transfer of control to another agent.	13 Agentic Workflows & Multi-Agent Systems
Supervisor / peer network	Central coordinator / agents coordinating directly with one another.	13 Agentic Workflows & Multi-Agent Systems
SVD	Singular Value Decomposition: factors a matrix into rotations and singular-value scaling.	02 Math for ML
SWA	Sliding Window Attention: each token attends only to the nearest W tokens.	03 Foundation Models
SWE-bench	Resolve real GitHub issues so the repository's tests pass; the standard coding-agent benchmark.	04 Evaluation & Benchmarks
SwiGLU	Swish-Gated Linear Unit: the gated FFN activation used by most modern LLMs.	03 Foundation Models

T

Term	Meaning	Module
Task / artifact	Tracked unit of agent work / output produced by that work.	12 Agent Protocols: MCP & A2A
Task contract	Goal, inputs, constraints and expected output for delegated work.	13 Agentic Workflows & Multi-Agent Systems
Temperature / top-k sampling	Scaling logits to sharpen or flatten them / sampling only among the k most likely tokens.	08 Training LLMs from Scratch
Temperature / top-p	Controls sampling sharpness / limits sampling to a probability mass.	03 Foundation Models
Tensor / autograd	Multidimensional array / automatic differentiation through recorded operations.	06 PyTorch Fundamentals
Terraform state / drift	Record of managed resources / mismatch between declared and actual resources.	18 Production Engineering
Threat model	Maps attackers, assets, entry points and controls.	22 Capstones
Throughput / goodput	Work completed per time / work completed within the specified targets.	17 Inference & Serving
Token / context window	A model input unit / the token budget available to a request.	03 Foundation Models
Tokens per step	Batch × block_size × gradient-accumulation steps × GPUs; the effective batch size for the optimizer.	08 Training LLMs from Scratch
Tool / function calling	A model requests a structured operation; application code executes it.	10 Introduction to AI Agents, 11 Agent Building Blocks
Tool / resource / prompt	Callable operation / readable context / reusable message template.	12 Agent Protocols: MCP & A2A
Tool poisoning / SSRF	Malicious tool metadata / Server-Side Request Forgery targeting unintended services.	12 Agent Protocols: MCP & A2A
Tool search	Loading tool definitions on demand instead of sending every tool on every call.	11 Agent Building Blocks
Tool-call ID	Connects a tool result to the model request that produced it.	10 Introduction to AI Agents, 11 Agent Building Blocks
Topology	Arrangement of agents and the communication paths between them.	13 Agentic Workflows & Multi-Agent Systems
torch.compile	Compiles model execution to reduce overhead and optimize operations.	06 PyTorch Fundamentals
ToT / self-consistency	Tree of Thoughts / choosing among multiple sampled reasoning paths.	05 Prompt & Context Engineering
Trade-off	A choice that improves one property while costing another.	23 Review & Readiness
train() / eval()	Selects training / evaluation layer behavior; eval does not disable gradients.	06 PyTorch Fundamentals
Train/validation split	Held-out data for measuring generalization; done before chunking so no window crosses the boundary.	08 Training LLMs from Scratch
Transformers / datasets	Libraries for model/tokenizer loading / dataset processing.	09 Post-Training & Fine-Tuning
TRL	Transformer Reinforcement Learning: Hugging Face's SFT, DPO and GRPO trainers.	09 Post-Training & Fine-Tuning
TTFT / TPOT	Time To First Token / Time Per Output Token: serving responsiveness metrics.	22 Capstones
TTFT / TPOT / ITL	Time To First Token / Time Per Output Token / Inter-Token Latency.	17 Inference & Serving

U

Term	Meaning	Module
Unit economics / payback	Cost and benefit per task / time to recover the initial investment.	21 Solutions Architecture
Unity Catalog	Databricks governance for data and AI assets.	19 Cloud AI Platforms

V

Term	Meaning	Module
Vendor lock-in	Switching cost caused by provider-specific interfaces, data or infrastructure.	19 Cloud AI Platforms
Verifier / rubric	Checks task completion / explicit criteria for grading it.	20 Agent Engineering
VPC / private endpoint	Virtual Private Cloud / private network access to a service.	18 Production Engineering

W

Term	Meaning	Module
Weight tying	Sharing one matrix between the token embedding and the output (LM) head.	08 Training LLMs from Scratch
Working memory / task state	Current context / persisted progress needed to continue a task.	11 Agent Building Blocks
Workload identity	Short-lived service identity instead of embedded long-lived keys.	18 Production Engineering
Worktree	Separate working directory for a Git checkout, useful for isolated changes.	20 Agent Engineering

Z

Term	Meaning	Module
ZeRO / FSDP	Zero Redundancy Optimizer / Fully Sharded Data Parallel: shard training state.	07 Pretraining at Scale
Zero-shot / few-shot	Instructions without examples / instructions with example demonstrations.	05 Prompt & Context Engineering