Generative Artificial Intelligence (Gen AI) is transforming how we create content, solve problems, and interact with technology. From writing emails and generating images to composing music and writing code, Gen AI models are demonstrating remarkable capabilities. However, understanding the technology requires familiarity with its specific vocabulary.
This glossary provides clear and concise definitions for common technical terms used in the world of Generative AI. Whether you’re a student, developer, business professional, or simply curious, this guide aims to demystify the language surrounding this exciting field.
Press enter or click to view image in full size
Core Concepts
Artificial Intelligence (AI): The broad field of computer science focused on creating systems that can perform tasks typically requiring human intelligence, such as learning, problem-solving, perception, and decision-making. Gen AI is a subfield of AI.
Machine Learning (ML): A subset of AI where systems learn patterns and make predictions from data without being explicitly programmed for every task. Most Gen AI models are built using ML techniques.
Deep Learning (DL): A subset of ML that uses artificial neural networks with many layers (deep networks) to learn complex patterns from large amounts of data. Deep learning powers most state-of-the-art Gen AI models.
Generative AI (Gen AI): A type of AI focused on creating new, original content (text, images, audio, code, etc.) based on the patterns and structures learned from its training data. Unlike discriminative AI (which classifies or predicts labels), generative AI generates novel outputs.
Model Architectures & Types
Neural Network (or Artificial Neural Network — ANN): A computational model inspired by the structure and function of the human brain’s biological neural networks. Composed of interconnected nodes (neurons) organized in layers, they process information and learn from data.
Transformer: A highly influential deep learning architecture, particularly successful in Natural Language Processing (NLP) and now used widely across Gen AI domains (text, images, etc.). Its key innovation is the Attention Mechanism.
Generative Adversarial Network (GAN): An architecture consisting of two neural networks — a Generator and a Discriminator — that compete against each other. The Generator creates fake data (e.g., images), and the Discriminator tries to distinguish between real and fake data. This competition improves the Generator’s ability to create realistic outputs. Often used for image generation.
Variational Autoencoder (VAE): A type of generative model that learns a compressed representation (latent space) of the input data and can then sample from this space to generate new data similar to the original data.
Diffusion Models: A class of generative models that work by gradually adding noise to training data and then learning to reverse the process (denoising). By starting with random noise and applying the learned denoising process, they can generate high-quality data, especially images.
Large Language Model (LLM): A type of Transformer model trained on vast amounts of text data, designed to understand and generate human-like text. Examples include GPT-4, LaMDA, Llama 2. Often serve as the “brain” or reasoning engine for AI Agents.
Foundation Model: A large AI model (often an LLM or similar) trained on broad data that can be adapted (e.g., through Fine-tuning) to a wide range of downstream tasks. They serve as a base or “foundation” for more specialized models and AI agents.
Multimodal AI: AI models capable of processing and understanding information from multiple types of data (modalities) simultaneously, such as text, images, and audio. They can generate outputs that integrate these different modalities.
Key Mechanisms & Components
Neuron (or Node): The basic computational unit in a neural network. It receives input signals, performs a calculation (often involving weights and a bias), and produces an output signal passed to other neurons.
Layer: A collection of neurons in a neural network that process information at a similar level. Networks typically have an input layer, one or more hidden layers, and an output layer.
Activation Function: A function applied to the output of a neuron that determines whether it should be “activated” (fire) and what its output signal should be. Introduces non-linearity, allowing networks to learn complex patterns.
Weights: Parameters within a neural network that determine the strength of the connection between neurons. Learning involves adjusting these weights.
Bias: An additional parameter in a neuron that helps adjust the output along with the weighted inputs, allowing the activation function threshold to be shifted.
Embedding: A way of representing data (like words, sentences, or images) as vectors (lists of numbers) in a multi-dimensional space. Similar items are expected to have similar vectors (be closer together in the space). Crucial for how models “understand” relationships between inputs.
Vector Database: A specialized database designed to efficiently store and query high-dimensional vectors, such as those generated by Embeddings. Used in retrieval-augmented generation (RAG) and similarity searches, often as a memory component for AI agents.
Token: A fundamental unit of data processed by models, especially LLMs. A token can be a word, part of a word (subword), character, or symbol. Text is broken down into tokens before being processed.
Tokenization: The process of breaking down raw input text into a sequence of tokens that the model can understand.
Attention Mechanism: A key component (especially in Transformers) that allows the model to weigh the importance of different parts of the input sequence when generating an output. It helps the model focus on relevant information.
Self-Attention: A specific type of attention mechanism where the model relates different positions of a single input sequence to compute a representation of that sequence. Allows the model to understand context and dependencies within the input itself.
Context Window: The maximum number of tokens (input + output) that a model can consider at one time. A larger context window allows the model to process and remember information from longer prompts or conversations, crucial for complex agent tasks.
AI Agents and Autonomous Systems
AI Agent: An AI system designed to perceive its environment (through data inputs or sensors), reason about situations (often using an LLM or other model), make decisions, and take autonomous actions (e.g., calling APIs, executing code, interacting with software) to achieve specific goals. Agents typically possess components for planning, memory, and tool use.
Agentic AI: A paradigm or approach in AI focused on building systems (agents) that exhibit autonomous, goal-directed behavior. It emphasizes capabilities like planning, reasoning over actions, memory persistence, and the ability to use tools or interact dynamically with an environment to accomplish complex, multi-step tasks.
Single Agent System: An AI system consisting of one AI agent operating within an environment (real or simulated) to achieve its designated goals.
Multi-Agent System (MAS): A system composed of multiple interacting AI agents. These agents may collaborate to solve problems too complex for a single agent, compete for resources, coordinate actions, or negotiate within a shared environment, exhibiting complex emergent behaviors.
Tool Use (in Agents): The capability of an AI agent to utilize external resources or functions (like accessing APIs, running code execution environments, searching the web, querying databases) to gather information, perform actions, or overcome limitations of its internal model.
Planning (in Agents): The process by which an AI agent determines a sequence of actions to achieve its goals, often involving breaking down a complex task into smaller, manageable steps. LLMs are increasingly used for this planning function.
Memory (in Agents): Mechanisms that allow an AI agent to retain and recall information over time, beyond the limits of its immediate context window. This can include short-term working memory for current tasks and long-term memory, potentially stored in vector databases or other structures.
Training & Learning Process
Training Data: The large dataset used to teach an AI model. The model learns patterns, structures, and relationships from this data. The quality and diversity of training data heavily influence model performance and potential biases.
Pre-training: The initial phase where a large model (like a foundation model) is trained on a massive, general dataset. It learns broad knowledge and capabilities during this stage.
Fine-tuning: An optional subsequent training phase where a pre-trained model is further trained on a smaller, task-specific dataset to adapt its capabilities for a particular application (e.g., fine-tuning an LLM for medical Q&A).
Prompt: The input text, image, or other data provided by a user to instruct or query a generative AI model or initiate an AI agent’s task.
Prompt Engineering: The art and science of crafting effective prompts to guide a generative AI model or AI agent towards producing the desired output or behavior. This becomes more complex with agents, involving defining goals, constraints, and available tools.
Parameters: The internal variables of a model that are learned from data during training. These include the Weights and Biases. Large models can have billions or even trillions of parameters.
Loss Function (or Cost Function): A function that measures how inaccurate the model’s predictions are compared to the actual target values in the training data. The goal of training is to minimize this loss.
Gradient Descent: An optimization algorithm used to minimize the loss function by iteratively adjusting the model’s parameters (weights and biases) in the direction that most steeply reduces the loss.
Backpropagation: The algorithm used to calculate the gradients (rate of change of loss with respect to parameters) needed for gradient descent in neural networks. It propagates the error backward through the network layers.
Epoch: One complete pass through the entire training dataset during the training process.
Batch Size: The number of training examples processed in one iteration (one update of the model parameters) during training.
Reinforcement Learning from Human Feedback (RLHF): A technique used to fine-tune models (especially LLMs) based on human preferences. Humans rank or rate different model outputs, and this feedback is used to train a reward model, which then guides the LLM’s training via reinforcement learning to produce more helpful, harmless, and honest outputs. Often used to improve agent behavior.
Model Outputs & Interaction
Text Generation: Creating human-like text (articles, emails, code, poetry, etc.).
Image Generation: Creating novel images from text descriptions (text-to-image) or other inputs.
Code Generation: Automatically writing computer code based on natural language descriptions or partial code snippets.
Inference: The process of using a trained AI model to make predictions or generate outputs on new, unseen data (i.e., using the model after it has been trained). This is the operational phase for models and agents.
API (Application Programming Interface): A set of rules and protocols that allows different software applications to communicate with each other. Many Gen AI models are made available via APIs, and AI agents use APIs as Tools.
Latency: The time delay between providing an input (prompt) to a model/agent and receiving the output or observing an action. Lower latency is generally better for real-time applications.
Throughput: The rate at which a model/agent can process inputs or generate outputs (e.g., tokens per second, tasks completed per minute).
Ethical & Societal Considerations
Hallucination: When a generative AI model produces outputs that are factually incorrect, nonsensical, or unrelated to the input prompt, often stated with high confidence. This is a significant concern for agents performing actions based on potentially flawed reasoning.
Bias (in AI): Systematic errors or unfair outcomes produced by an AI model, often reflecting biases present in the training data or flawed model/agent design. This can lead to discriminatory or inequitable actions by agents.
Alignment: The effort to ensure that AI models’ and agents’ goals and behaviors align with human values and intentions. Crucial for autonomous systems that can take actions in the real world or digital environments.
Safety (AI Safety): The field concerned with preventing unintended and harmful consequences from AI systems, especially autonomous agents, both in the short and long term. Includes topics like robustness, control, and avoiding negative side effects.
Responsible AI: A governance framework for developing, deploying, and managing AI systems (including agents) ethically and accountably, considering fairness, transparency, privacy, security, and societal impact.
Explainable AI (XAI): Methods and techniques aimed at making the decisions, predictions, and actions of AI models and agents understandable to humans. Important for debugging, trust, and accountability.
