AI Models & Roboflow Quick Reference

1. How to use this document

You will not read this end-to-end. Use it like a dictionary with a learning path on top.

Day 1 of your co-op: skim sections 2–4 (Prerequisites, Suggested Path, the Practical N list).
When you hit an unfamiliar term: Cmd-F the term in section 5.
When you want vision-specific concepts: jump to the sister Vision Models Quick Reference (link in the navbar).

2. Prerequisites

Prerequisite knowledge:

Python: most ML tooling (PyTorch, Hugging Face, Ultralytics, Roboflow SDK) is Python-first.
Basic ML/Vision concepts: training vs. inference, precision/recall, bounding boxes, object detection.
Command line: pip install, running scripts, managing GPU drivers (nvidia-smi).
Docker (for self-hosting): running the Roboflow Inference Server requires docker run.
Basic linear algebra & probability (helpful): dot products, softmax, sampling (temperature, top-k).

Hardware: a laptop with at least 8 GB RAM for running small models on CPU (via Ollama/llama.cpp) is sufficient for learning. A GPU (T4 or better) is needed for training. For edge deployment, a Raspberry Pi 5 or Jetson Orin.

3. Suggested learning path (5 steps)

Do these in order. Each step builds practical AI and deployment skills from scratch.

Step 1 — Train your first vision model from end to end

Upload images to Roboflow, annotate objects, apply preprocessing/augmentations, and train a YOLOv8 model. Learn the annotation → training → export pipeline end to end.

Step 2 — Run inference locally and on edge

Export the model to ONNX, wrap it with a Python script, and run inference on a webcam feed. Then deploy the same model on a Raspberry Pi or Jetson using ONNX Runtime or TensorRT.

Step 3 — Add LLMs and foundation models

Run an LLM locally with Ollama or llama.cpp. Use CLIP for zero-shot classification. Chain a detection model with an LLM for descriptions or Q&A about detected objects.

Step 4 — Build an automated data pipeline

Use Roboflow's API to upload images, trigger re-training, and deploy updated models. Set up active learning where the model flags uncertain predictions for human review.

Step 5 — Deploy as a production service

Self-host Roboflow Inference with Docker. Add a web dashboard showing live detections. Connect the pipeline to MQTT, ROS 2, or a database. Model versioning and rollback via Roboflow.

4. The Practical 11 — read these first

If you only read 11 entries in this glossary, read these. They cover ~90% of what you'll see in your first month on a microcontroller project.

🔥Context Window 🔥Embeddings 🔥Fine-Tuning 🔥LLM (Large Language Model)🔥Prompt Engineering 🔥RAG (Retrieval-Augmented Generation)🔥Token / Tokenizer 🔥Transformer Architecture 🔥Dataset Management (Roboflow)🔥Deployment (Roboflow Inference)🔥Roboflow Inference

Each entry: a one-line definition, then a practical note in italics that says what it means on the bench. Entries marked 🔥 are the practical-11.

Letter

A

2 terms

Attention (Self-Attention / Multi-Head)

The core mechanism in transformer models that lets each token “look at” every other token in the sequence and decide how much to weigh each one. Query, Key, Value vectors are computed for every token; the dot product of Q and K gives the attention weights, which are used to sum the V vectors. Practical: Multi-head attention runs this in parallel across N heads, each capturing different relationships (syntax, semantics, position). The result is concatenated and projected. This is the “A” in “Transformer” and the reason LLMs can handle long-range context. See also: Transformer Architecture, LLM (Large Language Model), Context Window.

Active Learning (Roboflow)

A workflow where the model helps select which unlabelled images should be labelled next. Roboflow's active learning flags images where the model is uncertain (low confidence, high entropy) or where the prediction disagrees with other models (ensemble disagreement). Practical: Active learning reduces labelling effort by 50–80% compared to random sampling. The workflow: deploy model → collect edge-case images → review flagged images in Roboflow → add labels → retrain. Enable on the “Active Learning” tab in your project settings. Works with any Roboflow Inference deployment — flagged images are automatically uploaded. See also: Dataset Management (Roboflow), Roboflow Inference, Model Versioning (Roboflow).

Letter

C

2 terms

Chat Template

A structured format that wraps user and assistant messages into the prompt format a specific model expects. Different models use different delimiters:

<|im_start|>user
...<|im_end|>

, <s>[INST] ... [/INST], or ### Human: ... ### Assistant: .... Practical: Using the wrong chat template produces gibberish output. Hugging Face's tokenizer.apply_chat_template() handles this correctly for most models. When self-hosting with llama.cpp or vLLM, check the model card for the template string. See also: Token / Tokenizer, LLM (Large Language Model), Prompt Engineering.

Context Window

The maximum number of tokens a model can process in a single forward pass. GPT-4: 8K/32K/128K (depending on version), Claude 3: 200K, Gemini 1.5: 1M+ tokens. Practical: A larger context window lets you feed entire documents, codebases, or conversation histories as input. But the attention mechanism scales as O(n²) in the context length (though modern architectures like Mamba, Flash Attention, and sliding-window attention reduce this). For co-op projects, 8K–32K tokens is usually enough. If you exceed the window, the model forgets the middle — not the beginning or end. See also: Attention (Self-Attention / Multi-Head), LLM (Large Language Model), Transformer Architecture.

Letter

D

2 terms

Dataset Management (Roboflow)

Roboflow's platform for curating, versioning, and preprocessing image datasets. Upload images, annotate (or import from COCO/YOLO/Pascal VOC), apply preprocessing (auto-orient, resize) and augmentations (flip, rotate, mosaic, blur, cutout), then export in any format (YOLOv8, COCO, TFRecord, CreateML, etc.). Practical: The dataset versioning system is the killer feature: each version archives a snapshot of images + annotations + preprocessing pipeline. You can train on v1, improve annotations, create v2, and compare model performance across versions. The API (pip install roboflow) lets you download datasets programmatically:

from roboflow import Roboflow; rf = Roboflow(api_key); project = rf.workspace().project('my-project'); dataset = project.version(1).download('yolov8')

Deployment (Roboflow Inference)

Two options: Roboflow Hosted API (upload your model, get a REST endpoint — no infrastructure) or Self-Hosted Inference Server (run Roboflow Inference on your own hardware via Docker). Practical: For prototyping and low-volume applications (< 1000 inferences/day), use the hosted API — no GPU needed. For production (real-time video, high volume, edge devices), self-host: docker run -it --rm --net=host roboflow/roboflow-inference-server:latest. Self-hosted supports GPU acceleration (NVIDIA, Jetson), runs on Raspberry Pi (CPU, 5–15 FPS with YOLOv8n), and works offline. Both expose the same REST API: POST /infer/{project_id}/{version}. See also: Roboflow Inference, Dataset Management (Roboflow), Model Versioning (Roboflow).

Letter

E

2 terms

Embeddings

Dense vector representations of tokens, sentences, images, or other data — typically 384–4096 floats. Semantically similar inputs produce nearby vectors in the embedding space. Practical: Embeddings are the backbone of RAG (Retrieval-Augmented Generation): convert documents to vectors, store in a vector database (Chroma, Qdrant, Pinecone), find the nearest neighbors to a query embedding, and feed them as context to an LLM. Popular embedding models: text-embedding-3-small (OpenAI, 1536-d), all-MiniLM-L6-v2 (384-d, runs on CPU). Use cosine similarity for comparing embeddings. See also: Token / Tokenizer, RAG (Retrieval-Augmented Generation), LLM (Large Language Model).

Export Formats (Roboflow)

Roboflow exports datasets and models in every major format. For datasets: COCO JSON, YOLOv5/v8/v11 PyTorch, Pascal VOC XML, TFRecord, CreateML (iOS), Multi-label CSV. For models: PyTorch, ONNX, TensorFlow SavedModel, TFLite, CoreML, TensorRT, OpenVINO, MediaPipe (for on-device). Practical: The export format determines where you can deploy. TFLite → Android / Raspberry Pi. CoreML → Apple devices. TensorRT → NVIDIA Jetson / GPU servers. OpenVINO → Intel CPUs / VPUs. ONNX → most other platforms. Roboflow handles conversion automatically — select your format in the Export tab. For edge deployment on a Raspberry Pi, choose TFLite or OpenVINO. See also: Deployment (Roboflow Inference), Roboflow Inference, YOLO (You Only Look Once).

Letter

F

1 term

Fine-Tuning

Taking a pre-trained model and continuing training on a smaller, task-specific dataset. Three approaches: Full fine-tuning (update all weights — expensive, needs a GPU), LoRA (train low-rank adapters while freezing the base model — cheap, fast), QLoRA (LoRA + 4-bit quantization of the base model — runs on a single consumer GPU). Practical: For a co-op project, use QLoRA (via Hugging Face PEFT + bitsandbytes) on a T4 GPU (Colab or RunPod, ~$0.50/hr). You need 100–1000 examples. Fine-tuning is for teaching the model a specific format, style, or domain knowledge — not for adding facts (that's RAG (Retrieval-Augmented Generation)). See also: LoRA (Low-Rank Adaptation), GGUF / GPTQ / AWQ (Quantization Formats), LLM (Large Language Model).

Letter

G

1 term

GGUF / GPTQ / AWQ (Quantization Formats)

File formats for quantized models that run on consumer hardware. GGUF is the format used by llama.cpp — runs on CPU (or partially on GPU). GPTQ is GPU-only, 4-bit, requires a calibration dataset. AWQ is similar to GPTQ but with better activation-aware quantization. Practical: For a laptop without a powerful GPU, use GGUF Q4_K_M (4-bit, ~4.5 GB for a 7B model) with llama.cpp or Ollama. For GPU inference, use AWQ or GPTQ — both run on a single 8 GB VRAM card for 7B models. Quantization adds 5–15% inference overhead but reduces memory by 4×. See also: Quantization, LLM (Large Language Model), Inference Server.

Letter

H

1 term

Hallucination

When an LLM generates plausible-sounding but factually incorrect information. Not a bug — it's a feature of the architecture: LLMs are next-token predictors, not databases. Practical: Three mitigations: (1) RAG — ground the model in retrieved documents (reduces hallucination by ~40% in benchmarked studies); (2) temperature — lower temperature (0.0–0.3) reduces creative/confabulated outputs; (3) system prompt — explicitly say “only answer from the provided context; say 'I don't know' if unsure.” Never trust an LLM for facts you cannot verify. See also: RAG (Retrieval-Augmented Generation), Prompt Engineering, Temperature / Top-k / Top-p.

Letter

I

1 term

Inference Server

A service that hosts a model and exposes an API (typically OpenAI-compatible) for generating responses. Options: vLLM (best throughput, PagedAttention, need GPU), llama.cpp / llama-server (CPU or hybrid, GGUF format), Ollama (wrapper around llama.cpp with CLI + REST API), RunPod / Replicate (serverless GPU hosting). Practical: For a co-op project, start with Ollama (download, ollama pull llama3.2:3b, curl localhost:11434/api/generate — done in 10 minutes). For production, use vLLM with AWQ-quantized models on at least one A10G GPU. All of these expose the same /v1/chat/completions endpoint that OpenAI uses. See also: LLM (Large Language Model), GGUF / GPTQ / AWQ (Quantization Formats), OpenAI Compatible API.

Letter

L

3 terms

LLM (Large Language Model)

A neural network trained on massive text corpora to predict the next token in a sequence. The “GPT” family (2, 3, 4, 4o, o1) are decoder-only transformers. Open-weight alternatives in 2026: Llama 3.x (Meta), Mistral, Qwen 2.5, Phi-3/4 (Microsoft), Gemma 2 (Google). Practical: For a co-op project, use an open-weight model (Llama 3.2 3B or Mistral 7B) via Ollama on a laptop — no GPU needed. The 3B models (3–4 GB RAM, CPU) are surprisingly capable for code completion, summarization, and structured output. The 7B models (8 GB+, better with GPU) handle reasoning and dialog. Scale up to 70B+ only when you have a GPU budget. See also: Transformer Architecture, Context Window, Fine-Tuning.

LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning method that inserts trainable low-rank matrices into the transformer layers while keeping the original weights frozen. Typically rank r = 8–64. Practical: LoRA reduces the trainable parameters from billions to millions. A LoRA adapter for a 7B model is ~10–50 MB (vs. 14 GB for the full model). Train with Hugging Face PEFT: from peft import LoraConfig, get_peft_model. Merge the LoRA weights into the base model for deployment (model = model.merge_and_unload()). See also: Fine-Tuning, Quantization, LLM (Large Language Model).

Labeling (Roboflow)

Roboflow provides a browser-based annotation tool for drawing bounding boxes, polygons (segmentation masks), keypoints (pose), and classification labels. Features: auto-segment with SAM (Segment Anything Model), smart polygon (click points, it snaps to edges), label shortcuts, and review/QA workflows. Practical: The SAM integration is the fastest way to label: click once on an object and SAM generates the mask. For bounding boxes, use the “smart brush” on difficult edges. Roboflow supports team annotation — assign labelers, track progress, and review each other's work. For a co-op project, the free tier (1000 images) is sufficient. See also: Dataset Management (Roboflow), SAM (Segment Anything Model), Active Learning (Roboflow).

Letter

M

1 term

Model Versioning (Roboflow)

Each time you create a new dataset version or train a new model, Roboflow archives the complete state: dataset version + preprocessing/augmentation config + model weights + training metrics. Versions are immutable and have unique IDs. Practical: Versioning enables “roll back to v2” when v3's model is worse, and “deploy v2 to staging, v4 to production”. The API endpoint includes the version number: POST /infer/project_id/4 runs version 4. You can compare any two versions side by side on the “Versions” page. This is the closest thing to Git for vision models. See also: Deployment (Roboflow Inference), Dataset Management (Roboflow), Roboflow Inference.

Letter

P

1 term

Prompt Engineering

The practice of designing input prompts to elicit desired behaviour from an LLM. Key techniques: system prompt (set the role and constraints), few-shot (provide 2–5 examples in the prompt), chain-of-thought (“think step by step”), structured output (“respond in JSON with fields x, y, z”). Practical: The system prompt is the most powerful lever — spend time there before changing anything else. Example:

You are a helpful robotics assistant. Answer based only on the provided context. If unsure, say “I don't know”. Format responses as markdown.

. For deterministic output, set temperature = 0. See also: LLM (Large Language Model), Temperature / Top-k / Top-p, Chat Template.

Letter

Q

1 term

Quantization

Reducing the precision of a model's weights (and sometimes activations) from 16-bit or 32-bit floats to 4-bit or 8-bit integers. This reduces memory usage by 2–4× and increases inference speed, with a small accuracy loss (typically 1–3% on benchmarks). Practical: Four levels: fp16/bf16 (no loss, 2× smaller than fp32), int8 (RTN or GPTQ, negligible loss), int4 (GPTQ/AWQ/GGUF Q4_K_M, small loss), int2/int3 (< 4-bit, significant loss — experimental). For a co-op project, use GGUF Q4_K_M (llama.cpp) or AWQ 4-bit (vLLM). Both fit a 7B model in ~5 GB RAM. See also: GGUF / GPTQ / AWQ (Quantization Formats), LLM (Large Language Model), Inference Server.

Letter

R

3 terms

RAG (Retrieval-Augmented Generation)

An architecture that retrieves relevant documents from a knowledge base and feeds them as context to an LLM at query time. The LLM then answers based on the retrieved context rather than its parametric memory. Practical: RAG pipeline: (1) chunk documents into ~500-token pieces, (2) compute embeddings for each chunk, (3) store in a vector DB, (4) at query time, embed the query and find nearest chunks, (5) prepend chunks to the LLM prompt. RAG is cheaper and more maintainable than Fine-Tuning — you can update the knowledge base without retraining. Use LangChain or LlamaIndex for the pipeline. See also: Embeddings, LLM (Large Language Model), Context Window.

Roboflow Inference

The open-source inference server by Roboflow that runs computer vision models (detection, segmentation, classification, keypoint) locally or in the cloud. Supports YOLO (You Only Look Once), ViT (Vision Transformer), SAM (Segment Anything Model), and custom models trained on Roboflow or exported from Ultralytics, Hugging Face, etc. Practical: pip install inference gives you the Python SDK. Or run the Docker container: docker run -it -p 9001:9001 roboflow/roboflow-inference-server:latest. Supports GPU acceleration (CUDA, Jetson), CPU-only mode, and edge devices (Raspberry Pi, NVIDIA Jetson). pip install inference-cli && inference server start is the quickest path for a local server. The API is OpenAI-style compatible for GPT-4o vision tasks. See also: Deployment (Roboflow Inference), Dataset Management (Roboflow), YOLOv8 / YOLOv11.

Roboflow Universe

A public repository of 200K+ vision datasets and pre-trained models, contributed by the community. Searchable, freely downloadable under permissive licenses. Practical: Before labelling your own dataset, search Universe — someone may have already done it. You can fork an existing dataset, add your own images, and retrain. Universe also hosts pre-trained models that you can deploy immediately via the Roboflow API. This is the fastest path to a working vision pipeline: find an existing model → deploy with one click → test in your environment → fine-tune if needed. See also: Dataset Management (Roboflow), Roboflow Inference, COCO (Common Objects in Context).

Letter

S

2 terms

Structured Output / JSON Mode

Constraining an LLM to output valid JSON (or another structured format) with a specified schema. Methods: (1) prompt engineering (“respond in JSON: {"summary": ..., "sentiment": ...}”), (2) constrained decoding (grammar-based sampling — llama.cpp's --grammar, Outlines library), (3) function/tool calling (OpenAI tools parameter, cohere's tool use). Practical: For production, use constrained decoding — it guarantees valid JSON and avoids the 5–15% failure rate of prompting alone. llama.cpp supports GBNF grammars; Outlines works with any Hugging Face model. Function calling is the easiest path for API-hosted models. See also: Prompt Engineering, LLM (Large Language Model), Inference Server.

Self-Hosted Inference (Edge / IoT)

Running Roboflow Inference on a device without internet connectivity or a persistent cloud connection. Supported devices: Raspberry Pi 4/5 (CPU), NVIDIA Jetson Nano/Orin, Intel NUC, any Linux machine with Docker. Practical: The self-hosted server uses the same REST API as the cloud version, but runs fully offline after the initial model download. The Docker image is ~2 GB and includes all supported model architectures. On a Raspberry Pi 5, YOLOv8n runs at 15–25 FPS (CPU, OpenVINO backend). For lower power, export to TFLite and use the Roboflow mediapipe integration. The inference Python package can also run without Docker for lightweight setups. See also: Deployment (Roboflow Inference), Export Formats (Roboflow), Roboflow Inference.

Letter

T

4 terms

Temperature / Top-k / Top-p

Sampling parameters that control the randomness of LLM output. Temperature (0.0–2.0): scales the logits before softmax — low = deterministic, high = creative. Top-k: pick from only the k highest-probability tokens. Top-p (nucleus sampling): pick from the smallest set of tokens whose cumulative probability exceeds p. Practical: For code generation or fact retrieval: temperature = 0 (always the same answer). For creative writing or brainstorming: temperature = 0.7–0.9, top-p = 0.9. For chatbot use: temperature = 0.3–0.5 balances coherence with variety. Top-k is rarely used in modern pipelines (top-p supersedes it). See also: LLM (Large Language Model), Prompt Engineering, Structured Output / JSON Mode.

Token / Tokenizer

A tokenizer converts text into integers (token IDs) that the model can process, and converts output IDs back to text. Subword tokenizers (BPE, Unigram, WordPiece) split words into common substrings: “unbelievable” → [“un”, “believable”]. Practical: Different models use different tokenizers. Llama 3 uses a BPE tokenizer with 128K tokens; GPT-4 uses a BPE tokenizer with ~100K tokens. One token is roughly 0.75 words for English (varies by language). The tokenizer determines the model's vocabulary and maximum Context Window. Always use the model's matching tokenizer — mismatched tokenizers produce garbage. Hugging Face's AutoTokenizer handles this automatically. See also: Context Window, LLM (Large Language Model), Embeddings.

Tool Calling / Function Calling

A capability where an LLM can request the execution of external functions (API calls, database queries, code execution) and use the results in its response. The model outputs a structured request (function name + arguments), the application executes it and feeds back the result. Practical: Supported natively by OpenAI (tools parameter), Anthropic Claude, and open models via the OpenAI-compatible API. For local models, use the --grammar feature of llama.cpp or Ollama's tool-calling support. The model needs to be fine-tuned for reliable tool calling — not all base models support it out of the box. See also: Structured Output / JSON Mode, LLM (Large Language Model), Prompt Engineering.

Transformer Architecture

The neural network architecture introduced in “Attention Is All You Need” (Vaswani et al., 2017) that underlies virtually every modern LLM. Composed of stacked encoder and decoder layers (GPT-style decoder-only, BERT-style encoder-only, T5-style encoder-decoder). Each layer has multi-head self-attention + feed-forward network, with residual connections and layer normalization. Practical: Decoder-only transformers (GPT, Llama, Mistral) are the dominant architecture for generative models because they are simpler to scale and train. Encoder-only models (BERT, RoBERTa) are still best for embedding/fine-tuning tasks. The “attention is all you need” insight was that attention alone (no RNNs or CNNs) captures sequence relationships well enough. See also: Attention (Self-Attention / Multi-Head), LLM (Large Language Model), Context Window.

Letter

U

1 term

Ultralytics HUB Integration

Roboflow and Ultralytics HUB are complementary platforms. Train models on Roboflow (free GPU hours for small datasets) or Ultralytics HUB, then deploy on Roboflow Inference. Both platforms support YOLOv8 / YOLOv11 and YOLOv8 / YOLOv11 training. Practical: Typical workflow: upload dataset to Roboflow → preprocess/augment → export in YOLOv8 format → train with Ultralytics HUB → get model weights → deploy on Roboflow Inference server. The Roboflow Inference server can load any Ultralytics-exported model (.pt file) directly. For the fastest path: annotate in Roboflow, train in one click with Roboflow Train, deploy to the hosted API — all within the Roboflow web UI. See also: YOLOv8 / YOLOv11, Roboflow Inference, Dataset Management (Roboflow).

Letter

W

1 term

Webhook / Callback (Roboflow)

An HTTP callback that Roboflow fires when an inference result is ready (for async/batch processing) or when an active learning image is uploaded. Practical: Use webhooks to pipe detection results into your application without polling: configure a webhook URL in Roboflow → every inference sends a POST with the results → your server processes them (e.g. save to database, trigger an alert, update a dashboard). Webhooks support retries with exponential backoff. For active learning, the callback sends the flagged image URL + model predictions so you can review them in your own labelling pipeline. See also: Active Learning (Roboflow), Roboflow Inference, Deployment (Roboflow Inference).

No matches

Try a different term, or press Esc to clear the search.

6. Quick reference tables

6.1 Popular model architectures

Model	Task	Size	Framework
YOLOv8 / v11	Object detection	3–100 MB	PyTorch, ONNX, TensorRT
SAM 2	Image segmentation	~350 MB	PyTorch, ONNX
CLIP	Zero-shot classification	~600 MB	PyTorch, ONNX
LLaMA / Qwen	LLM (text gen)	3–70B params	GGUF, vLLM
ResNet / MobileNet	Classification	5–30 MB	PyTorch, ONNX
ViT (Vision Transformer)	Classification	~350 MB	PyTorch, ONNX

6.2 Roboflow ecosystem

Tool	Purpose	Access
Roboflow Universe	Public dataset hub	Free
Roboflow Annotate	AI-assisted image annotation	Web UI
Roboflow Train	Cloud model training	Credit-based
Roboflow Inference	On-prem or cloud inference	Self-hosted / API
Roboflow Deploy	Edge device deployment	Docker / SDK
Roboflow API	Programmatic dataset management	REST

7. Further reading

🔥 Roboflow docs — docs.roboflow.com
🔥 Ultralytics YOLO docs — docs.ultralytics.com
🔥 ONNX Runtime — onnxruntime.ai
📚 Hugging Face Transformers — huggingface.co
📚 Roboflow Inference GitHub — github.com/roboflow/inference

Last updated: July 2026. Maintained as a living document — if a term on your co-op isn't here, add it.