What is GPT?
GPT (Generative Pre-trained Transformer) is OpenAI's family of generatively pre-trained Transformer models. The original GPT design learned a causal language-model objective over Token sequences, then adapted the resulting model to downstream language tasks.
Quick Facts
| Full Name | Generative Pre-trained Transformer |
|---|---|
| Created | 2018 (GPT-1 released by OpenAI) |
| Specification | Official Specification |
How It Works
Pre-training learns conditional token distributions
Given a prefix, causal language modeling assigns probability to the next Token and minimizes prediction loss across training sequences. Original GPT work then fine-tuned the same Transformer on labeled downstream tasks; later GPT research also demonstrated In-context Learning without gradient updates. Neither route proves that a generated statement is factual: the objective rewards probable continuations under the training and adaptation process.
Post-training changes behavior without redefining GPT
Instruction tuning, preference optimization, safety training, tool schemas, and system instructions can make a model easier to direct, but they are separate stages and contracts. Public model names do not disclose every architectural or data detail. Do not infer a parameter count, training corpus, modality, or reasoning mechanism unless the provider or a primary technical report states it for that exact model snapshot.
Inference is an autoregressive decoding loop
The runtime tokenizes the current context, performs a forward pass, obtains logits, applies any allowed-token constraints and decoding policy, selects a Token, appends it, and repeats until a stop condition or output limit. Temperature, top-p, Seed support, hidden reasoning, cached state, and tool-call formats are model- and API-specific. They should be versioned with the prompt and evaluation record.
Evaluate the model and the application separately
Model benchmarks test behavior under a particular prompt, dataset, and scoring protocol. A production application also depends on retrieval, tools, permissions, UI, latency, cost, refusal handling, and monitoring. Use representative held-out tasks, contamination checks, repeated trials where sampling is stochastic, subgroup and language slices, evidence verification, and a baseline that could solve the same task without GPT.
Key Characteristics
- Uses generative pre-training over token sequences as the original learning objective
- Applies causal attention so a position cannot use future target tokens during prediction
- Generates autoregressively by repeatedly selecting and appending a next Token
- Can be adapted through supervised, preference, safety, or other post-training stages
- May perform tasks from instructions or demonstrations without updating weights
- Has capabilities and limits that must be stated for an exact model and system version
Common Use Cases
- Drafting and transforming text with human review appropriate to the error cost
- Extracting structured candidates that undergo schema and business validation
- Question answering grounded in versioned, authorized evidence
- Code assistance backed by compilation, tests, security review, and repository context
- Conversational systems with explicit state, tool, escalation, and retention policies
Example
Loading code...Frequently Asked Questions
What does GPT stand for?
GPT stands for Generative Pre-trained Transformer. Generative refers to modeling and producing sequences, pre-trained means the model first learns from a broad corpus before task adaptation, and Transformer identifies the neural architecture family used by the original model. The acronym does not specify one API, parameter count, or product.
How does GPT generate text?
For each step, the model processes the available Token prefix and produces scores over possible next Tokens. The runtime applies its decoding policy, selects one Token, appends it, and repeats. The final sequence depends on the exact model, context, decoding settings, constraints, and stop conditions; fluent output is not proof of factual accuracy.
What is the difference between GPT and ChatGPT?
GPT refers to OpenAI's model family. ChatGPT is a user-facing service that can combine selected models with system instructions, conversation management, tools, retrieval, safety controls, and product policies. A ChatGPT feature should not be assumed to exist in every GPT model or API, and a model capability does not describe the complete product.
Does GPT learn from each conversation?
Ordinary inference changes the current context, not the deployed model weights. A provider may separately retain data, collect feedback, or train a later model according to its product policy, but that is a different lifecycle step. Application memory is also separate: it must be explicitly stored, authorized, expired, and deleted by the surrounding system.
How should GPT models be compared?
Freeze the task definition, prompt and tool contract, model snapshot, decoding settings, data, grading rules, repetitions, latency and cost accounting. Test target-language and risk slices, contamination, refusal behavior, and evidence support. Provider benchmarks can be useful evidence for their stated setup, but they do not replace evaluation on the application's own distribution.