What is Deep Learning?
Deep Learning is a family of machine-learning methods that composes multiple parameterized transformations, usually neural-network layers, to learn representations and task outputs from data under an explicit objective.
Quick Facts
| Created | 2006 by Geoffrey Hinton et al. |
|---|---|
| Specification | Official Specification |
How It Works
Representation learning is the central idea
Earlier layers transform inputs into intermediate representations that later layers can reuse. CNNs encode locality and shared spatial operations, RNNs carry state through a sequence, and Transformers mix token context with attention. These are different inductive biases, not a universal ranking. Raw inputs, engineered features, pretrained representations, and retrieval can all appear in one system.
Training couples architecture, objective, and optimization
A network learns only the signal rewarded by its loss on its data. Initialization, normalization, residual paths, optimizer state, precision, batch construction, and regularization affect whether gradients remain useful. Lower training loss can reflect leakage or shortcut learning, so optimization evidence must be separated from evidence of generalization.
Architecture selection is a workload decision
Choose using input structure, required context, available data, transfer-learning options, training budget, serving latency, memory, throughput, and hardware. A pretrained network can beat training from scratch with less labeled data; a tree or linear model can beat a deep network on a constrained tabular problem. Compare candidates under the same split, metric, and resource envelope.
Evaluation extends beyond one benchmark score
Use a leakage-resistant held-out set, simple baseline, repeated runs when variance matters, calibration checks, and error slices for classes, groups, environments, and input quality. Deployment validation also measures model size, peak memory, batch and tail latency, energy or cost, security, privacy, and behavior under distribution shift.
Key Characteristics
- Builds complex functions by composing multiple learned transformations
- Learns intermediate representations jointly with a task objective
- Uses architectural biases such as locality, recurrence, attention, or latent structure
- Depends on optimization, regularization, data coverage, and numerical stability
- Can exploit transfer learning but still requires target-domain evaluation
- Trades quality against training compute, serving latency, memory, energy, and explainability
Common Use Cases
- Learning visual representations for classification, detection, or segmentation
- Modeling language, speech, audio, and other high-dimensional sequences
- Generating text, images, audio, video, or latent representations
- Processing sensor streams where architecture and state lifecycle match the workload
- Adapting pretrained models under explicit quality, safety, and resource gates
Example
Loading code...Frequently Asked Questions
What makes a neural network deep?
Deep refers to composing multiple stages between input and output, not to one universal layer threshold. Different computational graphs can express similar functions with different apparent depths. What matters operationally is whether the architecture learns useful representations and satisfies the target quality and resource constraints.
How is deep learning different from machine learning?
Deep learning is a family within machine learning centered on layered neural networks and representation learning. Machine learning also includes linear models, trees, kernel methods, clustering, and reinforcement-learning algorithms. A deep model is not automatically better; the comparison must use the same data boundary and task metric.
Does deep learning remove the need for feature engineering?
It can learn many representations jointly from raw or lightly processed inputs, but teams still define sampling, labels, normalization, tokenization, augmentation, context construction, and output semantics. Those choices are feature and data engineering decisions, and they can introduce leakage or bias.
When should I use a pretrained model instead of training from scratch?
Start with a pretrained model when its input domain and license fit the task and labeled data or compute is limited. Compare frozen features, parameter-efficient adaptation, and full fine-tuning against a simple baseline. Train from scratch only when the domain, architecture, data rights, or scale justify the added cost.
How should deep-learning models be evaluated for production?
Measure task quality on untouched and shifted data, inspect failure slices and calibration, and compare with a baseline. Then test peak memory, throughput, tail latency, model loading, numerical precision, cost, privacy, security, and rollback behavior using the exact artifact and preprocessing path intended for deployment.