Original package image
AI Models & Assistants / Knowledge library
Adversarial Examples
Inputs deliberately perturbed to induce model errors
Why learn Adversarial Examples?
They expose robustness gaps that ordinary accuracy can hide.
Where it came from
Adversarial examples reveal that small or structured input changes can cause unexpected predictions. Threat models define which perturbations and attacker capabilities matter.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Identify an adversarial example: What makes an input adversarial?
- Specify a threat model: Why state an attacker's allowed perturbation norm and knowledge?
- Track clean performance: A defense improves attacked accuracy but sharply lowers clean accuracy. What should be reported?
- Use adaptive attacks: Why test a defense with attacks adapted to that defense?
- Avoid overclaiming: Robustness to one bounded pixel attack proves what?
Documentation and sources
- Primary reference (arxiv.org)
Connect the ideas
Explore next
Real World Example
Automated Logistics Security for Smart Warehousing
A high-speed automated logistics firm utilizing computer vision to sort packages and verify shipping labels.
The business problem
The firm faces a critical failure where malicious actors place subtle, non-random stickers on packages. These inputs, deliberately perturbed to induce model errors, cause the vision system to misclassify high-priority medical shipments as low-priority bulk waste, leading to severe operational delays and potential safety risks.
How this topic solves it
The firm implements adversarial training to harden the vision model. By generating adversarial examples—inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset—and injecting these into the training loop, the model learns to ignore noise-based manipulations that previously triggered high-confidence incorrect outputs.
- Identify the model's loss function and calculate the gradient of the loss with respect to the input pixels.
- Apply a small, calculated perturbation to the input image in the direction of the gradient to maximize the classification error.
- Label these perturbed images with their correct ground-truth categories.
- Retrain the neural network using a mixed dataset containing both original clean images and the generated adversarial examples.
- Validate the model against a hold-out set of adversarial inputs to ensure robustness.
This flow shows how adversarial examples are integrated into the training process to improve model robustness.
Calculates worst-case noise
Perturbed input image
Updates model weights
Final hardened classifier
- Clean Input → computes gradient → Perturbation Generator
- Perturbation Generator → applies noise → Adversarial Example
- Clean Input → original data → Training Loop
- Adversarial Example → adversarial data → Training Loop
- Training Loop → final weights → Robust Model
Expected business outcome
The vision system achieves increased resilience against intentional input manipulation, ensuring that package classification remains stable even when perturbed inputs are introduced into the sorting pipeline.
Limitations and checks
The primary trade-off is a potential decrease in classification accuracy on clean, non-perturbed data. This occurs because the model is forced to generalize across a wider, more complex input space. To check this, developers must monitor the validation accuracy on clean datasets after every training epoch to ensure the model does not overfit to the adversarial noise.