Process
Learned policy
You might say: robot learning, AI model for the robot
A learned policy maps observations to actions from demonstrations, reinforcement learning, or both.
- What it does
- Can represent task behavior that is difficult to specify with hand-built models.
- Ask
- How does it behave outside the data and state distribution used for training?
What to look for
- Observations mapped to robot actions
- Behaviour outside the training state distribution
Parts
- Demonstrations: Demonstrations record a human teleoperator's observations and actions while the robot performs a task.
- Data flywheel: A deployment loop collects new physical trajectories, failures, and human interventions, uses them to improve a policy, and redeploys the result.
Catalogue: Learning
- Imitation learning: Imitation learning fits behaviour to recorded demonstrations; Diffusion Policy and ACT are examples for robot manipulation. Fitting behaviour to recorded demonstrations defines it; Diffusion Policy is one example.
- Reinforcement learning: Reinforcement learning improves behaviour through trial-and-error interaction and a reward signal.
- Vision-language-action model: A vision-language-action model combines visual input and a language instruction to produce robot actions.
Sources
- Cheng Chi et al., Diffusion Policy: Visuomotor Policy Learning via Action Diffusion, Abstract
To fully unlock the potential of diffusion models for visuomotor policy learning on physical robots, this paper presents a set of key technical contributions.
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Ch. 1
Reinforcement learning problems involve learning what to do—how to map situations to actions—so as to maximize a numerical reward signal.
Use it on your project
teardown-apply: hold "Learned policy" (learned-policy) from the teardown at https://teardownatlas.com/robotics/ against my project
Paste it into a coding agent that has the teardown skills.
An AI agent wrote this map with the teardown skill. It can contain errors: check the sources before you rely on a part. About and legal