Inverse RL & imitation · RL foundations · Data-efficient learning / Open question
Learning Intentions That Transfer
The right demonstration can distinguish intentions that look identical in ordinary behavior. Active design could make inverse RL useful for new robotic tasks with few expert interactions.
Seed catalog · Showing the bundled record.
Problem statement
Use a finite menu of known discounted MDPs sharing features and reward , . Each purchased demonstration follows the entropy-regularized optimal policy at a fixed known temperature; menu entries have different transition kernels and costs. Determine the minimum adaptive demonstration budget for on a given finite test menu, including uncertainty from noisy or partially observed trajectories and irreducible reward equivalences.
Assumptions and scope
- Fix a common reward class, allowed environment changes and a stable demonstrator model; reward drift or changed human objectives are separate assumptions.
- Specify dynamics knowledge, demonstration noise and query costs. Some reward transformations may remain unidentifiable under every allowed intervention.
- Compare to an oracle designer with the same intervention budget and evaluate transfer regret, not arbitrary raw-parameter recovery.
What would settle it?
Matching intervention/trajectory complexity for a specified reward-environment family, with an adaptive design and robust transfer bound or a lower bound showing insufficient intervention richness.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Identification from sufficiently different environments or discount factors is already established, as are fast statistical IRL rates in specified models. The target is cost-optimal adaptive environment selection with imperfect observations and a transfer guarantee, not basic multi-environment identifiability. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Cao, Cohen and Szpruch — Identifiability in Inverse Reinforcement Learning ↗partial theory · source checked 2026-11-19
Primary search abstract checked: entropy-regularized reward equivalence; sufficiently different environments or discount factors can identify rewards up to a constant under the stated conditions.
- Schlaginhaufen and Kamgarpour — Fast Rates for Inverse Reinforcement Learning ↗partial theory · source checked 2026-11-19
Primary abstract checked: entropy-regularized Min-Max-IRL, O(1/n) trajectory-KL and Hessian-norm parameter rates for linear rewards, with a local minimax result in the well-specified deterministic case; misspecification and general Borel spaces are also addressed.
- Skalse and Abate — Partial Identifiability and Misspecification in Inverse Reinforcement Learning ↗partial theory · source checked 2026-11-19
Primary abstract checked: ambiguity and misspecification robustness are characterized for common behavioral models. Basic reward ambiguity is an established result, not a new conjecture.