Control & robotics · RL foundations / Open question
Learning While Staying in Control
A robot's actions change both its physical state and its knowledge of the system. Good experiments can be bad control actions, making learning and control inseparable.
Seed catalog · Showing the bundled record.
Problem statement
Study , , with known bounded smooth , unknown parameters in a compact set, bounded independent noise and compact state/input constraints. A supplied backup controller robustly stabilizes the initial uncertainty set. Determine the excitation cost and regret against the best feasible informed controller, while bounding constraint-violation probability over the learning trajectory.
Assumptions and scope
- Fix the dynamics/noise class, observation map, parameter identifiability and a stabilizable initial uncertainty set or backup controller.
- State the comparator policy class and how persistent excitation is obtained; do not assume the optimal policy supplies all needed information without checking.
- Account for estimation error, planning approximation and constraints across the entire learning trajectory, distinguishing local operating-region guarantees from global control.
What would settle it?
A sharp excitation–regret–stability frontier with a computationally specified adaptive controller, including lower bounds and regimes where near-logarithmic or square-root behavior is attainable.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Smooth parametric nonlinear systems already admit logarithmic regret under optimal-policy excitation and square-root bounds otherwise. Stochastic barrier-based safe learning also has regret guarantees. The target adds partial observation, constrained excitation and learned planning under a declared model, not first nonlinear-control regret. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Wang et al. — Logarithmic Regret for Nonlinear Control ↗partial theory · source checked 2026-11-19
Primary abstract checked: smooth parametric nonlinear dynamics admit logarithmic regret when the optimal policy is persistently exciting, with a square-root bound otherwise. Nonlinear-control regret is already established in restricted regimes.
- Luo, Sun and Kapoor — Sample-Efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions ↗partial theory · source checked 2026-11-19
Primary abstract checked: episodic optimistic control with stochastic barrier constraints, high-probability safety and regret analysis under specified assumptions.
- Wang et al. — Linear RNNs Provably Learn Linear Dynamic Systems ↗partial theory · source checked 2026-11-19
Primary abstract checked: stable linear-system learning with sufficiently wide linear RNNs, including width independent of sequence length and stability-dependent complexity.