← All AI/ML open problems

Control & robotics · RL foundations / Open question

Learning While Staying in Control

A robot's actions change both its physical state and its knowledge of the system. Good experiments can be bad control actions, making learning and control inseparable.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

Study xt+1=Axt+But+∑j=1rθjψj(xt,ut)+wtx_{t+1}=Ax_t+Bu_t+\sum_{j=1}^r\theta_j\psi_j(x_t,u_t)+w_t, yt=Cxt+vty_t=Cx_t+v_t, with known bounded smooth ψj\psi_j, unknown parameters in a compact set, bounded independent noise and compact state/input constraints. A supplied backup controller robustly stabilizes the initial uncertainty set. Determine the excitation cost and regret E∑t<Tℓ(xt,ut)−E∑t<Tℓ(xt⋆,ut⋆)\mathbb E\sum_{t<T}\ell(x_t,u_t)-\mathbb E\sum_{t<T}\ell(x_t^\star,u_t^\star) against the best feasible informed controller, while bounding constraint-violation probability over the learning trajectory.

Assumptions and scope

  • Fix the dynamics/noise class, observation map, parameter identifiability and a stabilizable initial uncertainty set or backup controller.
  • State the comparator policy class and how persistent excitation is obtained; do not assume the optimal policy supplies all needed information without checking.
  • Account for estimation error, planning approximation and constraints across the entire learning trajectory, distinguishing local operating-region guarantees from global control.

What would settle it?

A sharp excitation–regret–stability frontier with a computationally specified adaptive controller, including lower bounds and regimes where near-logarithmic or square-root behavior is attainable.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Smooth parametric nonlinear systems already admit logarithmic regret under optimal-policy excitation and square-root bounds otherwise. Stochastic barrier-based safe learning also has regret guarantees. The target adds partial observation, constrained excitation and learned planning under a declared model, not first nonlinear-control regret. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Wang et al. — Logarithmic Regret for Nonlinear Control ↗partial theory · source checked 2026-11-19

    Primary abstract checked: smooth parametric nonlinear dynamics admit logarithmic regret when the optimal policy is persistently exciting, with a square-root bound otherwise. Nonlinear-control regret is already established in restricted regimes.

  2. Luo, Sun and Kapoor — Sample-Efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions ↗partial theory · source checked 2026-11-19

    Primary abstract checked: episodic optimistic control with stochastic barrier constraints, high-probability safety and regret analysis under specified assumptions.

  3. Wang et al. — Linear RNNs Provably Learn Linear Dynamic Systems ↗partial theory · source checked 2026-11-19

    Primary abstract checked: stable linear-system learning with sufficiently wide linear RNNs, including width independent of sequence length and stability-dependent complexity.