AI Coaching for Accelerating Human Skill Development with Reinforcement Learning

Wei Wang Enlin Gu Antonio Loquercio Haimin Hu^† Rahul Mangharam^†

^†Equal advising

AI copilots can substantially boost human performance, but excessive assistance can induce over-reliance and skill atrophy. We study how an AI agent can act as a coach to accelerate human motor-skill development. We argue that effective coaching requires strategic scaffolding and stepping back that are aligned with the learner's capability, allowing productive failures that drive learning. We formalize the interactive AI coaching process as a non-cooperative dynamic game in which the coach targets the learner's independent competence. Building on this formalism, we develop a reinforcement learning framework to enable tractable training of coaching policies.

Overview Video

Formulation: The Human-AI Coaching Game

Formally, we cast the AI coaching problem as a two-player partially observable stochastic game (POSG), which we refer to as a Human-AI Coaching Game. We argue that coaching is fundamentally a non-cooperative game where the human and AI have different objectives: the learner seeks to maximize task performance, while the coach optimizes a pedagogical objective, namely the learner's independent competence.

A Human-AI Coaching Game is defined by the tuple \[ \mathcal{G} = \bigl(\mathcal{S},\mathcal{A},P, \Theta,\mathcal{O}^{\mathrm{H}},\mathcal{O}^{\mathrm{AI}}, O^{\mathrm{H}},O^{\mathrm{AI}},r^{\mathrm{H}},r^{\mathrm{AI}},p_0,\gamma\bigr). \] Here, \(\mathcal{S}\) is the set of physical states, \(\mathcal{A}\) is the action set shared by the learner and coach, and \(\Theta\subset[0,1]\) is a finite, totally ordered set of latent skill levels.

The joint transition captures how the physical states evolve and how the coach's action causally influence the learner's skill progressions:

\[ P(s', \theta' \mid s, \theta, a^{\mathrm{H}}, a^{\mathrm{AI}}). \]

The learner's objective is a task reward collected under the combined effect of their own actions and the coach's intervention:

\[ r^{\mathrm{H}}(s_t, a^{\mathrm{H}}_t, a^{\mathrm{AI}}_t) = r^{\mathrm{task}}(s_t, a_t). \]

The coach is graded on a different, pedagogical exam: not how well the learner performs now with help, but how well they will perform later without AI help. We define this as the Value of Independence (VoI):

\[ r^{\mathrm{AI}}(s_t; \theta_t) = \mathbb{E}\!\left[ \sum_{k=0}^{\infty} \gamma^k\, r^{\mathrm{task}}(s_{t+k},\, a^{\mathrm{H}}_{t+k}) \;\middle|\; \begin{array}{l} a^{\mathrm{H}}_{t+k} \sim \pi^{\mathrm{H}}(\cdot \mid o^{\mathrm{H}}_{t+k}; \theta_{t+k}), \\[2pt] j_{t+k+1} \sim P(\cdot \mid j_{t+k}, a^{\mathrm{H}}_{t+k}, a^{\mathrm{AI}}_{t+k}=0) \end{array} \right] \]

Intuitively, VoI asks: if the coach were to walk away right now, how well would the learner perform?

The AI coach teaches primarily through physical interventions: it implements shared control by fusing the learner's action with an expert control via an adaptive blending rule:

\[ a = \lambda \odot \pi^*(o^{\mathrm{AI}}) + (\mathbf{1}-\lambda) \odot a^{\mathrm{H}}, \]

where \(\lambda = \pi^{\mathrm{AI}}_{\lambda}(o^{\mathrm{AI}}) \in [0,1]^{\dim(\mathcal{A})}\) is a per-axis blending vector that modulates the coach's assistance level, continuously from 0 (no assistance) to 1 (full override).

Method: Learning to Coach (L2C)

We show that the Human-AI Coaching Game can be effectively reduced to a POMDP amenable to model-free reinforcement learning. To enable scalable, GPU-parallelized training of the coaching policy, we augment the robot's physical dynamics with two lightweight learner-side components grounded in cognitive science: a skill-conditioned noisily-rational control policy, and a probabilistic model of how the learner's latent skill evolves in response to coaching events such as success or failure on task attempts.

At deployment, the coach maintains a belief state over learner skill and uses the learned blending policy to combine the learner command with the expert action, allowing the system to execute personalized scaffoldding or stepping back according to the inferred learner skill level and task context.

L2C training and deployment overview — Overview of L2C: RL-based coach training, adaptive action blending, and live coaching with skill belief.

Experimental Results

We conducted a user study with \(N=33\) participants from diverse drone-flying backgrounds in a high-fidelity FPV drone racing simulator. Each participant was randomly assigned to one of three AI coaches: L2C and two baselines, RBF (rule-based fading) and MIA (minimally-invasive assistance). We measure human skill change with two metrics: lap time and failure count.

For L2C, a paired-samples t-test comparing pre- and post-coaching lap times yielded a mean reduction of \(27.9\%\) with \(p=0.005\) and Cohen's \(d_z=-1.08\). Similarly, a paired t-test on per-lap failure count yielded a mean reduction of 3.52 failures per lap with \(p \lt 0.001\) and Cohen's \(d_z=-2.13\).

For RBF, paired tests showed no reliable change in lap time, but the paired test on failure count showed significance. For MIA, paired tests showed no reliable change in lap time or failure count. All four between-method contrasts favor L2C with medium-to-large effect sizes.

Experimental results comparing L2C, RBF, and MIA — After coaching, learners trained with L2C show significant reductions in lap time and failure count and report higher safety, performance, agency, and satisfaction than both baselines.

Pre- and Post-Coaching Trajectories

The objective metrics show that L2C achieves larger reductions in lap time and failure count than both baselines. This figure provides qualitative examples of how human trajectories change after coaching.

Pre and post coaching trajectory examples — Example pre- and post-coaching evaluation trajectories for the three coaching methods. Rows correspond to coaching methods; columns compare pre- and post-coaching evaluation laps.

Assistance Adaptation

We inspect how L2C changes assistance based on the estimated learner skill and physical context. The magnitude of the blending vector along example trajectories shows two emerging forms of adaptation: across learners, a novice receives more assistance than a more skilled learner; within each trajectory, \(|\lambda|\) varies sharply near the gates, the task-critical states that determine whether the learner collides or passes successfully. As the human upskills, the coach gradually steps back, allowing the learner to take more control and experience productive failures.

L2C assistance coefficient visualization — L2C assistance coefficient \(|\lambda|\) visualized on example racing trajectories. Left: estimated skill \(\hat{\theta}=0\); right: estimated skill \(\hat{\theta}=0.5\).

Simulation Environment

We built an immersive first-person-view (FPV) and physically realistic drone-racing simulation environment in Isaac Lab. The environment consists of 12 gates in a figure-eight layout. Participants are instructed to fly through the gates in the prescribed order and complete the course as quickly as possible.

We structure each trial into three phases: a pre- and post-coaching test, and a main coached training session. To prevent participant fatigue, we limit the training session to 15 laps and restrict the human's control authority to yaw and roll rate while automating the pitch rate and thrust.

Drone racing simulation assets and track layout — Drone-racing simulation assets: quadrotor model visualization, true-scale gate-quadrotor comparison, and top-down track map showing gate order, positions, headings, and traversal direction.

Policy Training

We train both a expert racing policy and an L2C coaching policy with PPO. The coaching policy is trained with a skill-improvement objective, one that is provably aligned with the VoI, so that it optimizes the learner's long-term independent competence instead of assisted task performance.

The reward curves report the expert policy and coaching policy training runs. The auxiliary reward terms encourage a stable gate-facing flight style, making the resulting trajectories easier for human learners to interpret and imitate.

Training reward curves — Reward curves for the expert policy and the learned L2C coaching policy.

Citation

@misc{wang2026aicoachingacceleratinghuman,
      title={AI Coaching for Accelerating Human Skill Development with Reinforcement Learning},
      author={Wei Wang and Enlin Gu and Antonio Loquercio and Haimin Hu and Rahul Mangharam},
      year={2026},
      eprint={2606.25337},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2606.25337},
}