Adaptive automation—automation that is responsive to the human’s performance via the alteration of control laws or level of assistance—is an important tool for training humans to attain new skills when operating dynamical systems. When coupled with cognitive feedback, adaptive automation has the potential to further facilitate human learning, but this requires a system capable of online, automatic, and quantitative assessment of learning. Moreover, to maintain learner motivation, models of human cognitive states such as self-confidence or workload are necessary to predict human cognitive behavior in the learning setting and synthesize optimal policies to achieve training objectives or outcomes. In this talk I will begin by presenting a data-driven approach to assess learning stages in a complex quadrotor landing task that is responsive to stochastic, human-in-the-loop quadrotor dynamics. I will then describe our work on Markov-based modeling of the dynamics of human self-confidence as they evolve during the same quadrotor landing task which in turn enables design and synthesis of different polices aimed at choosing when to provide learners with automation assistance. I will show that policies designed to calibrate learners’ self–confidence to their performance and maximize self–confidence lead to significantly better task performance than is achieved when the decision to provide automation assistance is driven solely by learners’ performance.