Stochastic Optimal Control with Applications to Renewable Energy and Partially Observed Systems

Overview

This dissertation develops continuous-time dynamic programming frameworks for two settings in which classical tools are not directly applicable: the operation of power systems with renewable generation, energy storage, and time delays that induce non-Markovian dynamics; and stochastic optimal control problems in which the system state is only partially observed at discrete times.

The growing integration of renewable energy sources poses challenges for cost-efficient operation of power systems under uncertainty. In short-term power-system operation, decision making is complicated by variability in intermittent renewable generation, storage dynamics, and time delays, which limit the applicability of classical deterministic control methods. More broadly, uncertainty in stochastic control applications also arises from limitations in the availability, accuracy, and timing of state information, motivating continuous-time stochastic optimal control frameworks capable of accommodating dynamical and informational uncertainty.
This thesis addresses applying continuous-time stochastic optimal control to realistic settings that go beyond classical assumptions. It develops dynamic programming based frameworks that accommodate complex system dynamics, such as delays, and informational constraints arising from partial and intermittent observations.

The central questions are: (i) how can dynamic programming–based control be applied to large-scale continuous-time systems whose natural dynamics are non-Markovian due to delays, and (ii) how can optimal control be formulated and computed when the system state is only partially observed at discrete times?
The thesis develops a unified continuous-time optimal control framework for power systems with renewable generation and energy storage, starting from a deterministic formulation and extending it to stochastic uncertainty. Time delays are handled through continuous-time Lagrangian relaxation, yielding a tractable primal-dual formulation whose dual value function is characterized by a Hamilton-Jacobi-Bellman (HJB) partial differential equation. The resulting deterministic and stochastic HJB equations are solved numerically using monotone finite-difference methods, collocation, and operator splitting, and demonstrated on realistic power-system models.

In a complementary theoretical contribution, the thesis studies stochastic optimal control under discrete-time partial observations. By reformulating the problem in terms of conditional state distributions, a measure-valued dynamic programming framework with interlaced HJB equations and Bayesian updates is obtained. For linear-Gaussian systems, this admits a finite-dimensional representation in terms of the conditional mean and covariance.

Overall, the thesis shows how dynamic programming and HJB methods can be extended to settings that do not satisfy the classical Markovian and full-observation assumptions through appropriate relaxation and state-reformulation techniques.