Abstracts:The robust integral of the sign of the error (RISE) control approach results in a powerful continuous controller that yields exponential tracking error convergence despite the presence of time-varying and state-dependent disturbances. However, designing the RISE controller to yield exponential tracking error convergence in the presence of actuator saturation has been an open problem. Although there are existing results that provide a saturation scheme for RISE controllers, these results only guarantee asymptotic tracking error convergence using a Lyapunov-based analysis. In this article, a new design strategy is developed using a projection algorithm and auxiliary filters to yield exponential tracking error convergence. This new strategy does not employ trigonometric or hyperbolic saturation functions inherent to previous saturated (or amplitude limited) controllers. As a result, a Lyapunov-based analysis can be constructed that yields exponential convergence of the tracking errors. Comparative simulation results demonstrate the performance of the developed method in comparison with a baseline controller. The developed method can operate at a lower saturation limit than the baseline method while maintaining stability and achieving exponential tracking error convergence.

Abstracts:The application of distributed model predictive controllers (DMPC) for multiagent systems (MASs) necessitates communication between agents, yet the consequence of communication data rates is typically overlooked. This work focuses on developing stability-guaranteed control methods for MASs with limited data rates. Initially, a distributed optimization algorithm with dynamic quantization is considered for solving the DMPC problem. Due to the limited data rate, the optimization process suffers from inexact iterations caused by quantization noise and premature termination, leading to suboptimal solutions. In response, we propose a novel real-time DMPC framework with a quantization refinement scheme that updates the quantization parameters on-line so that both the quantization noise and the optimization suboptimality decrease asymptotically. To facilitate the stability analysis, we treat the suboptimally controlled MAS, the quantization refinement scheme, and the optimization process as three interconnected subsystems. The cyclic-small-gain theorem is used to derive sufficient conditions on the quantization parameters for guaranteeing the stability of the system under a limited data rate. Finally, the proposed algorithm and theoretical findings are demonstrated in a multi-AUV formation control example.

Abstracts:Learning policies in an asynchronous parallel way is essential to numerous successes of reinforcement learning for solving complex problems. However, their convergence has not been rigorously evaluated. To improve the theoretical understanding, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, multiple workers independently perform system rollouts to estimate zero-order policy gradients (PGs), which are then aggregated in a central node for policy updates. Moreover, each worker is allowed to interact with the central node asynchronously, leading to delayed PG estimates. By quantifying the convergence rate of AZOPG, we show a linear speedup property both in theory and simulation, which reveals the advantages of using asynchronous parallel workers in learning policies.

Abstracts:This article studies the contraction property of time-varying differential-algebraic equation (DAE) systems by embedding them to higher dimension ordinary differential equation (ODE) systems. The first result pertains to the equivalence of the contraction of a DAE system and the uniform global exponential stability of its variational DAE system. Such equivalence inherits the well-known property of contracting ODE systems on a specific manifold. Subsequently, we construct an auxiliary ODE system from a DAE system whose trajectories encapsulate those of the corresponding variational DAE system. Using the auxiliary ODE system, a sufficient condition for contraction of the time-varying DAE system is established by using matrix measures, which allows us to estimate a lower bound on the parameters of the auxiliary system. Finally, we apply the results to analyze the stability of time-invariant DAE systems, and to design observers for time-varying ODE systems.

Abstracts:We consider the problem of optimally controlling stochastic, Markovian systems subject to joint chance constraints over a finite-time horizon. For such problems, standard dynamic programming is inapplicable due to the time correlation of the joint chance constraints, which calls for non-Markovian, and possibly stochastic, policies. Hence, despite the popularity of this problem, solution approaches capable of providing provably optimal and easy-to-compute policies are still missing. We fill this gap by augmenting the dynamics via a binary state, allowing us to characterize the optimal policies and develop a dynamic programming-based solution method.

Abstracts:In this article, we investigate the optimal control problem for an unknown linear time-invariant system. To solve this problem, a novel composite policy iteration algorithm based on adaptive dynamic programming is developed to adaptively learn the optimal control policy from system data. The existing methods require the initial stabilizing control policy, the persistence of excitation (PE) condition and the data storage to ensure the algorithm convergence. Fundamentally different from them, these restrictions can be relaxed in the proposed method. Specifically, an adaptive parameter is elaborately designed to remove the requirement of the initial stabilizing control policy. Besides, an online data calculation scheme is proposed, which cannot only replace the stored historical data by online data, but also can relax the PE condition to the interval excitation condition. The simulation results demonstrate the efficacy of the proposed algorithm, and its superiority is also demonstrated by comparing it with existing algorithms.

Abstracts:This article addresses the problem of robust performance analysis and synthesis for uncertain linear systems with state-space matrices containing parameters in bounded intervals. The novelty of the approach relies on treating the interval bounds as optimization variables, leading to two significant results. First, an analysis tool that evaluates the sensitivity of closed-loop performance criteria, such as the $\mathcal {H}_\infty$ norm, with respect to the bounds, is presented. The second contribution is an output-feedback synthesis tool aimed to enhance the resilience of controllers while maintaining or even optimizing closed-loop performance. Both contributions are formulated as linear matrix inequality-based algorithms, ensuring efficient computational solutions. The capability and effectiveness of the method are illustrated through numerical experiments.

Abstracts:This article addresses the exponential regulation problem for uncertain nonlinear impulsive systems with generalized triangular structures. Different from the existing works, the nonlinear growth rates under consideration are allowed to contain unknown constants and state-input-dependent functions, and the impulse effects are allowed to have unknown sizes. This problem has remained unsolved because it exceeds the coverage of common assumptions in impulsive systems. To this end, we propose a novel logic-based switching gain approach. Specifically, the well-designed logic-based switching mechanism detects the Lyapunov function in real time to adjust the candidate gain online such that the gain grows large enough to effectively dominate the strong nonlinearities and serious uncertainties. Interestingly, the proposed approach removes the restriction on impulse frequency (reflected by the average impulsive interval constant) within the lower-triangular structural framework, and executes only one switching action to obtain the upper bound of the required gain within the upper-triangular structural framework. It is shown that the logic-based switching adaptive controller ensures that all the closed-loop signals are bounded and the system states converge to the origin at an exponential rate. Finally, the effectiveness of the presented results is demonstrated by two representative examples.

Abstracts:Pursuing faster convergence rates and smaller input magnitudes seem to be two conflicting goals in studying multiagent systems. To give a tradeoff between the two, this article focuses on the bipartite synchronization problems over signed topologies and aims to achieve finite-time control for general linear agents subject to input saturation constraints. First, this article considers homogeneous agents and presents a class of bipartite synchronization protocols with saturation constraint, which exploits the solution of the time-varying Riccati equation (TVRE) to design the control gain. Then, a time-varying parameter scheduler is tactically designed for TVRE and achieves finite-time bipartite synchronization. Note that the design uses the solution computed online and brings a bit of conservatism in determining the settling time. So, for heterogeneous agents, this article constructs a modified parameter scheduler computed off-line to reduce the conservatism. A class of finite-time bipartite synchronization generators and generator-based finite-time protocols are proposed. It shows that, in both designs, the control input subjects to the bound saturation during convergence even if the gain escapes to infinity towards the settling time. Moreover, the tradeoff among the finite convergence time, the saturation bound of the input, and the initial domain are analyzed explicitly in theory. Finally, two simulations verify the validity of the theoretical results.

Abstracts:This article deals with the gradient extremum seeking control for static scalar maps with actuators governed by distributed diffusion partial differential equations (PDEs). To achieve the real-time optimization objective, we design a compensation controller for the distributed diffusion PDE via backstepping transformation in infinite dimensions. A further contribution of this article is the appropriate motion planning design of the so-called probing (or perturbation) signal, which is more involved than in the nondistributed counterpart. Hence, with these two design ingredients, we provide an averaging-based methodology that can be implemented using the gradient and Hessian estimates. Local exponential stability for the closed-loop equilibrium of the average error dynamics is guaranteed through a Lyapunov-based analysis. By employing the averaging theory for infinite-dimensional systems, we prove that the trajectory converges to a small neighborhood surrounding the optimal point. The effectiveness of the proposed extremum seeking controller for distributed diffusion PDEs in cascade of nonlinear maps to be optimized is illustrated by means of numerical simulations.