Research on virtual entity decision model for LVC tactical confrontation of army units
Keywords:TrainingHeuristic algorithmsNeural networksStochastic processesReinforcement learningGamesReal-time systemsdecision makinglearning (artificial intelligence)military computingmulti-agent systemsstochastic gamesvirtual entity decision modelLVC tactical confrontationarmy unitslive-virtual-constructive tactical confrontationgraded combat capabilitydiversified actionsreal-time decision-makingconfrontation processzero-sum stochastic gamedynamic relative power potential fieldreward sparsityreward shapingextensible multiagent deep reinforcement learning frameworksolving methodmodel solvinglive-virtual-constructive (LVC)army unittactical confrontation (TC)intelligent decision modelmulti-agent deep reinforcement learning
Abstracts:According to the requirements of the live-virtual-constructive (LVC) tactical confrontation (TC) on the virtual entity (VE) decision model of graded combat capability, diversified actions, real-time decision-making, and generalization for the enemy, the confrontation process is modeled as a zero-sum stochastic game (ZSG). By introducing the theory of dynamic relative power potential field, the problem of reward sparsity in the model can be solved. By reward shaping, the problem of credit assignment between agents can be solved. Based on the idea of meta-learning, an extensible multi-agent deep reinforcement learning (EMADRL) framework and solving method is proposed to improve the effectiveness and efficiency of model solving. Experiments show that the model meets the requirements well and the algorithm learning efficiency is high.
Improved adaptively robust estimation algorithm for GNSS spoofer considering continuous observation error
Keywords:Technological innovationAdaptation modelsTarget trackingSatellitesAutonomous aerial vehiclesReal-time systemsTrajectoryautonomous aerial vehiclesestimation theoryGlobal Positioning SystemKalman filtersremotely operated vehiclesrobust controlsatellite navigationstate estimationcontinuous observation errorerror convergencesteady-state linear quadratic estimatorUAV statusconvergence timeerror controlestimated trajectory errorbroadcast trajectory errorGNSS spooferadaptively robust estimation algorithmnormalized innovation squaredNISspoofingunmanned aerial vehicle (UAV)spooferadaptively robust estimationglobal navigation satellite system (GNSS)normalized innovation squared (NIS)
Abstracts:Once the spoofer has controlled the navigation system of unmanned aerial vehicle (UAV), it is hard to effectively control the error convergence to meet the threshold condition only by adjusting parameters of estimation if estimation of the spoofer on UAV has continuous observation error. Aiming at this problem, the influence of the spoofer's state estimation error on spoofing effect and error convergence conditions is theoretically analyzed, and an improved adaptively robust estimation algorithm suitable for steady-state linear quadratic estimator is proposed. It enables the spoofer's estimator to reliably estimate UAV status in real time, improves the robustness of the estimator in responding to observation errors, and accelerates the convergence time of error control. Simulation experiments show that the mean value of normalized innovation squared (NIS) is reduced by 88.5%, and the convergence time of NIS value is reduced by 76.3%, the convergence time of true trajectory error of UAV is reduced by 42.3%, the convergence time of estimated trajectory error of UAV is reduced by 67.4%, the convergence time of estimated trajectory error of the spoofer is reduced by 33.7%, and the convergence time of broadcast trajectory error of the spoofer is reduced by 54.8% when the improved algorithm is used. The improved algorithm can make UAV deviate from preset trajectory to spoofing trajectory more effectively and more subtly.
Design and simulation of the ATP system considering the advanced targeting angle in quantum positioning system
Keywords:Analytical modelsTarget trackingSatellitesSoftware packagesAzimuthOrbitsartificial satellitesGlobal Positioning Systemposition controlquantum theorytarget trackingATP systemadvanced targeting anglequantum positioning systemtracking systemadvanced targeting subsystemadvanced targeting azimuth anglepitch angleQPSacquisition tracking and pointing systemorbital parametersquantum satelliteMozidynamic tracking centerSimulinkdeviation compensationquantum positioning system (QPS)acquisitiontracking and pointing (ATP)advanced targeting
Abstracts:A compensation implementation scheme of the advanced targeting process based on the fine tracking system is proposed in this paper. Based on the working process of the quantum positioning system (QPS) and its acquisition, tracking and pointing (ATP) system, the advanced targeting subsystem of the ATP system is designed. Based on six orbital parameters of the quantum satellite Mozi, the advanced targeting azimuth angle and pitch angle are transformed into the dynamic tracking center of the fine tracking system in the ATP system. The deviation of the advanced targeting process is analyzed. In the Simulink, the simulation experiment of the ATP system considering the deviation compensation of the advanced targeting is carried out, and the results are analyzed.
Impact angle constrained fuzzy adaptive fault tolerant IGC method for Ski-to-Turn missiles with unsteady aerodynamics and multiple disturbances
Keywords:Fuzzy logicFault toleranceMissilesAdaptation modelsActuatorsUncertaintyFault tolerant systemsactuatorsadaptive controlaerodynamicscontrol nonlinearitiescontrol system synthesisfault tolerancefuzzy controlfuzzy logicmissile controlmissile guidancemissilesnonlinear control systemstrajectory controluncertain systemsfuzzy adaptive fault tolerant IGC methodSki-to-Turn missilesunsteady aerodynamicsmultiple disturbancesfuzzy adaptive fault tolerant integrated guidancecontrol methodactuator failuresmultisource model uncertaintiesmultisource uncertaintiesimpact angle constraintfuzzy logic systemsunknown nonlinearitiesunknown uncertaintiesspecified impact angleintegrated guidance and control (IGC)impact angle constraintunsteady aerodynamicsfault tolerant control (FTC)actuator failures
Abstracts:An impact angle constrained fuzzy adaptive fault tolerant integrated guidance and control method for Ski-to-Turn (STT) missiles subject to unsteady aerodynamics and multiple disturbances is proposed. Unsteady aerodynamics appears when flight vehicles are in a transonic state or confronted with unstable airflow. Meanwhile, actuator failures and multisource model uncertainties are introduced. However, the boundaries of these multisource uncertainties are assumed unknown. The target is assumed to execute high maneuver movement which is unknown to the missile. Furthermore, impact angle constraint puts forward higher requirements for the interception accuracy of the integrated guidance and control (IGC) method. The impact angle constraint and the precise interception are established as the object of the IGC method. Then, the boundaries of the lumped disturbances are estimated, and several fuzzy logic systems are introduced to compensate the unknown nonlinearities and uncertainties. Next, a series of adaptive laws are developed so that the undesirable effects arising from unsteady aerodynamics, actuator failures and unknown uncertainties could be suppressed. Consequently, an impact angle constrained fuzzy adaptive fault tolerant IGC method with three loops is constructed and a perfect hit-to-kill interception with specified impact angle can be implemented. Eventually, the numerical simulations are conducted to verify the effectiveness and superiority of the proposed method.
Maneuvering target state estimation based on separate modeling of target trajectory shape and dynamic characteristics
Keywords:Target trackingShapeHeuristic algorithmsSimulationRoadsDynamicsLength measurementradar trackingstate estimationtarget trackingtarget state estimationseparate modelingtarget trajectory shapedynamic characteristicsmaneuvering targetroad-targetsea-route-targetflight route-target trackingtarget velocity propertiesunknown target trajectorylatest estimated base stateconventional coupled model-based algorithmstarget maneuversmaneuvering target trackingseparate modelingnatural parametric functioninteracting multiple model (IMM) filterdata fittingstate augmentation
Abstracts:The state estimation of a maneuvering target, of which the trajectory shape is independent on dynamic characteristics, is studied. The conventional motion models in Cartesian coordinates imply that the trajectory of a target is completely determined by its dynamic characteristics. However, this is not true in the applications of road-target, sea-route-target or flight route-target tracking, where target trajectory shape is uncoupled with target velocity properties. In this paper, a new estimation algorithm based on separate modeling of target trajectory shape and dynamic characteristics is proposed. The trajectory of a target over a sliding window is described by a linear function of the arc length. To determine the unknown target trajectory, an augmented system is derived by denoting the unknown coefficients of the function as states in mileage coordinates. At every estimation cycle except the first one, the interaction (mixing) stage of the proposed algorithm starts from the latest estimated base state and a recalculated parameter vector, which is determined by the least squares (LS). Numerical experiments are conducted to assess the performance of the proposed algorithm. Simulation results show that the proposed algorithm can achieve better performance than the conventional coupled model-based algorithms in the presence of target maneuvers.
Hybrid Q-learning for data-based optimal control of non-linear switching system
Keywords:Q-learningSwitching systemsSystem dynamicsHeuristic algorithmsSimulationOptimal controlSwitchesiterative methodslearning (artificial intelligence)neurocontrollersnonlinear control systemsoptimal controloptimisationhybrid Q-learningdata-based optimal controlnonlinear switching systemsystem dynamicsHamilton-Jacobi-Bellman equationhybrid action spacenovel data-based hybrid Q-Iearning algorithmoptimal solutionconvergencelinear-in-parameter neural networksdata-driven methodswitching systemhybrid action spaceoptimal controlreinforcement learninghybrid Q-Iearning (HQL)
Abstracts:In this paper, the optimal control of non-linear switching system is investigated without knowing the system dynamics. First, the Hamilton-Jacobi-Bellman (HJB) equation is derived with the consideration of hybrid action space. Then, a novel data-based hybrid Q-Iearning (HQL) algorithm is proposed to find the optimal solution in an iterative manner. In addition, the theoretical analysis is provided to illustrate the convergence and optimality of the proposed algorithm. Finally, the algorithm is implemented with the actor-critic (AC) structure, and two linear-in-parameter neural networks are utilized to approximate the functions. Simulation results validate the effectiveness of the data-driven method.
Hierarchical reinforcement learning guidance with threat avoidance
Keywords:TrainingSimulationDecision makingLine-of-sight propagationReinforcement learningAerospace electronicsSystems engineering and theoryaerospace computingcontrol engineering computingdeep learning (artificial intelligence)gradient methodsmilitary computingmissile guidancereinforcement learningmissile operationguidance lawdeep reinforcement learninghierarchical deep deterministic policy gradient algorithmline-of-sight angle ratehierarchical reinforcement learning guidancethreat avoidancestriking effectreward functionDRLDDPGLOSaction penaltyguidance lawdeep reinforcement learning (DRL)threat avoidancehierarchical reinforcement learning
Abstracts:The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation. A novel guidance law is presented by exploiting the deep reinforcement learning (DRL) with the hierarchical deep deterministic policy gradient (DDPG) algorithm. The reward functions are constructed to minimize the line-of-sight (LOS) angle rate and avoid the threat caused by the opposed obstacles. To attenuate the chattering of the acceleration, a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward. The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.
Operational effectiveness evaluation based on the reduced conjunctive belief rule base
Keywords:Design methodologyDecision makingSystems engineering and theoryCognitionExplosionsFacesbelief networksinference mechanismsknowledge based systemsmilitary computingreduced conjunctive belief rule baserule premise combination explosionorthogonal design methodreasoning methodreduced conjunctive BRBoperational mission effectiveness evaluationoperational effectiveness evaluationreduced conjunctive belief rule base (BRB)orthogonal designevidence reasoning (ER)
Abstracts:To address the issue of rule premise combination explosion in the construction of the traditional complete conjunctive belief rule base (BRB), this paper introduces an orthogonal design method to reduce the conjunctive BRB. The reasoning method based on reduced conjunctive BRB is designed with the help of the conversion technology from conjunctive BRB to disjunctive BRB. Finally, the operational mission effectiveness evaluation is taken as an example to verify the proposed method. The results show that the method proposed in this paper is feasible and effective.
UAV safe route planning based on PSO-BAS algorithm
Keywords:Three-dimensional displaysCostsUncertaintySimulationUrban areasAutonomous aerial vehiclesSystems engineering and theoryautonomous aerial vehiclescollision avoidanceparticle swarm optimisationpath planningremotely operated vehiclesrisk managementsplines (mathematics)vehicle routingspatial 3D routediscrete pathflyable pathrisk valuepath costpath redundancyUAV performance constraintsUAV safe route planningPSO-BAS algorithmunmanned aerial vehiclessafe operationlow-altitude airspaceUAV route planning methodregional risk assessmentUAV operating characteristics3D risk mappath risk valueparticle swarm optimization-beetle antennae searchgenerated path redundancyunmanned aerial vehicle (UAV)low-attitude airspacemission planningrisk assessmentparticle swarm optimizationbeetle antennae search (BAS)cubic B-spline
Abstracts:In order to solve the current situation that unmanned aerial vehicles (UAVs) ignore safety indicators and cannot guarantee safe operation when operating in low-altitude airspace, a UAV route planning method that considers regional risk assessment is proposed. Firstly, the low-altitude airspace is discretized based on rasterization, and then the UAV operating characteristics and environmental characteristics are combined to quantify the risk value in the low-altitude airspace to obtain a 3D risk map. The path risk value is taken as the cost, the particle swarm optimization-beetle antennae search (PSO-BAS) algorithm is used to plan the spatial 3D route, and it effectively reduces the generated path redun dancy. Finally, cubic B-spline curve is used to smooth the plan ned discrete path. A flyable path with continuous curvature and pitch angle is generated. The simulation results show that the generated path can exchange for a path with a lower risk value at a lower path cost. At the same time, the path redundancy is low, and the curvature and pitch angle continuously change. It is a flyable path that meets the UAV performance constraints.
Scenario-oriented hybrid particle swarm optimization algorithm for robust economic dispatch of power system with wind power
Keywords:EconomicsAdaptation modelsAnnealingSimulated annealingWind power generationHybrid power systemsGeneratorsoptimisationparticle swarm optimisationpower generation dispatchpower generation economicspower generation schedulingsimulated annealingwind powerscenario-oriented hybrid particle swarm optimization algorithmpower systemwind powereconomic dispatch problemdiscrete scenariouncertain wind powersbad scenariobad-scenario-set robust economic dispatch modelspecialized hybrid particle swarm optimization algorithmhybridizing simulated annealing operatorsscenario-oriented adaptive search rulewind powerrobust economic dispatchscenariosimulated annealing (SA)particle swarm optimization (PSO)
Abstracts:An economic dispatch problem for power system with wind power is discussed. Using discrete scenario to describe uncertain wind powers, a threshold is given to identify bad scenario set. The bad-scenario-set robust economic dispatch model is established to minimize the total penalties on bad scenarios. A specialized hybrid particle swarm optimization (PSO) algorithm is developed through hybridizing simulated annealing (SA) operators. The SA operators are performed according to a scenario-oriented adaptive search rule in a neighborhood which is constructed based on the unit commitment constraints. Finally, an experiment is conducted. The computational results show that the developed algorithm outperforms the existing algorithms.