Abstract. This document provides a summary of the research needs in the area of dynamics and control for use in uninhabited aerial vehicles (UAVs). It is based on the conclusions of a AFOSR sponsored meeting in August 1997 chaired by Gunter Stein. A list of research needs is presented along with specific recommendations.

The use of uninhabited aerial vehicles (UAVs) for surveillance and combat operations is likely to increase rapidly in the next 10-20 years as the performance requirements for military aircraft go beyond the capabilities of human pilots and the costs of manned flight become prohibitive. UAVs help eliminate these bottlenecks by allowing the pilot to remain on the ground (or in another aircraft), controlling the UAV remotely. This approach represents an enormous challenge to control theory since many of the functions of a pilot will now have to be taken over by control systems. In addition, UAVs will be more maneuverable and more modular, necessitating improved tools for control law design.

The Air Force Scientific Advisory Board (SAB) prepared a report on "UAV Technologies and Combat Operations" in November of 1996 (SAB-TR-96-01). The SAB concluded that

Thus, in addition to challenges in traditional areas of flight control, new technical challenges will be introduced in areas of man-machine systems, vehicle management systems, distributed control across multiple-platforms, and uncertainty and complexity management.

In this report, we summarize some of the important basic research issues that must be addressed in order to support the development and deployment of advanced UAV systems. Some of these issues are central areas of research in the controls community while others are at the edge of what the controls community currently considers as its core competencies. It is these new areas that the controls community must incorporate if it is to have an impact on UAV systems.

In developing a strategy for long term research, it is important to identify the aspects of the problem that are likely to remain fixed for long periods of time. Identification or specification of these "fixed points" is essential for effective development of a research strategy.

In the context of UAVs, it was felt that the notion of a single, multi-purpose UAV platform was likely not to be a fixed point. UAVs of the future will be more modular and will undergo more rapid design innovations than current aircraft. This increased modularity and flexibility is enabled, in part, by the fact that the pilot is no longer physically inhabiting the aircraft and hence extensive training on a single, fixed physical platform will not be required. Many of the existing control paradigms---such as painstaking development of control laws for each configuration of an aircraft---will no longer be appropriate or economically feasible.

One likely fixed point for UAVs is the hierarchical decomposition of control functions depicted below.

This hierarchy separates the control problem into three basic levels:

• Inner Loop Control: the traditional low level functions that control surfaces and engines based on the desired motion of the aircraft.
• Vehicle Management: responsible for overall coordination of the vehicle during the mission, including generating commands for the inner loops, managing vehicle configuration and operating modes, and performing vehicle health and failure monitoring.
• Multi-Vehicle Mission Management: the highest level control function, responsible for controlling ingress and egress trajectories for all vehicles, managing time lines, trajectory patterns and attack sequences, replanning missions in response to attrition, and evaluation of mission success.

While the functionality within each of these levels is expected to change rapidly as technology advances, this basic hierarchy will remain in place and will define the overall structure of controller interaction. New paradigms will be needed for control in each of these sublayers as well as interaction and communication between the sublayers.

A second fixed point will be the increased reliance on modeling and simulation in designing, testing, and evaluating UAVs. Since UAVs dramatically open the range of design possibilities from aircraft structures through hierarchical command and control, they are likely to interact with this emerging trends more broadly and profoundly than any other parts of military technology.

The following areas have been identified as important areas for increased basic research effort.

This represents a core competency of the controls community applied to the lowest level "inner loop flight control" layer of UAVs. It includes:

• Reusable control law structures/processes capable of full-envelope design
• Software tools for automated control design and analysis
• Auto-code generation from high-level design tools
• Simulation models of sufficient fidelity to demonstrate the performance of the inner-loop controller when incorporated into the hierarchical UAV control system

UAVs of the future will likely undergo frequent configuration and mission changes. This will require the ability to rapidly design, redesign and implement inner-loop control laws; however, most of the current design techniques (especially for nonlinear systems) are not up to this task. Therefore, researchers need to develop automated design tools to facilitate the use of recent and new control design techniques. In order to accommodate the rapid prototyping that will be required for UAVs, these tools should take full advantage of the auto-code generation capabilities of many of the higher level programming languages that controls researchers currently use. The spirit of this effort is to develop tools that automate the entire control design process from model development to hardware-in-the-loop simulation.

Man-machine interfaces will play a central role in the performance of UAVs and these have been identified by the SAB and others as an area that requires increased emphasis. Research issues related to dynamics and control include:

• Quantification and analysis of autonomy

• Development of mathematical framework for autonomous decisions

• Identification/quantification of the classes of uncertain events that UAVs will encounter

• Development of metrics measuring a systems ability to maintain operability in the face of uncertain events in the allowable classes
• Development of algorithms that can analyze autonomy using the proposed metrics within the defined classes of uncertain events.

• Tailoring dynamics of autonomous systems for human interaction

• Given that autonomous systems make decisions independent of human action, what behavioral/dynamical properties should the UAV present to the human operator?
• When is the human allowed to override or take over decisions?

• Handling qualities of UAVs

• Characterization of human operator performance for RPVs and UAVs
• Determining potential adverse interactions between humans and automation
• Developing design guidelines for UAVs and UAV interfaces (displays etc.) using systems and control metrics.

In order for UAVs to be effective elements of combat operations, functionality that could be accomplished remotely by humans (at the cost of high work load) must be computerized. The character of the computerized tasks will evolve from automatic execution of high level commands to autonomous decision making to meet high level goals in a dynamic environment of threats, countermeasures, and information from other vehicles. This evolution will necessitate tools for evaluation of the robustness of autonomous systems. Robustness theory has reached a high level of maturity in control theory, and in the context of autonomy, can be used to provide guarantees on the size and type of events that an autonomous system can negotiate successfully.

Robustness in control theory is the study of a feedback controller’s ability to maintain stability and performance when faced with uncertainty in, for instance, the dynamics of the system being controlled. Analysis and synthesis of autonomous systems can be cast in the same theoretical framework by conceptualizing autonomy as an extension of robustness: a system’s autonomy is its ability to meet high-level goals and make correct decisions when unforeseen (i.e. uncertain) events occur. The mathematical problems encountered in robust control, although very complex, were solved using efficient approximation algorithms. Similar algorithms must be developed to assess autonomy and its effectiveness.

Mathematical treatment of autonomy, leveraged by expertise in dynamics and control, may also impact human-machine issues in UAVs. An autonomous aircraft is a new dynamical system, with behavior that can be tailored (within physical limits) to interface smoothly with a remote operator’s approach to vehicle command and control. This tailored behavior will be unlike that of a typical dynamical system because the autonomous UAV makes decisions based on information that is not available to the operator, or that is too low-level or high-bandwidth for the operator to consider.

Unpredictable behavior, or uncertainty, associated with autonomous decisions must be systematically mitigated to avoid adverse interaction with the human agent. Dynamical systems theory will help insure that UAV dynamics, displays, and operator interfaces are compatible and do not lead to human-machine limit cycles, or loss of situation awareness. In the latter context, data fusion methods can be used to generate vehicle state and condition information, and convey these in a helpful way.

Historically, the interaction between aircraft and their pilots has been studied in the context of dynamics and control. The conditions leading to pilot induced oscillations, as well as handling qualities metrics, have been quantified using the language and tools of systems and control theory. More recently, the area of humans and automation has become a discipline of its own, concerned primarily with display design, situation awareness, management of air traffic, design and evaluation of complex automatic flight systems, biomechanics, ergonomics, and psychometrics. The design of autonomous systems, which are fundamentally different than automatic systems, for reliable interaction with humans is a new field, which will require the combined efforts of experts in dynamics and control as well as humans and automation. Mathematical characterization of autonomy, a necessary step if analysis and eventually synthesis of these systems is to occur, will enable research in human interaction with autonomy.

This is a key requirement of the next higher "vehicle management" layer of UAVs, not well addressed by core control concepts nor by core computer science concepts. It includes:

• Discrete event decisions managing the vehicle and all its on-board systems autonomously from power-up though taxi, take-off, ingress, mission phase flight, egress, landing, and return to the ground support facility.
• Contingency management as the status of on-board systems, mission parameters, or external environments change

The engineering state-of-the-art for this functionality is largely heuristic. Extensive time is spent examining nominal operations and their contingencies, determining appropriate vehicle manager actions, and then programming those actions as if-then-else rules in vehicle management computers. Software/computer science techniques are applied to improve the programming aspects of this process (e.g. expert systems, formal logic, verification proofs, ) but the initial specifications for actions remain in responsibility of "domain experts". Research is needed to devise ways to formalize the generation of actions on the basis on underlying continuous dynamics of vehicles (synthesis), and also to devise ways to verify these actions such that all contingencies are covered and no undesired properties appear in any possible combinations of states (analysis). As in standard control theory, analysis improvements will probably precede synthesis. Examples of efforts to formalize these design steps can be found in the work on Intelligent Vehicle Highway Systems (IVHS).

This is a core competency of the controls community applied to a critical aspect of the next higher "mission management" level of UAVs. Issues that should be addressed by these algorithms include:

• Effect of vehicle attitude and trajectory on radar cross section and susceptibility to jamming
• Constraints on trajectory due to vehicle dynamics
• Stationary threats (e.g. radar, jammer, etc.)
• Variable number of dynamic threats (e.g. mobile radar/jammer, etc.)
• Collision avoidance
• Vehicle and threat modeling
• Computational requirements

As part of the autonomous decision making process that will be inherent to UAVs, algorithms for path generation need to be developed. These algorithms will have to account for a number of stationary and moving threats such as radar, jammers, and aircraft. Additionally, under some scenarios, the algorithms will need to exploit the dependence of the vehicle’s radar cross section and communication antenna gain on the vehicle’s orientation in order to obtain a feasible path. Furthermore, to provide utility in dynamic environments, these tools must be sufficiently simple for real-time implementation. The computational burden of these path-finding techniques must be such that the desired trajectory can be updated (autonomously) many times during a mission. Current robot motion planning systems and existing optimization-based route planners provide the first step; however, more sophistication will be required, both in the modeling of the obstacles (e.g., radar, jammers, etc.) and in accounting for the constraints imposed on feasible trajectories by the aircraft dynamics.

From the mission systems viewpoint, one of the most important findings of SAB was that in most operational tasks, UAVs frequently should be employed in coordinated clusters rather than as independent platforms. This scenario can be described as a dynamic network where each node is a UAV.. A dynamic network is characterized by a spatially distributed set of dynamic nodes which are coordinated (or integrated) by the mission objectives and possible dynamic coupling between the nodes. The mission objectives are to be obtained in the presence of large uncertainties due largely to a hostile environment. Within this context, nodes may fail at various levels, measurements may be highly corrupted and communication channels may be severely limited due to jamming. Communication links are further challenged due to power constraints and spatial dispersion producing tradeoffs between noisy information, latency and bandwidth constraints. For this class of problems current mathematical paradigms break down and focused research is required for new paradigms

Since the control of dynamic networks is essentially in its infancy, general procedures based on computational algorithms used for implementation is a long term goal of any well conceived research program. However, near term goals can be posed which represent important elements of the more complete problem:

• Distributed Control Architectures. The essence of determining the architecture for a decentralized system is to minimally degrade the dynamic network performance, assuming that all information is available to a centralized processor, while obtaining the advantages of decentralization. This process is intimately involved with the determination of the value of information on system performance and stability in the vehicle network.
• Disturbance Propagation and String Stability. For systems which require very tight formation of the nodes to obtain performance benefits, the propagation of disturbance through the formation, if amplified, can produce serious loss of performance and possibly the collapse of the formation. Resolution of this stability issue in spatially distributed systems, called string stability, is challenging, especially as the information distributed in the network is reduced or latent.
• Failure Modes and Health Monitoring. An important issue in the development of self-organizing dynamic networks is the resolution of the controller when a subsystem or a node has failed. Self-organizing dynamic networks can be characterized by the integration of health monitoring redundancy management and reconfiguration of the spatially distributed system. A systematic methodology is required based on the management of the redundant information in the network to detect, identify and circumvent faults at a particular node.
• Control Over Noisy Communication Channels. Further complexity is introduced if additional noise is introduced within the data communication channel between nodes. The effect of channel noise on decentralized control is an important issue when operating in a hostile jamming environment.

A consequence of cooperative missions is that UAVs require robust, high-performance data networks for information exchange. This need drives a new research direction in the theory of the control of dynamic networks. This new theory should be molded by the mission goals and the possible composition and capabilities of the UAVs. Mission objectives may differ where threat warning, jamming, reconnaissance, surveillance, tactical and strategic functions are performed. Although no single theory will evolve to embrace the totality of system complexity, a theory should emerge which provides guidance to the designer’s intuition, proposes new decision architectures, and ultimately constructs nearly-optimal decision rules that coherently coordinate the multiple platforms. In this way the enormous leverage provided by autonomous dynamic networks for system performance and affordability can be realized.

Uncertainty management is a fundamental concept in analysis and design of complex systems, and is poorly understood by the modeling and simulation community. Research issues include:

• Propagating uncertainty in models from component materials and geometry through system performance, cost, and risk.
• Designing complex systems to operate in the presence of significant uncertainties in the environment as well as uncertainties in system components, using concepts such as averaging, protocols, and feedback.
• Using model-based assessments of sensitivities to reduce risk by guiding cost-effective use of selected physical prototyping to augment virtual prototyping for those system components whose uncertainty descriptions are most critical.

Note that traditional design involves a tight feedback loop around testing and redesign. Like any well-designed high-gain feedback loop, this allowed the sub-processes of model, design, and build to be relatively sloppy and still lead to an overall robust process, but the high expense leads us to perform the same cycle in a virtual environment. The "error" in this cartoon is meant to capture all the design objectives including cost and performance.

A naive presumption in the new paradigm is that reducing the virtual error, which the virtual design loop will do, will also reduce the real error. The many sources of real error, from the intrinsic variability of the reality being modeled to the assumptions and approximations that are introduced throughout the modeling and simulation process, are handled ad-hoc and implicitly. This sloppiness in modeling was tolerated within the old paradigm because of the robustness provided by the tight, but expensive, feedback with reality. Thus a critical need in the new virtual paradigm is for systematic and explicit methods to represent and propagate the uncertainty throughout the modeling and design process. This is a major challenge. There are, however, incremental gains to be achieved, and we hope it will not take major failures and disasters to ensure the acceptance of the real paradigm shift needed.

The emergence of UAVs represents a significant opportunity for controls research to have an impact on Air Force operations. It is important that the controls community respond to this opportunity and that the funding agencies recognize the role that controls can play in developing and deploying UAVs.

AFOSR should:

• Continue to provide support for traditional control areas such as aggressive inner loop control, automated design procedures, real-time trajectory planning and optimization, and control of distributed systems.
• Establish new programs in networked control systems for tightly coupled, multi-aircraft systems.
• Establish new programs in control of man-machine systems, perhaps in conjunction with the human factors community.

It is also important for the control community to get involved in demonstrations of key research results and capabilities.

The Air Force is currently flight testing a number of UAVs, and it is anticipated that this number will continue to increase. These flight tests provide a unique opportunity for the control community to transition its research to operational systems. However, in order to be considered a serious player, the control community must demonstrate what it brings to UAVs. To do this, AFOSR in conjunction with the rest of the Air Force Research Laboratory needs to develop well-planned technology demonstration programs that utilize Small Business Innovative Research (SBIR), Small Business Technology Transfer (STTR), 6.2, and 6.3 programs. To gain momentum, these programs should start by demonstrating on UAVs techniques that we currently do well on manned aircraft (e.g., inner-loop control). As the programs progress, they should focus on new fields associated with the UAV problem such as advanced autopilots, real-time trajectory generation, flight management, and vehicle management.

The authors would like to thank Lt. Robert Behnken, Lt. Paul Blue and Phil Chandler for participating in the workshop and contributing portions of this document. Jim Buffington and Andrew Sparks also attended the workshop and provided comments and feedback on this document.