ANNOTATION

This manual introduces the basic conditions of optimality and methods for solving problems in the calculus of variations and optimal control. It will be useful for preparing and conducting practical classes in the section "Optimal control", as well as when students do homework on this topic.

Tutorial is an electronic version of the book:
Optimal control in examples and problems. Sotskov A.I., Kolesnik G.V. - M.: Russian Economic School, 2002 - 58 p.

Preface

1. The simplest problem in the calculus of variations.
Euler's equation
Examples
Exercises

2. Optimal control problem. Maximum principle
Examples
Exercises

3. Phase constraints in the optimal control problem
Examples
Exercises

4. Dynamic programming and the Bellman equation
Examples
Exercises

Literature

Preface

Optimal control theory is one of the sections of the course “Mathematics for Economists” taught at the Russian School of Economics.
Teaching experience shows that this section is one of the most difficult to master. This is primarily due to the conceptual differences between the optimal control problems studied in it and the finite-dimensional optimization problems, and, as a consequence, to the significant complication of the optimality conditions used in them.
In this regard, it seems useful to provide a clear illustration of the application of these optimality conditions to solving problems various types. This manual is an attempt to provide such an illustration. It contains examples and problems on four topics:
. calculus of variations;
. the maximum principle in problems without restrictions;
. the maximum principle in the presence of phase restrictions;
. dynamic programming.
Each section consists of a theoretical part describing the basic concepts and results used in solving the corresponding problems, examples with solutions, as well as problems for students’ independent work.
It should be emphasized that this manual is in no way a theoretical course, but is aimed primarily at practical application optimal control methods. As a theoretical guide to this section, we can recommend, for example, a book.
According to the authors, this manual will be useful for teachers when preparing and conducting practical classes on the “Optimal Control” section, as well as for students when doing homework on this topic.

Electronic version of the book: [Download, PDF, 633.8 KB].

To view the book in PDF format required Adobe program Acrobat Reader, new version which can be downloaded for free from the Adobe website.

In general, an automatic system consists of a control object and a set of devices that provide control of this object. As a rule, this set of devices includes measuring devices, amplifying and converting devices, as well as actuators. If we combine these devices into one link (control device), then the block diagram of the system looks like this:

IN automatic system information about the state of the controlled object is supplied to the input of the control device through the measuring device. Such systems are called feedback systems or closed systems. The absence of this information in the control algorithm indicates that the system is open. We will describe the state of the control object at any time variables
, which are called system coordinates or state variables. It is convenient to consider them as coordinates - dimensional state vector.

The measuring device provides information about the state of the object. If based on the vector measurement
the values ​​of all coordinates can be found
state vector
, then the system is said to be completely observable.

The control device generates a control action
. There can be several such control actions; they form - dimensional control vector.

The input of the control device receives a reference input
. This input action carries information about what the state of the object should be. The control object may be subject to a disturbing influence
, which represents a load or disturbance. Measuring the coordinates of an object is usually carried out with some errors
, which are also random.

The task of the control device is to develop such a control action
so that the quality of functioning of the automatic system as a whole would be the best in some sense.

We will consider control objects that are manageable. That is, the state vector can be changed as required by correspondingly changing the control vector. We will assume that the object is completely observable.

For example, the position of an aircraft is characterized by six state coordinates. This
- coordinates of the center of mass,
- Euler angles, which determine the orientation of the aircraft relative to the center of mass. The aircraft's attitude can be changed using elevators, heading, aileron and thrust vectoring. Thus the control vector is defined as follows:

- elevator deflection angle

- well

- aileron

- traction

State vector
in this case it is defined as follows:

You can pose the problem of selecting a control with the help of which the aircraft is transferred from a given initial state
to a given final state
with minimal fuel consumption or in minimal time.

Additional complexity in solving technical problems arises due to the fact that, as a rule, various restrictions are imposed on the control action and on the state coordinates of the control object.

There are restrictions on any angle of the elevators, yaws, and ailerons:

- traction itself is limited.

The state coordinates of the control object and their derivatives are also subject to restrictions that are associated with permissible overloads.

We will consider control objects that are described by the differential equation:

(1)

Or in vector form:

--dimensional vector of object state

--dimensional vector of control actions

- function of the right side of equation (1)

To the control vector
a restriction is imposed, we will assume that its values ​​belong to some closed region some -dimensional space. This means that the executive function
belongs to the region at any time (
).

So, for example, if the coordinates of the control function satisfy the inequalities:

then the area is -measured cube.

Let us call any piecewise continuous function an admissible control
, whose values ​​at each moment of time belongs to the region , and which may have discontinuities of the first kind. It turns out that even in some optimal control problems the solution can be obtained in the class of piecewise continuous control. To select control
as a function of time and initial state of the system
, which uniquely determines the movement of the control object, it is required that the system of equations (1) satisfy the conditions of the theorem of existence and uniqueness of the solution in the area
. This area contains possible trajectories of the object’s movement and possible control functions.
. If the domain of variation of variables is convex, then for the existence and uniqueness of a solution it is sufficient that the function

. were continuous in all arguments and had continuous partial derivatives with respect to variables

.

As a criterion that characterizes the quality of system operation, a functional of the form is selected:

(2)

As a function
we will assume that it is continuous in all its arguments and has continuous partial derivatives with respect to

.

Optimal automatic control systems are systems in which control is carried out in such a way that the required optimality criterion has an extreme value. Boundary conditions defining the initial and required final states of the system; the technological goal of the system. tн It is set in cases where the average deviation over a certain time interval is of particular interest and the task of the control system is to ensure a minimum of this integral...

Share your work on social networks

If this work does not suit you, at the bottom of the page there is a list of similar works. You can also use the search button

Optimal control

Voronov A.A., Titov V.K., Novogranov B.N. Fundamentals of the theory of automatic regulation and control. M.: Higher School, 1977. 519 p. P. 477 491.

Optimal self-propelled guns these are systems in which control is carried out in such a way that the required optimality criterion has an extreme value.

Examples of optimal object management:

1. Controlling the movement of a rocket in order to achieve a given height or range with minimal fuel consumption;
2. Controlling the movement of a mechanism driven by an engine, which would minimize energy costs;
3. Controlling a nuclear reactor for maximum performance.

The optimal control problem is formulated as follows:

“Find such a law of change in control time u(t ), in which the system, under given restrictions, will move from one given state to another in an optimal way in the sense that the functional I , expressing the quality of the process, will receive an extreme value “.

To solve the optimal control problem, you need to know:

1. Mathematical description of the object and environment, connecting the values ​​of all coordinates of the process under study, control and disturbing influences;

2. Physical restrictions on coordinates and control law, expressed mathematically;

3. Boundary conditions defining the initial and required final states of the system

(technological goal of the system);

4. Objective function (quality functional

mathematical goal).

Mathematically, the optimality criterion is most often presented as:

t to

I =∫ f o [ y (t), u (t), f (t), t ] dt + φ [ y (t to), t to ], (1)

t n

where the first term characterizes the quality of control over the entire interval ( tn, tn) and is called

integral component, second term

characterizes the accuracy at the final (terminal) point in time t to .

Expression (1) is called a functional, since I depends on the choice of function u(t ) and the resulting y(t).

Lagrange problem.It minimizes functionality

t to

I=∫f o dt.

t n

It is used in cases where the average deviation over time is of particular interest.

a certain time interval, and the task of the control system is to ensure a minimum of this integral (deterioration in product quality, loss, etc.).

Examples of functionality:

I =∫ (t) dt criterion for minimum error in steady state, where x(t)

1. deviation controlled parameter from the set value;

I =∫ dt = t 2 - t 1 = > min criterion for maximum speed of self-propelled guns;

I =∫ dt = > min criterion of optimal efficiency.

Mayer's problem. In this case, the functional that is minimized is the one defined only by the terminal part, i.e.

I = φ =>min.

For example, for an aircraft control system described by the equation

F o (x, u, t),

you can set the following task: determine the control u (t), t n ≤ t ≤ t k so that for

specified time flight to achieve maximum range, provided that at the final moment of time t to The aircraft will land, i.e. x (t to ) =0.

Boltz problem reduces to the problem of minimizing criterion (1).

The basic methods for solving optimal control problems are:

1.Classical calculus of variations Euler’s theorem and equation;

2. The principle of maximum L.S. Pontryagin;

3.Dynamic programming by R. Bellman.

EULER'S EQUATION AND THEOREM

Let the functionality be given:

t to

I =∫ f o dt ,

t n

Where some twice differentiable functions, among which it is necessary to find such functions ( t ) or extremals , which satisfy the given boundary conditions x i (t n), x i (t k ) and minimize functionality.

Extremals are found among solutions of the Euler equation

I = .

To establish the fact of minimizing the functional, it is necessary to make sure that the Lagrange conditions are satisfied along the extremals:

similar to the requirements for the positivity of the second derivative at the minimum point of the function.

Euler's theorem: “If the extremum of the functional I exists and is achieved among smooth curves, then it can only be achieved on extremals.”

L.S.PONTRYAGIN’S MAXIMUM PRINCIPLE

The school of L.S. Pontryagin formulated a theorem about the necessary condition of optimality, the essence of which is as follows.

Let us assume that the differential equation of the object, together with the unchangeable part of the control device, is given in the general form:

To control u j restrictions may be imposed, for example, in the form of inequalities:

, .

The purpose of control is to transfer the object from the initial state ( t n ) to the final state ( t to ). The end of the process t to may be fixed or free.

Let the optimality criterion be the minimum of the functional

I = dt.

Let's introduce auxiliary variables and form a function

Fo ()+ f () f ()+

The maximum principle states that for the system to be optimal, i.e. to obtain the minimum of the functional, it is necessary to exist such non-zero continuous functions satisfying the equation

That for any t , located in a given range t n≤ t ≤ t k , the value of H, as a function of admissible control, reaches a maximum.

The maximum of the function H is determined from the conditions:

if it does not reach the boundaries of the region, and as the supremum of the function H, otherwise.

Dynamic programming by R. Bellman

R. Bellman's principle of optimality:

“Optimal behavior has the property that, whatever the initial state and decision at the initial moment, subsequent decisions must constitute optimal behavior relative to the state resulting from the first decision.”

The “behavior” of the system should be understood movement these systems, and the term"decision" refers tothe choice of the law of change in time of control forces.

In dynamic programming, the process of searching for extremals is divided into n steps, while in the classical calculus of variations the search for the entire extremal is carried out.

The process of searching for an extremal is based on the following premises of R. Bellman’s principle of optimality:

1. Each segment of the optimal trajectory is itself an optimal trajectory;
2. The optimal process at each site does not depend on its history;
3. Optimal control (optimal trajectory) is sought using backward movement [from y (T) to y (T -∆), where ∆ = T/ N, N number of sections of the trajectory, etc.].

Heuristically, the Bellman equations for the required problem statements are derived for continuous and discrete systems.

Adaptive Control

Andrievsky B.R., Fradkov A.L. Selected chapters of automatic control theory with examples in language MATLAB . St. Petersburg: Nauka, 1999. 467 p. Chapter 12.

Voronov A.A., Titov V.K., Novogranov B.N. Fundamentals of the theory of automatic regulation and control. M.: Higher School, 1977. 519 p. P. 491 499.

Ankhimyuk V.L., Opeiko O.F., Mikheev N.N. Theory of automatic control. Mn.: Design PRO, 2000. 352 p. P. 328 340.

The need for adaptive control systems arises due to the significant complication of control problems being solved, and specific feature This complication lies in the lack of practical opportunity for a detailed study and description of the processes occurring in the controlled object.

For example, modern high-speed aircraft, accurate a priori data on the characteristics of which under all operating conditions cannot be obtained due to significant variations in atmospheric parameters, large ranges of flight speeds, ranges and altitudes, as well as due to the presence of a wide range of parametric and external disturbances.

Some control objects (airplanes and missiles, technological processes and power plants) are distinguished by the fact that their static and dynamic characteristics change over a wide range in a manner that was not anticipated in advance. Optimal management of such objects is possible with the help of systems in which the missing information is automatically replenished by the system itself during operation.

Adaptive (lat.) adaptio ” device) are those systems that, when changing the parameters of objects or the characteristics of external influences during operation, independently, without human intervention, change the parameters of the regulator, its structure, settings or regulatory influences to maintain the optimal operating mode of the object.

The creation of adaptive control systems is carried out in fundamentally different conditions, i.e. adaptive methods should help achieve high quality control in the absence of sufficient completeness of a priori information about the characteristics of the controlled process or under conditions of uncertainty.

Classification of adaptive systems:

Self-adapting

(adaptive)

Control systems

Self-adjusting Self-learning Systems with adaptation

Systems systems in special phases

States

Search Searchless- Training- Training- Relay Adaptive

(extreme (analysed with incentives without self-oscillation system with

New) tic incentive variables

Systems systems systems structure

Block diagram AS classifications (according to the nature of the adaptation process)

Self-tuning systems (SNS)are systems in which adaptation to changing operating conditions is carried out by changing parameters and control actions.

Self-organizingThese are systems in which adaptation is carried out by changing not only parameters and control actions, but also the structure.

Self-learningthis is an automatic control system in which the optimal operating mode of the controlled object is determined using a control device, the algorithm of which is automatically purposefully improved in the learning process by automatic search. The search is carried out using a second control device, which is an organic part of the self-learning system.

In search engines systems, changing the parameters of the control device or control action is carried out as a result of searching for conditions for the extremum of quality indicators. The search for extremum conditions in systems of this type is carried out using test influences and assessmentobtained results.

In non-search systems, the determination of the parameters of the control device or control actions is carried out on the basis of the analytical determination of the conditions that ensure the specified quality of control without the use of special search signals.

Systems with adaptation in special phase statesuse special modes or properties of nonlinear systems (self-oscillation modes, sliding modes) to organize controlled changes in the dynamic properties of the control system. Specially organized special modes in such systems either serve as an additional source of operational information about the changing operating conditions of the system, or endow the control systems with new properties, due to which the dynamic characteristics of the controlled process are maintained within the desired limits, regardless of the nature of the changes that arise during operation.

When using adaptive systems, the following main tasks are solved:

1 . During the operation of the control system, when parameters, structure and external influences change, control is provided in which the specified dynamic and static properties of the system are maintained;

2 . During the design and commissioning process, in the initial absence of complete information about the parameters, structure of the control object and external influences, automatic setup systems in accordance with specified dynamic and static properties.

Example 1 . Adaptive aircraft angular position stabilization system.

f 1 (t) f 2 (t) f 3 (t)

D1 D2 D3

VU1 VU2 VU3 f (t) f 1 (t) f 2 (t) f 3 (t)

u (t) W 1 (p) W 0 (p) y (t)

+ -

Rice. 1.

Adaptive aircraft stabilization system

When flight conditions change, it changes transfer function W 0 (p ) aircraft, and, consequently, the dynamic characteristics of the entire stabilization system:

. (1)

Disturbances from the external environment f 1 (t), f 2 (t), f 3 (t ) leading to controlled changes in system parameters are applied to various points of the object.

Disturbing influence f(t ) applied directly to the input of the control object, in contrast to f 1 (t), f 2 (t), f 3 (t ) does not change its parameters. Therefore, during system operation, only f 1 (t), f 2 (t), f 3 (t).

According to the principle feedback and expression (1) uncontrolled changes in characteristics W 0 (p ) due to disturbances and interference cause relatively small changes in the parameters Ф( p) .

If we set the task of more complete compensation of controlled changes, so that the transfer function Ф(р) of the aircraft stabilization system remains practically unchanged, then the characteristics of the controller should be changed appropriately W 1 (p ). This is done in an adaptable self-propelled gun, made according to the scheme in Fig. 1. Environmental parameters characterized by signals f 1 (t), f 2 (t), f 3 (t ), for example, velocity head pressure P H(t) , ambient temperature T0(t) and flight speed v(t) , are continuously measured by sensors D 1, D 2, D 3 , and the current parameter values ​​are sent to computing devices B 1, B 2, B 3 , producing signals with the help of which the characteristic is adjusted W 1 (p ) to compensate for changes in characteristics W0(p).

However, in ASAU of this type(with an open loop of adjustment) there is no self-analysis of the effectiveness of the controlled changes it makes.

Example 2. Extreme aircraft flight speed control system.

Z Disturbance

Impact

X 3 = X 0 - X 2

Automatic device X 0 Amplification X 4 Executive X 5 Adjustable X 1

Mathematical converter device object

Extremum iska + - device

Measuring

Device

Fig. 2. Functional diagram of an extreme aircraft flight speed control system

The extremal system determines the most profitable program, i.e. then the value X 1 (required aircraft speed), which is needed in at the moment maintain that a minimum fuel consumption per unit of path length is produced.

Z - characteristics of the object; X 0 - control influence on the system.

(fuel consumption value)

y(0)

y(T)

Self-organizing systems

These standards separately normalize each component of the microclimate in the working area of ​​the production premises: temperature, relative humidity, speed of air movement, depending on the ability of the human body to acclimatize at different times of the year, the nature of clothing, the intensity of the work performed and the nature of heat generation in the work area. Changes in air temperature in height and horizontally, as well as changes in air temperature during a shift, while ensuring optimal microclimate values ​​in the workplace, should not... Management: concept, features, system and principles Government bodies: concept, types and functions. In terms of content, administrative law is public administrative law that realizes the legal interest of the majority of citizens, for which the subjects of management are endowed with legally authoritative powers and representative functions of the state. Therefore, the object of action of legal norms are specific managerial social relations that arise between the subject of management by the manager and the objects... Government regulation socio-economic development of regions. Local budgets as the financial basis for the socio-economic development of the region. Different territories of Ukraine have their own characteristics and differences both in terms of economic development and in social, historical, linguistic and mental aspects. Of these problems, we must first of all mention the imperfection of the sectoral structure of most regional economic complexes; their low economic efficiency; significant differences between regions in levels...

To design an optimal automatic control system, complete information about the op-amp, disturbing and master influences, and the initial and final states of the op-amp is required. Next, you need to select an optimality criterion. One of the system quality indicators can be used as such a criterion. However, the requirements for individual quality indicators are usually contradictory (for example, increasing the accuracy of the system is achieved by reducing the stability margin). In addition, the optimal system should have a minimum possible error not only when executing a specific control action, but throughout the entire operating time of the system. It should also be taken into account that the solution to the optimal control problem depends not only on the structure of the system, but also on the parameters of its constituent elements.

Achieving optimal functioning of the ACS is largely determined by how control is carried out over time, what the program is, or control algorithm. In this regard, to assess the optimality of systems, integral criteria are used, calculated as the sum of the values ​​of the system quality parameter of interest to designers for the entire time of the control process.

Depending on the adopted optimality criterion, the following types of optimal systems are considered.

1. Systems, optimal for performance, which provide the minimum time for transferring the op-amp from one state to another. In this case, the optimality criterion looks like this:

where / n and / k are the moments of the beginning and end of the control process.

In such systems, the duration of the control process is minimal. The simplest example- an engine control system that ensures the minimum time for acceleration to a given speed, taking into account all existing restrictions.

2. Systems, optimal in terms of resource consumption, which guarantee the minimum criterion

Where To- proportionality coefficient; U(t)- control action.

Such an engine management system ensures, for example, minimum fuel consumption during the entire control period.

3. Systems, optimal in terms of control losses(or accuracy), which provide minimal control errors based on the criterion where e(f) is the dynamic error.

In principle, the problem of designing an optimal automatic control system can be solved by the simplest method of enumerating all possible options. Of course, this method requires a lot of time, but modern computers allow you to use it in some cases. To solve optimization problems, special methods of the calculus of variations have been developed (maximum method, dynamic programming method, etc.), which make it possible to take into account all the limitations of real systems.

As an example, let us consider what the optimal speed control of a DC electric motor should be if the voltage supplied to it is limited by the limit value (/lr, and the motor itself can be represented as a 2nd order aperiodic link (Fig. 13.9, A).

The maximum method allows you to calculate the law of change u(d), ensuring the minimum time for engine acceleration to rotation speed (Fig. 13.9, b). The control process of this motor must consist of two intervals, in each of which the voltage u(t) takes its maximum permissible value (in the interval 0 - /,: u(t)= +?/ ex, in the interval /| - / 2: u(t)= -?/ pr)* To ensure such control, a relay element must be included in the system.

Like conventional systems, optimal systems are open-loop, closed-loop and combined. If the optimal control that transfers the op-amp from the initial state to the final state and is independent or weakly dependent on disturbing influences can be specified as a function of time U= (/(/), then we build open-loop system program control (Fig. 13.10, A).

The optimal program P, designed to achieve the extremum of the accepted optimality criterion, is embedded in the PU software device. According to this scheme, management is carried out

Rice. 13.9.

A- with a common control device; b - with two-level controller

device

Rice. 13.10. Schemes of optimal systems: A- open; b- combined

CNC machine tools program controlled and the simplest robots, rockets are launched into orbit, etc.

The most advanced, although also the most complex, are combined optimal systems(Fig. 13.10, b). In such systems, the open loop provides optimal control according to given program, and a closed loop, optimized to minimize errors, processes the deviation of the output parameters. Using the disturbance measurement rope /*, the system becomes invariant with respect to the entire set of driving and disturbing influences.

In order to implement such a perfect control system, it is necessary to accurately and quickly measure all disturbing influences. However, this possibility is not always available. Much more often, only averaged statistical data are known about disturbing influences. In many cases, especially in telecontrol systems, even the driving force enters the system along with noise. And since the interference is, in the general case, random process, then it is possible to synthesize only statistically optimal system. Such a system will not be optimal for each specific implementation of the control process, but it will be on average the best for the entire set of its implementations.

For statistically optimal systems, averaged probabilistic estimates are used as optimality criteria. For example, for a tracking system optimized for a minimum error, the mathematical expectation of the squared deviation of the output effect from the specified value is used as a statistical criterion for optimality, i.e. variance:

Other probabilistic criteria are also used. For example, in a target detection system, where only the presence or absence of a target is important, the probability of an erroneous decision is used as an optimality criterion Rosh:

Where R p ts is the probability of missing the target; R LO- probability of false detection.

In many cases, the calculated optimal automatic control systems turn out to be practically impossible to implement due to their complexity. As a rule, it is required to obtain accurate values ​​of high-order derivatives from input influences, which is technically very difficult to achieve. Often, even a theoretical exact synthesis of an optimal system is impossible. However, optimal design methods make it possible to build quasi-optimal systems, although simplified to one degree or another, but still allowing one to achieve values ​​of the accepted optimality criteria that are close to extreme.

In recent years, optimal management has begun to be used both in technical systems to improve the efficiency of production processes, and in organizational management systems to improve the activities of enterprises, organizations, and sectors of the national economy.

In organizational systems, one is usually interested in the final, established result of the team, without exploring

efficiency during the transition process between issuing a command and obtaining the final result. This is explained by the fact that usually in such systems the losses in the transition process are quite small and do not significantly affect the overall gain in the steady state, since the steady state itself is much longer than the transition process. But sometimes dynamics are not studied due to mathematical difficulties. Courses of methods are devoted to methods for optimizing final states in organizational and economic systems. optimization and operations research.

In dynamic management technical systems optimization is often essential precisely for transient processes, in which the efficiency indicator depends not only on the current values ​​of the coordinates (as in extreme control), but also on the nature of the change in the past, present and future, and is expressed by some functional on the coordinates, their derivatives and, maybe be, time.

An example is the control of an athlete running over a distance. Since his energy reserve is limited by physiological factors, and the consumption of the reserve depends on the nature of the run, the athlete can no longer give the maximum possible power at each moment, so as not to use up the energy reserve prematurely and not run out of energy over the distance, but must look for the optimal running mode for his characteristics .

Finding optimal control in such dynamic problems requires solving a rather complex mathematical problem in the control process using the methods of calculus of variations or mathematical programming, depending on the type mathematical description (mathematical model) systems. Thus, the calculating device or computer becomes an organic component of the optimal control system. The principle is explained in Fig. 1.10. The input of the computing device (machine) VM receives information about the current values ​​of coordinates x from the output of the object O, about controls and from its input, about external influences z on the object, as well as setting various conditions from the outside: the value of the optimality criterion boundary conditions information about valid values ​​Computational

How to work