AutomAtic Assessment methods of A pilot‘s flight in the trAffic circuit

With an increasing number of aircraft, navigation requirements and fast development of avionics systems, as well as the need for the integration of UAVs into airspace, pilots are facing new challenges that need to be mitigated by the development of additional training and assessment systems. These measures would both assess and encourage student pilots to improve their performance by using predetermined flight accuracy requirements based on established flight corridors. In this paper, automatic assessment methods to be used for pilot flight assessment while flying in the circuit are analysed.


Introduction
The rapid development of aviation systems generates the need to design automatic pilot flight assessment systems which would aid and simplify the work of pilot-instructors. Since it is difficult to guarantee the uniformity of assessment, the same student may obtain different marks from different instructors. "Each pilot action should be assessed according to existing regulations that determine flight parameter monitoring methods" (Gruszecki 2007). Therefore, this paper aims to experimentally test and provide some suggestions for automatic assessment methods of student-pilot flights in the traffic circuit by using real flight data.

Flights in the traffic circuit
The traffic circuit consists of 5 parts (legs) (Fig. 1): 1. upwind; 2. crosswind; 3. downwind; 4. base; 5. final.  (Žilienė 2008) When performing flights in the traffic circuit, usually, all flight stages are completed. According to established procedures, the following stages of a flight are distinguished (Savičienė 2012;ICAO 2009): -taxi from apron to holding point; -take-off and climb according to ATC clearances and instructions to the chosen route; -flight en route according to clearances and instructions of the ATC; -descent to the approach; -approach and missed approach; -landing and taxi to apron. It is evident by analysing training flights completed by our students (Fig. 2) and typical North-South flight circuits provided by Airservices Australia (Fig. 3) (McConnell 2011) that flights in the traffic circuit are scattered and do not follow specifically predetermined flight trajectories.
Sometimes, due to lack of time, or in order to minimise fuel consumption, instructors may allow shortening some stages of the flight and focusing on the most important elements only, e.g. in the flight traffic circuit, take-off and landing are the fundamental elements. Therefore, as seen in the flight trajectories in Figure 3 (purple lines), after climbing to a certain level (according to Fig.1) a turn in one point is carried out (1) and, with the exclusion of stages (2) and (3), the flight is continued directly to stage (4) by joining the base leg and completing the rest of the legs.
According to interviews of flight instructors, the trajectory is shortened in this way because the downwind does not require a lot of work from the part of the pilot, since he only needs to maintain the flight parameters stable, i.e. retain a stable altitude, speed, course. Whereas in all other parts of the circuit, it is also important to additionally monitor vertical speed of climb and descent, position of flaps and landing gear, etc. For the downwind to become more important and an advanced pilot to be trained for en route flights more, it is suggested to include two additional points in which the student should change the flight altitude. This would encourage maintaining an accurate traffic circuit and allow for a clearer trajectory according to which an automatic assessment system could assess the flight.

Model of pilot traffic circuit flight assessment
Pilot flight assessment is a multi-parameter task. During the flight, the pilot must complete four fundamental tasks (Lacabanne 2012): -operate: maintain aircraft in the flight trajectory; -navigate: guide the aircraft from the point of departure to the destination; -communicate: provide data, ask for and follow instructions, and give/obtain information; -monitor: the supplies available (fuel, temperature, oil, etc.). This paper reviews a traffic circuit model designed using MATLAB software, based on the desgin presented in Figure 4, published in (Masiulionis, Stankūnas 2013). According to this model, out of the 4 tasks presented, only the aircraft's operation and navigation can be assessed. Other piloting skills should be assessed by the instructor or more complex systems should be designed that would be able to assess the pilot's actions in the cockpit.
The designed traffic circuit assessment model is universal (Fig. 4). The model is entirely adjustable and is, therefore, universal: different traffic circuit parameters can be set and the assessment corridors can be projected onto any RW (runway) by adding RW center coordinates.  Other RW parameters: length, width and course are also pre-set. Afterwards, the width of all parts of the assessment corridors is included (inner, middle and external corridors) as well as the length of the assessment circuit (2 parameters): length of the upwind and final (x1 and x2). The distances of these parameters differ (usually the real upwind distance from the RW center is shorter, whereas the final is longer). Also, the width of the assessment circuit is determined (2 parameters): y1 -initial width (determines the length of radius R of all assessment circuit turns) and y2 -secondary width, which determines the main width of the flight assessment circuit.
In order to simplify calculations, the whole circuit is programmed according to Cartesian coordinates, whereas real flight GNSS (Global navigation satellite system) data based geographical coordinates (WGS-84) are converted into Cartesian coordinates, so that it would be possible to project real flight data and the designed assessment circuit in the same place. The first variant of the traffic circuit model was designed by converting real flight data into the ECEF (Earth-centered Earth-fixed) coordinates (Fig. 5). Since these coordinates are calculated from the Earth's center, the x and y axis coordinates are represented in very high numbers, which are cumbersome for the assessment of flight distances. Furthermore, after the conversion of the coordinates, the whole flight trajectory shifts 180° anticlockwise and does not match the real direction of the RW. This does not have an impact on the designed flight assessment model, since by changing its angle, it can be adjusted to the required flight trajectory.
In order to avoid the high numbers of the coordinates and the wrong direction of real flights, it was decided to use another coordinate system. The ENU (East-North-Up) coordinate system ( Fig. 6) was chosen, which solved the previously described problems. In these coordinates a predetermined geographical ground point becomes the reference point for the ENU coordinate system (x 0 , y 0 , z 0 ). In our case, in the circuit flight assessment model, the reference point is the center of the RW (Fig. 7), which makes it easier to calculate flight assessment circuit distances further from the aerodrome. The coordinates from the RW center are shown as Cartesian coordinates. Figure 7 also shows the designed visual holding zones (Z1, Z2, Z3 and P (additional)), also with three levels of assessment.
The designed initial (0°) direction of the assessment circuit is horizontal (Fig. 8). In order to adjust it to any aerodrome's RW course, it can be rotated to match any angle. The coordinates are rotated around a chosen point (in our case the coordinates of the RW center), all other points of the assessment circuit are then rotated accordingly.
The following equations are used to rotate the coordinates of the main points of the assessment circuit (1): where x rot , y rot indicate rotated (recalculated) point coordinates; x c , y c -center point coordinates according to which other points are rotated; x, y -rotated point initial position coordinates; rad -rotation angle in radians. sin , x r rad x y r rad y where x, y -arc point coordinates; x c , y c -arc center coordinates; r -arc radius; rad -arc angle in radians. With a rotated traffic circuit, traffic circuit assessment arcs need to be rotated by the same angle. According to equation system (1), the initial angle of the arc and arc initial point coordinates are determined. Since turn arcs form steep angles, the final angle of the arc is determined automatically: initial arc angle + 90°, and the final point coordinates of the arc are obtained. Finally, all these main points are joined by straight lines.
In the programming code, the flight traffic assessment circuit can be projected onto a map by using the wmline function (Fig. 9), which allows visualizing the flight using objects on the ground: forests, towns, roads, railroads, rivers, etc.
The deviations mentioned above could only be compared according to the assessment corridors. It would be more difficult to determine accurate deviations, especially when a real flight moves outside the determined zones of the assessment corridors. In order to measure deviations more accurately, the D'Errico (2013) devised Matlab deviation measurement library was used, which, according to set points, determines the closest distances to a defined figure or line. In this case, the central (ideal) trajectory line of the flight assessment circuit was determined and the results of across track errors were plotted, as illustrated in Figures 10 and 11. The red lines in Figure 11 correspond to the inner, middle and outer corridor deviation limits from the ideal trajectory.
Of course, in Figure 11, it is difficult to determine in which parts of the circuit the greatest deviations occurred; so Figures 10 and 11 complement each other. In the future, the flight stages will be segmented, for a better view of flight deviation locations.
This way the problem of measuring across track errors in the outer part of the traffic circuit is solved. However, in the inner part of the traffic circuit deviation measurement inaccuracies and errors may occur when the aircraft deviates too much from the assigned trajectory. The first inaccuracies occur in the turns when the aircraft flies too far from the assigned flight trajectory on the inner side of the traffic circuit. Since the deviation is measured according to the shortest distance, when the aircraft is too far from the assigned trajectory in the turn, the closest point is found not on the turn arc but on the straight lines of the circuit. This is evident in Fig.  10 where the blue deviation lines from the traffic circuit downwind straight trajectory suddenly move to the base straight trajectory lines and the deviations in the base turn are not measured.
Other inaccuracies occur when the pilot varies the distance of the traffic circuit during the training flights in the circuit. In this case, the measurement points are moved to the traffic circuit leg on the opposite direction circuit on the other side, e.g. as in Figure 10, the points move from the downwind to final. In Figure 11, these movements are indicated by the high amplitudes in the graph.
In order to avoid such inaccuracies, the highest permissible error limits have to be set for error measurement which, if exceeded, would generate a negative pilot flight assessment. Another way would be to not permit the pilot to change the traffic circuit distance during flight, so that these measurement point jumps would not occur.
Apart from the flight trajectory deviation assessment in the flight traffic circuit, an analysis of flight distance from the center of the RW was carried out (Fig. 12). Fig. 13 illustrates the results of distance measurement, in which it is visible that during a complete circuit, two distance maximums are achieved -the first distance maximum is reached in the turn to the downwind, and the second in the turn to the base leg. Between these distance maximums a parable form graph is obtained. If the aircraft's flight upwind is shorter than the final, the first distance maximum is lower than the second.
According to the form and amplitude of the described distance maximum, it can be determined whether the upwind or the final was longer. Also, after additional consideration of the amplitude of the graphs, the uniformity of completed flight circuits (whether the pilot is able to repeat the performed flight circuits) can be observed.

Pilot flight circuit altitude assessment
The assessment of pilot flight in the traffic circuit requires attention not only to horizontal deviations, but also consideration of how well the pilot is able to maintain a defined altitude, which is done by assessing the aircraft's vertical deviations. Figure 14 shows the altitude graph of the flight traffic circuit of a real flight illustrated in Figure 9.
The altitude graph must include the altitude of the RW (in Fig. 14, the RW altitude is illustrated in a straight red line), so that the altitudes may be compared not according to the vertical distance from the MSL, but from the RW altitude. In the case analysed, low passes (at 27-77 m altitude) above the RW were carried out.
Usually, altitude data is shown according to the axes of either time or distance. The GPS provided UTC time is not as accurate, as flight duration. Therefore, the initial flight time t UTC is converted to the start of flight duration, i.e., t UTCn -t UTC0 = 0, where t UTCn is the UTC time of the following point. In Fig. 14, on the x axis an hour's flight duration is shown, during which the student completed 14 traffic circuits. Instead of duration, the axis may laso indicate flight distance. Several distance measurement variants are possible. The Great Circle route distance may be calculated using geographical coordinates from point A(φ 1 , λ 1 ) to point B(φ 2 , λ 2 ) by adding the distances using equation (3) where r -Earth's radius; φ 1 , φ 2 -A and B latitude coordinates; λ 1 , λ 2 -A and B longitude coordinates (Eddie 2012). Or, after convertion of geographical coordinates into Cartesian coordinates, the distance can be calculated by calculating the triangle's hypotenuse using Pythagoras's law.
Despite this, it is difficult to compare and automatically assess data on such an altitude graph, so a side by side comparison of flight traffic ciurcuit altitude graphs is required, when after each traffic circuit the graph is moved to the start of the duration axis. For this, the Matlab function inpolygon was used, which checks whether the chosen point is within a defined zone. For the experiment, the RW area was chosen as the trigger zone: as the aircraft overflies it, the altitude graph is brought back to the start of the axis and another traffic circuit altitude graph is drawn. The results of the experiment are provided in Figure 15. As seen in Figure 15, a few times the aircraft did not cross the trigger zone and some altitude graphs are lengthened. The juxtaposed altitude graphs indicate that the duration of one traffic circuit is from 3 to 5 min (~4 min.). There are several reasons for the occurence of errors in the juxtaposition of the altitude graphs.
Firstly, the juxtaposition is not necessarily made at the RW threshold (marked in red blurbs in Fig. 16). This depends on the GPS receiver location update rate, which may not be simultaneous with the RW trigger zone crossing. Therefore, it is better if the GPS receiver is capable of a more frequent location update. Secondly, the aircraft may enter the runway area not by crossing the RW threshold but by crossing its side (blue blurb in Fig. 16). In this case, the return of the altitude graph to the start is also delayed.
In order to avoid missing the trigger zone and measurement errors, it is suggested to pre-set two trigger zones, which would not permit exclusions of areas in the altitude graphs if an aircraft overflies the RW not very accurately, as marked in Figure 17.
At the beginning of the flight, the programme will check which of the two trigger zones is activated first, Fig. 17. Trigger zones of the flight circuit and it will automatically detect which direction is used for the traffic circuit. The first activated trigger zone will deactivate the trigger zone in the opposite direction, so that a double return of altitude graphs would be avoided. The active trigger zone will be used for the calculation of completed traffic circuits, also for measuring flight duration, and will help to assess all traffic circuits.
According to the altitude graph in Figure 15, it is clear that the pilot retains a constant climb in the upwind and crosswind. The traffic circuit climb speed of all performed circuits forms an aircraft climb corridor of a certain width, according to which a pilot's ability to climb appropriately based on aircraft chacteristics could be assessed. Altitudes of downwind and other legs are scattered. Presumably, due to the fact that the pilot did not maintain the prescribed flight trajectory appropriately. If they flew as accurately as possible, an altitude corridor could also be formed in the last part of the diagramme.
The start of the traffic circuit can also be determined according to the lowest flight altitude, by finding the minimums in the altitude graph. In oder to avoid the return of the minimum of the whole flight, an altitude from which the minimum will be calculated needs to be set. The pre-set altitude in Figure 14 is marked by a horizontal blue line, so that the lowest point of each parable could be found. In the previously described assessment based on the RW threshold, the aircraft has not completed the traffic circuit (has not landed). In such a case, the lowest position of the aircraft in the RW zone would be chosen.

conclusions
1. For the traffic circuit flight assessment model to work properly, it is necessary for the student-pilot to know and comply with a specific task, otherwise the assessment might generate a fail. 2. The flight assessment model must be designed according to the requirements of the specific task, so the assessment system must include different assessment modes for different tasks. 3. The flight assessment model should have acces to the cinematic and aerodynamic model data base of the aircraft used, to match an appropriate aircraft type to an appropriate traffic circuit model. 4. The designed traffic circuit flight assessment model allows assessing flight accuracy parameters, but does not assess the pilot's emotional state and the strategy used in the cockpit to complete the task. 5. When using trigger zones, it is important that the GNSS data updates would be as frequent as posssible, so that the trigger zones could be reduced, as well as the time update errors. If the trigger zone is too narrow, and the GNSS location data update is too rare, there is a possibility that the trigger zone will be excluded and not activated.