EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE GNSS PERFORMANCE VALIDATION SUMMARY

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE GNSS PERFORMANCE VALIDATION SUMMARY EEC Report No. 345 Project NAV-4-E1 Issued: April 2000 The information contained in this document is the property of the EUROCONTROL Agency and no part should be reproduced in any form without the Agency s permission. The views expressed herein do not necessarily reflect the official views or policy of the Agency.

REPORT DOCUMENTATION PAGE Reference: EEC Report No. 345 Originator: EEC SNA CoE (Satellite Navigation Applications Centre of Expertise) Sponsor: DSA (Directorate of Safety, Airspace, Airports & Information Services) Security Classification: Unclassified Originator (Corporate Author) Name/Location: NLR The Netherlands GMV -Spain European Space Agency Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency Rue de la Fusée, 96 B -1130 BRUXELLES Telephone : +32 2 729 9011 TITLE: GNSS PERFORMANCE VALIDATION - SUMMARY Authors A. van den Berg J. Vermeij S. Lannelongue Date 04/00 Pages x+40 Figures 7 Tables 6 Appendix - References 5 EATCHIP Task Project Specification - NAV-4-E1 Distribution Statement: (a) Controlled by: Head of SNA CoE (b) Special Limitations: None (c) Copy to NTIS: YES / NO Descriptors (keywords): Navigation, GPS, GLONASS, GNSS, EGNOS Abstract: Task No. Sponsor - Period 1998-99 This is a summary of a report on the results of the GNSS Performance Validation (GPV) study carried out on behalf of the Eurocontrol Satellite Navigation Applications Centre of Expertise at he EEC. The work analysed the operational validation of the GNSS-1 concept from an aeronautical user point of view. The Total System Performance (TSP) was broken down into three topics: the Signal in Space (SIS), the onboard equipment and the dynamic system performance. This report describes the findings of this study focussing on requirements for measurement, simulation and analysis necessary to validate that GNSS is meeting the user requirements. In addition it describes the configuration options for a simulation environment that is proposed for performing some of the validation tests.

This document has been collated by mechanical means. Should there be missing pages, please report to: EUROCONTROL Experimental Centre Publications Office B.P. 15 91222 BRETIGNY-SUR-ORGE CEDEX France

GNSS Performance Validation EUROCONTROL FOREWORD FOR GNSS PERFORMANCE VALIDATION STUDY This report presents a summary of the final report of a study into Global Navigation Satellite System (GNSS) Performance Validation. The study was initiated to provide background material to support a EUROCONTROL responsibility within the European Tripartite Group (ETG) to perform the validation of GNSS for civil aviation applications. The ETG consisting of the European Commission, the European Space Agency and EUROCONTROL is coordinating the development and implementation of a European component of GNSS called the European Geostationary Navigation Overlay Service (EGNOS). EGNOS is a Satellite Based Augmentation System (SBAS), to the GPS and GLONASS navigation constellations. There are currently three SBAS systems under development, the US WAAS system, the Japanese MSAS system and the European EGNOS system. These regional systems include a ground segment, which monitors the constellation of navigation satellites and transmits corrections and integrity information to the users through the geo-satellites (Figure 1-2). The geostationary satellites also provide GPS-like ranging signals. The GPV report proposes a validation strategy and detailed procedures for the validation of the core GNSS elements, GPS and GLONASS and the satellite based augmentation system. These validation methods are presented in the following structured way: 1. Analysis of critical parameters for the validation process 2. Definition of required tests 3. Definition of a simulation environment to validate: Signal-in-space User Equipment including integrated architectures Aircraft Performance The validation approach evaluates the systems from a user perspective. Procedures for the validation of different levels of performance are defined against the applicable user requirements. Much of the proposed validation methodology is generic to all GNSS and it will provide valuable input to future validation activities for Ground Based Augmentation Systems and GNSS-2. The complete four-volume set of final reports is available, on request, from the EUROCONTROL GNSS Programme. Edward Breeuwer Richard Farnworth EUROCONTROL Project Officers v

EUROCONTROL GNSS Performance Validation This page is intentionally blank vi

GNSS Performance Validation EUROCONTROL EXECUTIVE SUMMARY The purpose of the GNSS Performance Validation (GPV) project was to investigate validation procedures for GNSS-based navigation systems. The results will be used in the definition of a future EUROCONTROL project for the operational validation of GNSS-1 in the ECAC region. The term GNSS-1 is used throughout this document to refer to the GPS and GLONASS satellite positioning systems augmented by the European Geostationary Navigation Overlay Service (EGNOS). Procedures have been developed for the validation of the required signal-in-space parameters and the performance of the user equipment, which together define the overall navigation system performance. Objectives The objective of the GPV study was to propose suitable validation procedures for the operational validation of the use of GNSS-1 (GPS + GLONASS + EGNOS) for civil aviation navigation purposes. This high-level objective was achieved by: analysing the critical parameters for the validation process the definition of adequate tests the definition of a simulation environment for the validation of the signal-in-space, the user equipment and the aircraft performance Validation strategy The validation strategy applied within the framework of this study was to approach the system from the user point of view. From this perspective, it is proposed that the operational validation of GNSS-1 be performed at different levels (SIS, user equipment and aircraft) against the applicable requirements. First the validation procedures which are required to assess the performance of the GNSS signal-in-space are investigated. Once the SIS has been validated, it is necessary to validate the performance of the receiver, possibly integrated with additional navigation sensors such as the inertial navigation system. Finally, the effect of the total system is investigated introducing the aircraft elements that are within the scope of the validation. The developed validation procedures are a combination of the following methods: flight trials static measurements simulation analysis data extrapolation Tests with real data and flight trials are very important and the final evidence of the GNSS performance, however they are very demanding and expensive in terms of resources. Therefore a large part of the system performance will be validated through the use of simulations based on reliable hardware and software models. Those models will need to be validated based on real data and flight trials. Once validated, the defined simulation environment allows many realistic scenarios to be investigated at a lower cost than flight trials. vii

EUROCONTROL GNSS Performance Validation This page is intentionally blank viii

GNSS Performance Validation EUROCONTROL TABLE OF CONTENTS 1. INTRODUCTION... 1 1.1. BACKGROUND... 1 1.2. OBJECTIVE... 2 1.3. VALIDATION STRATEGY... 2 1.4. STRUCTURE OF THE DOCUMENT... 3 2. SCOPE OF THE GNSS PERFORMANCE VALIDATION PROJECT... 5 2.1. INTRODUCTION... 5 2.2. OPERATIONAL REQUIREMENTS FOR GNSS-1... 5 2.3. FINAL REPORT PART 2 - SIS PERFORMANCE VALIDATION... 7 2.4. FINAL REPORT PART 3 - (INTEGRATED) RECEIVER PERFORMANCE VALIDATION... 8 2.5. FINAL REPORT PART 4 - TOTAL SYSTEM PERFORMANCE VALIDATION... 9 3. SIGNAL-IN-SPACE PERFORMANCE VALIDATION... 10 3.1. GENERAL... 10 3.2. GPS AND GLONASS SIS PERFORMANCE VALIDATION... 10 3.3. GEO SIS PERFORMANCE VALIDATION... 11 3.4. SIS FAILURE MODES... 12 3.5. GNSS SIS PERFORMANCE VALIDATION... 13 4. (INTEGRATED) RECEIVER VALIDATION... 15 4.1. SAR SIMULATION ENVIRONMENT AND EXPERIMENTS DEFINITION... 15 4.2. SAR SIMULATION ENVIRONMENT VALIDATION... 16 4.3. STAND ALONE RECEIVER FAULT TREE ANALYSIS... 17 4.4. INTEGRATED RECEIVER ARCHITECTURES... 17 4.5. INTEGRATED RECEIVER SIMULATION ENVIRONMENT DEFINITION... 18 4.6. ADDED COMPLEXITY / LIMITATIONS TO THE FAULT TREE ANALYSIS... 19 4.7. RECOMMENDATIONS FOR FLIGHT TRIALS FOR (INTEGRATED) RECEIVER PERFORMANCE VALIDATION... 20 5. TOTAL SYSTEM PERFORMANCE VALIDATION... 22 5.1. DEFINITION OF SCENARIOS... 22 5.2. CONTRIBUTION OF THE NSE TO THE TSE... 23 5.3. CONTRIBUTION OF THE FTE TO THE TSE... 24 5.4. SIMULATION ENVIRONMENT DEFINITION... 24 5.5. RECOMMENDED SENSOR INTEGRATION... 26 5.6. RECOMMENDATIONS FOR ADDITIONAL FLIGHT TRIALS... 27 6. RECOMMENDATIONS FOR GNSS PERFORMANCE VALIDATION... 28 6.1. SIGNAL-IN-SPACE PERFORMANCE VALIDATION... 28 ix

EUROCONTROL GNSS Performance Validation 6.2. PERFORMANCE VALIDATION OF RECEIVERS AND INTEGRATED SYSTEMS... 28 6.3. TOTAL SYSTEM PERFORMANCE VALIDATION... 29 7. REFERENCES... 30 8. ABBREVIATIONS... 31 Green Pages: Pages Vertes: French translation of the foreword, the introduction, objectives, conclusions and recommendations. Traduction en langue française de l'avant-propos, de l'introduction, des objectifs, des conclusions et recommandations. VALIDATION DES PERFORMANCES DU GNSS... 33 1. INTRODUCTION... 34 1.1. OBJECTIFS... 34 1.2. STRATÉGIE DE VALIDATION... 34 1.3. PORTÉE DU PROJET DE VALIDATION DES PERFORMANCES DU GNSS... 35 2. VALIDATION DES PERFORMANCES DU SIS... 35 2.1. CONCLUSIONS ET RECOMMANDATIONS... 35 3. VALIDATION DES PERFORMANCES DU RÉCEPTEUR (INTÉGRÉ)... 36 3.1. CONCLUSIONS ET RECOMMANDATIONS... 37 4. VALIDATION DES PERFORMANCES DU SYSTÈME GLOBAL... 38 4.1. CONCLUSIONS ET RECOMMANDATIONS... 39 5. STRUCTURE DE LA DOCUMENTATION... 40 x

GNSS Performance Validation EUROCONTROL 1. INTRODUCTION 1.1. Background Phase 1 of the Global Navigation Satellite System (GNSS-1) is the first phase of the introduction of satellite navigation in civil aviation in Europe. The core of the system is the US GPS constellation and the Russian GLONASS constellation. The set up of GNSS-1 consists mainly in the development of three satellite based augmentation systems (SBAS) that cover a large part of the globe. The specific service areas of these systems, called EGNOS, WAAS and MSAS, cover respectively Europe, North America and Japan. Their function is both to provide additional ranging capability and to supply the user with differential corrections and integrity information on GPS and GLONASS, valid for a "wide" area. For critical operations or specific areas, this satellite based augmentation system can be complemented by ground based augmentation systems (GBAS) using multiple local reference stations, or airborne augmentation systems using other sensors available such as inertial navigation systems or altimeters. GPS GLONASS Local Area Systems GNSS-1 User Equipment Hybridisation & Processing EGNOS WAAS MTSAT Figure 1-1: GNSS-1 components A European Tripartite Group (ETG) encompassing the European Commission, the European Space Agency and Eurocontrol, agreed on the development of the European satellite based augmentation system (European Geostationary Navigation Overlay Service, EGNOS). This regional system includes a ground monitoring segment and geo-stationary satellites that are visible in the service zone. The ground segment monitors all the satellites of the navigation constellation and transmits corrections and integrity information to the users through the geo-satellites (Figure 1-2). Additionally these geostationary satellites also provide GPS-like ranging information. 1

EUROCONTROL GNSS Performance Validation GLONASS Geo GPS Master Control Center Monitoring network Figure 1-2: GNSS-1 Architecture The goal of GNSS-1 is to be approved as a basis for the navigation function of civil aviation. Since the Safety of life concept is in the balance, the system has to fulfil very stringent requirements in terms of accuracy, integrity, continuity and availability. Within the tripartite group, Eurocontrol is in charge of the organisation of the validation of GNSS for civil aviation applications. For this purpose, Eurocontrol has initiated the GPV project at the beginning of 1998 in order to develop a validation methodology. A team lead by the Netherlands Aerospace Laboratory NLR and including the Spanish company Grupo de Mecánica del Vuelo GMV and specialised sections within the European Space Agency ESA/ESTEC, were selected to carry out this task of which this report is the main outcome. 1.2. Objective The objective of the GNSS Performance Validation (GPV) study is to propose suitable validation procedures for the operational validation of the use of GNSS-1 (GPS + GLONASS + SBAS (EGNOS)) for civil aviation navigation purposes. This high-level objective is achieved by: analysing all the critical parameters for the validation process the definition of adequate tests the definition of a simulation environment for the validation of the signal-in-space, the user equipment and the aircraft performance 1.3. Validation strategy The validation strategy applied within the framework of this study is to approach the system strictly from the user point of view. From this perspective, the operational validation of GNSS-1 is measured at different levels (SIS, user equipment and aircraft) against the applicable operational requirements from the user as laid down in various standard documents. First the validation procedures, required to assess the performance of the GNSS Signal-In-Space (SIS) that arrives at the user, are investigated. Once the SIS has been validated, it is necessary to validate the performance of the user receiver, possibly integrated with additional navigation sensors such as inertial navigation systems. Finally, the effect of the total system is investigated, introducing aircraft elements in the scope of the validation. 2

GNSS Performance Validation EUROCONTROL The developed validation procedures are a combination of the following methods: Flight trials Static measurements Simulation Analysis Data evaluation and extrapolation Tests with real data and flight trials are very important and constitute the final evidence of the GNSS performance. However, they are very demanding and expensive in terms of resources. Therefore a large part of the system performance will be validated with simulations based on reliable hardware and software models. These models will need to be validated with real data and flight trials. Once validated, the defined simulation environment allows many realistic validation scenarios at reduced costs. 1.4. Structure of the document The results of this study have been combined into one document containing four Parts, each consisting of several Volumes. Document title: GNSS Performance Validation Part 1: Executive Part 2: GNSS Signal-In-Space Performance Validation - Volume I: of GNSS SIS Performance Validation - Volume II: GPS/GLONASS SIS Performance Validation - Volume III: Geostationary Overlay Satellite SIS Performance Validation - Volume IV: SIS Failure Modes - Volume V: GNSS SIS Performance Validation Part 3: (Integrated) Receiver Performance Validation - Volume I: of (Integrated) Receiver Performance Validation - Volume II: SAR Simulation Environment and Experiments Definition - Volume III: SAR Simulation Environment Validation - Volume IV: SAR Fault Tree Analysis - Volume V: IR x Architecture Definition - Volume VI: IR x Simulation Environment Definition - Volume VII: IR x Fault Tree Analysis - Volume VIII: Recommendations for Flight Trials for (Integrated) Receiver Performance Validation 3

EUROCONTROL GNSS Performance Validation Part 4: Total System Performance Validation - Volume I: of the Total System Performance Validation - Volume II: Definition of Scenarios - Volume III: Contribution of the Navigation System Error to the Total System Performance - Volume IV: Contribution of the Flight Technical Error to the Total System Performance - Volume V: Total System Simulation Environment Definition - Volume VI: Recommended Sensor Integration - Volume VII: Recommendations for Additional Flight Trials 4

GNSS Performance Validation EUROCONTROL 2. SCOPE OF THE GNSS PERFORMANCE VALIDATION PROJECT 2.1. Introduction Within the framework of the European Tripartite Group agreement Eurocontrol is responsible for the organisation of validation activities to allow the operational approval of Global Navigation Satellite Systems (GNSS) for air navigation purposes. The GNSS system performance must be validated against the user requirements, which are described by the International Civil Aviation Organisation (ICAO) in terms of Required Navigation Performance (RNP) levels, related to different flight procedures. As mentioned, the strategy within the framework of this project is to validate the navigation system from the user point of view. From this perspective, the system can be divided into four parts (Figure 2-1): the various components of the GNSS Signal-In-Space (SIS) the GNSS receiver other navigation sensors the aircraft GNSS SIS Receiver Other sensors Navigation system performance Aircraft Total system performance Figure 2-1: GNSS system performance In a first step, the operational requirements for GNSS-1 are assessed together with their impact on signal-inspace, receiver and aircraft performance. 2.2. Operational requirements for GNSS-1 Over the past ten years different ICAO panels - RGCSP and AWOP - have attempted to define generic operational requirements for navigation systems in the different phases of flight. The applicable airspace requirements were used as a baseline associated with the operations that were expected in a future ATM environment. These activities have resulted in the endorsement of the concept of Required Navigation Performance (RNP) by ICAO. The RNP parameter values, as proposed by RGCSP and AWOP, are reviewed in Table 2-1. 5

EUROCONTROL GNSS Performance Validation Procedure Expected RNP Type Accuracy - 95%TSE Integrity Continuity Availability Lateral Vertical Risk HAL VAL TTA En-Route 20 37,000 m Not specified 12.6 23,310 m B-RNAV 5 1 7400 m N/A 10-6 / h P-RNAV 2 1 1850 m NPA 0.3 555 m IPV 3 0.3/125 555 m 41 m 10-5 / h CAT I 0.03/50 37m 12m 3.5 10-7 per oper. Not specified 1110 m N/A 1110 m 82 m Not specified Not specified 1-10 -4 / h 167 m 50 m 6 s 1-10 -5 in any 15s Not specified 0.95 0.9975 1 The original ICAO concept uses RNP 4. 2 Currently the usefulness of RNP 1 operations is being investigated, possibly P-RNAV operations may require RNP 0.3. 3 The IPV is not yet an existing operation in European airspace Table 2-1: ICAO RNP parameter values for total system performance derived from the ICAO manuals related to performance types associated with GNSS operations currently expected in European airspace. Using the RNP manuals for the en-route and approach phases of flight, the ICAO GNSS Panel has attempted to actually translate these top-level requirements into world-wide signal-in-space requirements. Moreover EUROCONTROL (in co-operation with ESA in Europe) has attempted to translate the GNSS requirements into GNSS system performance parameters applicable to the European Region. This process has been very difficult, illustrated by the fact that it is now very difficult to re-establish the link between the GNSS signal-inspace requirements, as currently listed in the draft ICAO SARPs - for ECAC, and the original numbers in Table 2-1 RNP Type (nmi) Accuracy (95%) Integrity Continuity Availability Lateral Vertical Risk TTA HAL VAL Global Local 20 to 10 3700 m 300 s 7400 m 1-10 -6 /h 0.999 5 to 2 740 m 15 s 3700 m 1-10 -7 /h 0.9999 N/A 10-7 /h N/A 1 740 m 15 s 1850 m 1-10 -7 /h 0.99999 0.5 to 0.3 220 m 10 s 555 m 1-10 -5 /h 0.9999 N/A 0.3/125 220 m 9.1 m 2x10-7 10 s 555 m 22.8 m 0.9999 1-8x10-6 in 0.03/50 to 16.0 m 7.7 m to in any 6 s 40 m 20m to any 15 s 0.99999 0.9975 0.02/40 4.0 m 150 s 10 m Table 2-2: Proposed performance parameters for GNSS-1 in Europe [EUROCONTROL, June 1998] (nmi - nautical mile 1850 m; approach duration = 150 s) The link between the GNSS SIS specifications and the service performance specifications from ICAO are the receiver MOPS [RD-02]. RTCA and EUROCAE have put together minimum performance requirements for GNSS receivers, again in terms of RNP, including testing procedures. The requirements for SBAS receivers, intended to serve as a primary means of navigation, are shown in Table 2-3. 6

GNSS Performance Validation EUROCONTROL Phase of flight Accuracy Lateral Vertical UEE Integrity FDE SBAS Integrity P MD P FA TTA FD availability FE availability En-route/ Terminal NPA 100m N/A 10-3 10-3 10-5 /hr 8 s 99.80%- 99.90% 99.80%- 99.90% 94.55%- 98.20% 94.55%- 98.20% PA 0.4/0.8/ 0.55 m 1 No req. 2.10-5 / approach 4 s 95% No req. 1 DO-229 Range error to GPS/GLONASS/WAAS satellite, excluding multipath/troposphere Table 2-3: RTCA/EUROCAE MOPS for EGNOS/WAAS receiver Ideally there should be a direct link between Table 2-1 and Table 2-3. Unfortunately, in the process of drafting all the requirements the traceability between the tables has been lost. It would be desirable to re-establish this traceability between the tables. This is a difficult task and the first issue is to start from a clear set of definitions. 2.3. Final Report Part 2 - SIS Performance validation The GNSS signal-in-space performance is specified via the parameters: accuracy, integrity, continuity, and availability. These parameters are discussed with respect to the different elements involved in the emission of the signals to the end-users. In the frame of the GPV project the SIS validation part has been divided in the following elements: GPS & GLONASS SIS performance validation GEO SIS performance validation SIS failure modes GNSS SIS performance validation The first element aims at the definition of procedures to assess the performance of the existing GPS and GLONASS satellites that are the core of the future GNSS-1. The GEO SIS performance is considered separately because of the new complexity of the augmentation performance. The purpose of the SIS failure mode assessment is to gain a better insight into the non-nominal performance and the high level implications on the user of the performance of the system in terms of accuracy and especially integrity, continuity and availability. As the first three elements only cover the performance of individual satellite signals, the final element on total GNSS SIS Performance combines the contributions of the different elements including their failure modes. In this element the performance of the system is assessed in terms of navigation system error (NSE) i.e. the service provided to the user, including the geometrical aspects of the satellite constellation, whereas in the former ones the focus was on satellite pseudorange performance only. 7

EUROCONTROL GNSS Performance Validation 2.4. Final Report Part 3 - (Integrated) receiver performance validation Once the Signal-In-Space (SIS) has been validated, the second important point with considerable impact on the navigation system performance is the GNSS stand-alone receiver (SAR). The GNSS receiver has to fulfil certain requirements in terms of accuracy, time to first fix, ability of interpreting the satellite navigation messages. The third point is the validation of the navigation system performance after the integration of the GNSS receiver with possible other navigation sensors. As there are many different options for sensor integration, selected integrated architectures have been investigated. The strategy to develop a validation environment for an (integrated) GNSS receiver is divided in three steps: Definition of validation scenarios Definition of simulation environment Validation of the simulation environment First the test scenarios needed for the validation of the stand alone and integrated receiver are defined. For the stand-alone receiver, those scenarios are mainly based on the tests as described in the MOPS [RD-02]. For the stand-alone receiver validation, these tests are critically reviewed and specific test scenarios are defined for integrated receivers. The second step is to deduce from the previously defined test scenarios, the required simulation environment. Finally it is necessary that the simulation environment is validated to ensure that it will represent the real environment in a very accurate and reliable way in order to produce meaningful results. This step consists of two elements: The first element is to validate that the individual theoretical models included in the simulator reflect the real environment of the receiver. Those models are e.g. the multipath and interference models and the orbits, satellite error, SA models. Secondly, the final implementation of the complete set of models in the simulation environment must be validated. The only way to do this is to compare the simulation results with real data. Depending on the model, this validation may demand different kinds of real data (e.g. Figure 2-2). The three possibilities are tests using a static receiver in a laboratory, tests with a receiver onboard of a static aircraft and tests with receivers on dynamic aircraft. Recommendations are made about the necessity for flight trials in order to validate the defined simulation facilities. Static Receiver in lab environment Models Receiver on board of a static plane Receiver on board of a dynamic plane Figure 2-2: Types of tests with real data The performance of the receiver under nominal conditions (i.e. without receiver failures) can be validated with the simulation facilities. But, as GNSS performance is defined in terms of accuracy, integrity, continuity and availability, procedures concerning the availability of the receiver are necessary. The establishment of a receiver fault-tree analysis for both a stand-alone as well as an integrated receiver can deal with this problem and is therefore described. 8

GNSS Performance Validation EUROCONTROL 2.5. Final Report Part 4 - Total System Performance validation The main topic of this part is to assess the scope of validation of Total System Performance (TSP) of GNSS usage in civil aviation. The requirements on TSP are determined by the concept of Required Navigation Performance and are established by the International Civil Aviation Organisation (ICAO). These requirements are given in terms of accuracy, integrity, availability, and continuity of the Total System for the different phases of flight (see Table 2-1). The NSE is not the only source of error that has to be accounted for in the total system performance of GNSS- 1. The FTE, which is the error introduced by the guidance system (pilot or autopilot), is also quite important for civil aviation applications. FTE and NSE together determine the TSE, which is the difference between the true position and the desired position of the aircraft (Figure 2-3) NSE FTE Figure 2-3: TSE, NSE and FTE combination The FTE depends on many factors such as phase of flight, actual guidance system used, aircraft type, pilot workload and operational environment. As the GNSS-1 system must meet the requirements in any combination of the possible variables, all of these variables must be input for the validation of the total system. The validation strategy for the total system performance consists of the following steps: To identify the different contributing elements to the total system performance To define of validation scenarios To define the simulation environment To validate the simulation environment Performing the validation using the simulation environment First the identification of the different contributing elements in the total system performance is described. Then the contribution of the NSE and the FTE to the TSP are characterised. Using these inputs different scenarios have been identified for validation. From the scenario description, the requirements on the simulation environment are identified. With respect to the simulation environment for the (integrated) receiver, additional software models are required specifically for the aircraft dynamics and the guidance and control elements. Consequently different applicable scenario environments are proposed. Individual recommendations are made for flight trials to validate the different added models. Finally additional recommendation are made about what is necessary in terms of flight trials in order to validate the full simulation facilities. 9

EUROCONTROL GNSS Performance Validation 3. SIGNAL-IN-SPACE PERFORMANCE VALIDATION 3.1. General For Signal in Space (SIS) performance validation it has been found important to have a clear set of definitions. Especially ambiguousness in the terms: accuracy, integrity, availability and continuity contribute to a lot of confusion. Furthermore, one of the main problems to validate the GNSS SIS is to differentiate the performance of the SIS from the performance of the receiver. Performance of the SIS can only be measured at the output of the receiver, and furthermore, the GNSS requirements are expressed in terms of NSE, which means after processing in a receiver. Since this study aims at validating the SIS some options to remove the receiver failures from the validation procedure are proposed. The ICAO GNSSP selected one option, which is used in the proposed validation procedures. Therefore, the way the SARPS definition of the SIS including a fault-free receiver should be interpreted is shown in Table 3-1. Included in Signal in Space: Physical performance of the signal Quality of information contained within the signal (i.e. clock, ephemeris, integrity, etc.) Environment (tropo- and ionosphere) Receiver accuracy and time-to-alert Included in Receiver budget: Multi-path Interference Masking Receiver failures Table 3-1: Signal in Space error budgets. 3.2. GPS and GLONASS SIS Performance Validation For GPS and GLONASS the RNP parameters have been analysed and a list of critical parameters that require further validation is proposed. As a general rule, those elements which have never been reported as a potential source of SIS errors will not require any validation, for instance the carrier frequency. This list of critical SIS parameters can be summarised as follows: The integrity of GPS and GLONASS SIS is the most critical parameter, since little information exists on this parameter. Accuracy is also a critical parameter, although a lot of information exists on both GPS and GLONASS SIS accuracy, the detailed characteristics of some of the individual sources of SIS accuracy errors are not very well known. Additional information is required on the following components: Space segment errors, especially SV ephemeris and clocks. Environmental errors affecting SIS, especially the ionosphere will require much attention and it strongly recommended to set-up a data collection effort as soon as possible. To a lesser extent also tropospheric effects will need to be considered. SIS availability is probably the least critical parameter, as official information published on GPS NANUs and GLONASS NAGUs characterise the SIS availability very well. However, this official information is known to contain errors. In addition, in the case of GLONASS, the satellite constellation has deteriorated in the past year and a half, and therefore availability figures need to be updated. In the case of GPS, the introduction of a new generation of satellites (Block II-F) will require an additional validation effort to characterise the new satellites. 10

GNSS Performance Validation EUROCONTROL Validation methods have been proposed for each of the identified critical parameters. These are mostly based on measurements, though data extrapolation (especially for integrity and availability) and simulation (especially for ionosphere and troposphere) will also be needed. A network of GPS/GLONASS receivers all over the world seems to be the best solution to have continuous access to GPS/GLONASS SIS. Advantage could be taken from existing infrastructure, e.g. IGS, though this does not yet cover GLONASS satellites. 3.3. GEO SIS performance validation For GEO SIS the same considerations apply as for GPS and GLONASS SIS performance characteristics. The main difference in the SIS is the data-rate, the information in the message and the fact that the messages are generated on the ground instead of onboard the satellite. The Ranging function of the GEO can be treated similar to GPS, but especially the Wide Area Differential (WAD) function adds extra complexity to the validation due to the fact that they can not be treated independent from the corrected errors. Accuracy has three different aspects regarding the GEO SIS performance: The accuracy of the ranging function. This refers to the accuracy of the pseudorange measurement of the GEO SV itself; The accuracy of the fast and long term corrections. This refers to the accuracy of the pseudorange measurement of any navigation satellite after application of the basic corrections provided by the GEO; The accuracy of the ionospheric vertical delay estimates as they are provided per Ionospheric Grid Point (IGP). Most of the accuracy tests are quite similar to the tests for GPS/GLONASS validation. Though now differential corrections are added. It is advised to combine these tests as much as possible. The term integrity is used for the probability that the augmented system accuracy performance is less than what is communicated via the GEO. This occurrence is called an integrity failure for the SIS. Only integrity that is provided through the SIS is assessed here. Other integrity sources (i.e. Receiver Autonomous Integrity Monitoring (RAIM)) are not part of the SIS performance. Validation of integrity is very difficult, as achieving confidence in performance figures of 1-10 -7 requires very long periods of measurements. Therefore it is advised to complement this with a detailed analysis of the safety implications and the failure modes leading to these integrity failures. However the quality of the integrity parameters (UDRE and GIVE) can be validated through simulation. Signal availability is considered to depend only on the coverage area and on failures, resulting in a loss of service (both signal loss and unusable contents). For validation of GEO SIS availability only the message update rate is considered as a critical parameter. In Figure 3-1 a functional breakdown is shown of the GEO SIS performance, including the parameters to be validated. The figure illustrates that the GEO SIS can not be validated completely separated from the core systems (GPS and GLONASS). 11

EUROCONTROL GNSS Performance Validation GEO SIS Performance - System - Signal - Message RF Signal Characteristics Signal quality - received C/N o Availability - Failure rates and modes - Coverage Ranging (GEO) - Clock + ephemeris - URA Basic differential (GPS/Glonass/GEO) - Fast corrections - Long term corr. - UDRE Precise differential (IGP) - Ionospheric corrections - GIVE Accuracy (GEO) - clock/ephemeris - ionospheric (without corr.) - tropospheric Integrity - URA Accuracy (GPS/Glonass/GEO) - clock/ephemeris (with correction) Integrity (GPS/Glonass/GEO) - UDRE - Delay Accuracy (IGP) - ionospheric (with correction) Integrity (IGP) - GIVE - Delay Figure 3-1: Performance breakdown of GEO SIS. 3.4. SIS Failure modes For civil aviation, safety is one of the most important design aspects driving the development of the European Geostationary Navigation Overlay System (EGNOS). Therefore not only baseline requirements such as accuracy, integrity, availability and continuity are important. Also a thorough knowledge of possible failure modes of the Global Navigation Satellite System (GNSS) and their effects is indispensable. The main goal of this part of the project is to identify: Signal In Space (SIS) failure modes, failure effects, and failure statistical figures Procedures to characterise failures. It must be clear that this report cannot cover all the possible failure modes. One of the reasons for this is that the Global Positioning Service (GPS) and the Global Navigation Satellite System (GLONASS) are under military control with information hard to get at. The aim of this document is to structure the current information available on GNSS SIS failures, to identify what is still missing and to provide insight from the user point of view in the way that SIS failure modes can affect the performance. Also, methodologies are proposed to gather this additional information and to characterise failure modes. 12

GNSS Performance Validation EUROCONTROL A SIS failure is defined here as the state where the SIS is not working according to its design specifications. The design specifications are derived from the user performance requirements, in this case mainly from the aviation community where they are defined in terms of accuracy, availability, continuity and integrity. The individual SIS specifications can be defined in the following terms: Pseudorange accuracy of individual satellite signals Integrity information on pseudorange accuracy Physical presence of individual signals With these terms a failure state matrix can be defined: States Integrity information Correct Wrong Unavailable Pseudorange measurement Nominal Nominal conditions False alarm Degraded Correct alarm Hazardous situation Unavailable Unavailability Unavailability Table 3-2: Failure state matrix. The matrix shows that there is only one real dangerous situation for GNSS users: when integrity shows everything is correct while the pseudorange accuracy is severely degraded (combination of two failures). In the matrix, physical presence of the signal is a part of pseudorange unavailability. Depending on a user s requirements both alarm conditions (correct or false) could affect his availability. Failure modes are identified by grouping them into accuracy, integrity and availability failures. Also, to structure analysis, a distinction is made between the stand-alone failure modes and failures due to the augmentation system (SBAS). For these failure modes, duration and frequency figures are partly derived. Establishing these figures however, is not the prime goal of this report. The information on GPS and GLONASS mainly stems from literature, with emphasis on outages and pseudorange degraded performance. The EGNOS system is analysed more specifically, dividing the system into major elements. To gain more insight in the character of SIS failures, data acquisition procedures are established. Real time monitoring using an observation station network is needed to identify real failure effects. To reduce the amount of data handling, data for post-processing and failure analysis should only be preserved over a small period before and during a failure. Until the EGNOS system becomes available, failure injection in a simulated environment is the only other option to support validation. During the EGNOS pre-operational phase, real failure injection might be a useful but remains a limited method for characterising the failure modes already identified. 3.5. GNSS SIS performance validation In the previous sections, GPS and GLONASS SIS performance validation and GEO SIS validation the performance of the signal emitted by individual GPS, GLONASS and GEO satellites have been assessed. The performance has been studied in terms of pseudo-range errors. Here, the geometry aspect of the constellation will be included in order to assess the performance of the system in terms of navigation system performance at the user level. Firstly different architectures that may be included in the future GNSS-1 have been identified. The GNSS-1 elements GBAS and ABAS are not part of this study. Then, procedures to validate the GNSS SIS are proposed. This validation is done against the same RNP requirement parameters: accuracy, integrity, availability and continuity, but now at user level. 13

EUROCONTROL GNSS Performance Validation The main problem with the validation of accuracy is that this parameter is not stationary. This is due to the fact that accuracy varies with satellite geometry (which changes with time). Therefore, it should not be averaged over a long period of time. An option to validate accuracy via the measurement of a stationary parameter is proposed. For integrity, the critical point is the stringency of the requirements. This makes it very difficult to validate it with real data only. However, real data is selected as a basis for two reasons. The first one is that no models exist that can represent the real system with the degree of confidence required. This is especially the case for ionospheric models and errors due to the ionosphere will be most likely the main threat for the integrity of the SIS. Therefore, in order to take advantage of the ionospheric peak activity that will occur in 2000-2002, a measurement campaign should be undertaken during this period. The GPS outages observed recently in the tropics due to scintillation problems confirms that an action of this kind is necessary. The second reason is that the goal of this study is to validate the signal-in-space from a user point of view, therefore the validation shall rely as much as possible on what is available to the user, that is to say the real data. For continuity and availability, validation procedures rely partly on software tools and partly on real measurements. This is due to the fact that these parameters have to be estimated for the whole lifetime of the system. Therefore, measurements recorded have to be extrapolated to other possible scenarios using software simulations. The result of this is a set of procedures that validates the performance of the GNSS SIS against the RNP requirements from En Route to the Precision Approach phase of flight. 14

GNSS Performance Validation EUROCONTROL 4. (INTEGRATED) RECEIVER VALIDATION Part 3 of the Final Report, addresses the validation methods and procedures and the simulation environment required for the validation of the stand-alone and integrated receiver performance. 4.1. SAR simulation environment and experiments definition According to the validation strategy followed in this project, once the SIS has been validated (using a standard receiver), it is necessary to validate the performance at the output of a standard receiver. This validation consists of testing the GNSS stand-alone receiver against the tests that are defined in the MOPS (RD-02). These tests require a GNSS radio frequency (RF) signal simulator that can generate the RF signal as it would be received by the receiver in a real operational environment (Figure 4-1). This means, that the simulator must be able to emulate all the disturbances that could affect the receiver performance in a real environment (i.e. interference, multipath, obstacles, dynamics, etc). Therefore, the currently available simulation models have been analysed. SIS simulator Local disturbances (mpath, interf) GNSS RF simulator Receiver Navigation Performances Receiver Performances Figure 4-1: Receiver performance validation The receiver (Rx) behaviour in its environment can be divided in three error sources: Thermal noise Multipath Interference Thermal noise and RF interference (RFI) are different phenomena, but, in general, they both have a similar effect on the receiver performance. The models used to analyse them share many commonalties. The effects of these disturbances are analysed in terms of acquisition, code tracking loop (code measurement error and tracking losses), carrier tracking loop (phase measurement error and cycle slips), demodulation (data-bit error) and quantisation. These analytical models can predict the effect of interference in the measurement noise. The accuracy of these models to predict the performance of the receiver in the presence of interference can be assessed in the laboratory using a well-characterised signal and a receiver for which the detailed design parameters are available. This is the case of the Multistandard Receiver, for which both the receiver itself and the detailed design are available at ESTEC. These kind of tests can be used to validate the protection masks as described in the MOPS [RD-02]. However, it is important to point out that there is still a lot of work to be performed in order to have really accurate models on the effects of interference on GNSS receivers. 15

EUROCONTROL GNSS Performance Validation Concerning multipath, different models, which can be found in the technical literature, have been reviewed. This review covered the complete spectrum. From simple models (statistical) to high fidelity models (including electromagnetic field and receiver correlation effects). From these models, a high-level overview can be obtained on which multipath model should be applied in any particular analysis. Multipath models can be divided into a signal propagation model, an antenna model and a receiver model. For the signal propagation model, the Geometrical Theory of Diffraction is recommended to be implemented in the GNSS RF simulator and to be included in the MOPS tests description [RD-02]. Concerning the receiver antenna models, the two aspects, which need to be considered, are depolarisation and the antenna gain pattern. A GNSS receiver antenna model proposed by Eisfeller covers both aspects and should therefore be used. The majority of the GNSS receiver models, except the Eisfeller model, model the stationary behaviour of the receiver by neglecting the dynamic effects. This approach leads to less complex and more manageable models, which could be implemented in the receiver S/W model as part of the TSE simulator. The RF simulator aims at reproducing the real environment of a receiver. The advantage of this is that one can reproduce all kinds of situations such as dynamic manoeuvres of the aircraft, multipath or interference scenarios or signal-in-space failures. This kind of tool is thus clearly more flexible than the real environment of the receiver. Therefore, the receiver can be tested in many representative situations. This section of the report contains also a review of the RTCA MOPS [RD-02] resulting in a total of 26 recommendations. Some of them are proposing test methods to be included. Other recommendations concern missing information that must be available in order to complete or clarify some dark points. Finally, recommendations are made to include an additional paragraph in section 2.5 of the MOPS [RD-02]. 4.2. SAR simulation environment validation To obtain correct results from the stand-alone receiver validation it is necessary to validate the simulation environment. This means validation of the GNSS RF simulator and the models involved. In order to be able to validate the GNSS RF Simulator, a clear set of requirements against which to validate the simulator is required. These requirements have been derived from the following sources: The Interface Control Document (ICD) for GPS and GLONASS [RD-03, RD-04] for the signal requirements. The Civil Aviation Requirements as expressed in the ICAO GNSSP SARPs [RD-01] are used for the accuracy requirements of GNSS including SBAS (Table 4-1). The RTCA/DO-229 (MOPS) document, [RD-02], describes the tests for stand-alone receivers. From these tests the different capability requirements are derived. (E.g. Selective Availability (SA), satellite errors and WAD corrections). The aircraft dynamics that must be taken into account in the GNSS RF Simulator are retrieved from the ICAO document 8168 [RD-05]. Simulator Accuracy GPS GLONASS GEO Pseudorange (m) <0.15 <0.3 <0.15 Pseudorange Rate (m/s) <0.21 <0.42 <0.21 Delta Range (mm) < 3 (maximum dynamics) < 0.1 (static user) Table 4-1: GNSS SAR Simulator Accuracy Requirements 16

GNSS Performance Validation EUROCONTROL Concerning the detailed models applied in the RF simulator, each of the software models and RF parameters are evaluated and some of these parameters require additional validation methods when considered critical for the validation of the simulation environment. 4.3. Stand alone receiver fault tree analysis In this volume a fault-tree analysis has been performed for a generic GNSS airborne stand-alone receiver. The analysis is based on a generic GNSS receiver as described in the RTCA MOPS Do 229A [RD-02], from the input signals until reconstruction of navigation data (power supply and data presentation are not taken into account). Presuming reasonable reliability figures for the distinguished receiver components as provided in literature, an illustrative example of quantitative GNSS receiver analysis by means of a fault-tree is given. The failure modes and rates used in the fault tree analysis presented are based on collected field data. Of course the failure rates of a particular component used at a certain receiver could differ from the field-determined average. Nevertheless this approach has led to a fault tree, appropriate to obtain a sensible first indication of the receiver reliability, with the consideration of practical failure modes for the distinguished functions. Assuming practical values for the unknown model parameters, the fault tree resulted in the following Mean Time Between Failure (MTBF) for a generic GNSS receiver: MTBF 30 10 3 hrs (order of magnitude) It is obvious that this is a rough indication of receiver reliability. On basis of studying one particular receiver it is not possible to establish the MTBF for other receivers, without additional analysis for the differing parts. Due to bringing in only one very unreliable component at the one hand or the introduction of redundancy at the other hand, receiver reliability could decrease or respectively increase enormously. Nevertheless the outcome for the theoretical receiver denotes the plausibility that the receiver MTBF requirement can be fulfilled. For detailed evaluation of a particular receiver implementation, the fault-tree should be used with care. For a conscious use of the fault-tree, knowledge of both the receiver and the fault-tree are necessary. The normal procedure applied at reliability studies, is to consider worst case values for the unknown parameters. This results in conservative outcomes for the expected MTBF values, but will never be a guarantee for the actual reliability. 4.4. Integrated receiver architectures The need to provide navigation during periods of shading of the GNSS receiver s antenna and through periods of interference, will be the impetus for integrating GNSS with various additional navigation sensors. This section focuses on the most promising integration options, including their architectures, performance and benefits. Furthermore the civil aviation user perspective is taken into account in the study. There are various ways to improve the integrated receiver performance in terms of availability and continuity, especially in cases of signal loss. The integrated receiver architectures are discussed concisely and an analysis in terms of accuracy performance, improvements in availability, integrity and continuity are provided. An important aspect is also the capability to detect and isolate failures in the combined system. The augmentation options considered are barometric altimeter aiding, GNSS with high stability clock and the use of the radio navigation system, Loran-C. Although various architectures of GNSS integration with different sensors are discussed, the most promising solution assessed is the integration of INS with GNSS, where accuracy is not compromised at all. Although most of the benefits can be discussed in general, certain benefits depend specifically on how the INS is integrated with GNSS. An important advantage that makes the INS such a good candidate is that many commercial aircraft are already equipped with INS and the integration with GNSS equipment can be conducted in several ways, allowing lower costs of a retrofit. 17

EUROCONTROL GNSS Performance Validation The inertial navigation information enables both additional integrity and a near term navigation solution in case of malfunctioning of the satellite navigation solution. Still it is assumed that GPS augmented with GLONASS and EGNOS can comply with the RNP for CAT I. However in the literature there is some doubt if this statement is still true in case of thunderstorm lightning, unintentional RF interference, as well as during deliberate jamming or spoofing. In that case it is certainly beneficial to consider an integrated GNSS/INS system which can be extended by an Airborne Autonomous Integrity Monitoring system. The validation procedures of the integrated receiver are a mixture of three types of validation methods: measurements (e.g. flight trials), simulation and analysis. Principally the flight trials will serve for the final evidence and are therefore recommended, however, there are practical limitations on the extent and amount of data that can be obtained from these flight trials. 4.5. Integrated Receiver simulation environment definition For the validation of an integrated receiver (IR x ) a more sophisticated simulation environment has to be designed. Therefore the need for a GNSS-1 Performance Validation Simulator is identified. The objective of this simulator is to evaluate GNSS-1 performances, algorithms, new digital receiver techniques, sensor integration and most important, to serve as a means of certification. The simulator requirements have been derived from civil aviation and certification requirements. These requirements, defined in the previous sections and in the relevant RTCA documents, define the characteristics of the GNSS equipment to be tested and the procedures. The IR X Simulator has been defined to cope with those procedures. Three different configurations have been defined: GNSS SIS Simulator Configuration This configuration will be running entirely on the GNSS SIS Simulator. This facility should consist of a GNSS Signal Generator and a DEC workstation used for the monitoring and control of the simulation session. Also, the constellation model, the aircraft trajectory and additional sensor models run on this workstation. In this configuration, actual GNSS Receiver hardware is under test. All Software Configuration This configuration will be running entirely on the Software Simulator. It does not require the availability of the GNSS SIS Simulator hardware and the GNSS Receiver. Therefore, this configuration might be more convenient to use e.g. for analysing different navigation filters in combination with various sensor models, etc. There are no constraints on real-time behaviour and on the interfaces to various hardware parts. This simulator can also run on inexpensive computer systems. Hybrid Configuration This configuration is a combination of the GNSS SIS Simulator and the Software Simulator, connected via various interfaces. It has to be mentioned that this configuration is more complex than the previous two configurations, but that it also allows for the most flexible solution. With this configuration, while using the real GNSS Receiver hardware, additional sensor models that are not implemented in the GNSS Simulator can be integrated in the Software Simulator. Various navigation filters can be verified in combination with the GNSS Receiver hardware using this configuration. The various configurations as described above will each have their specific domain of use, depending on the objectives of a specific simulation test. Moreover this simulation environment can be used during the GNSS system life for the certification of GNSS Receivers and integrated equipment, according to the standards dictated by the RTCA for the certification of satellite navigation sensors. A high-level validation plan has been presented for the IRX Simulator including the respective configurations. The validation approach is based on the specific requirements on the simulation architecture. To achieve full validation of the simulation environment it is important to build up a database within the GPV/IRX Simulator containing the results of flight test data. 18

GNSS Performance Validation EUROCONTROL 4.6. Added complexity / limitations to the fault tree analysis The main topic of this part of the study is to identify the limitations of building a fault tree for an integrated GNSS receiver. The investigation is focused on the most promising integration technique for civil aviation uses being GNSS/INS integration. To validate an integrated GNSS receiver by means of a fault tree, the reliability figures of the following 3 major components must be identified: 1. GNSS receiver (first sensor); 2. Additional sensor; 3. Integration algorithm (most commonly: Kalman filter). For the GNSS receiver a fault tree has been set up before (see section 4.3). For the additional sensor a similar approach could be followed if necessary, but in practice most additional sensors are in use for a long time yet, what means that their reliability figures will often be available or at least obtainable by (already existing) field data. For very elementary aiding methods (i.e. an aided receiver is looked at as a stand-alone receiver with one additional element, which can generate the navigation solution on basis of 3 satellites), the integration algorithm will introduce no additional complexity. Unfortunately most integrated receivers cannot be looked at that elementary, when reliability is considered. Validation of the integration algorithm by fault tree analysis will be more complex then, since reliability of the algorithm highly depends on the performance of software. To obtain a complete reliability overview of the (complex) integration algorithm, some software reliability models are available. Those models always require certain input parameters (such as software development time, lines of code or scale factors) to characterise the software performance. Usually not all of those parameters can be determined accurately a priori, what means that finally, the reliability of the software can only be identified after extensive research. Then through a fault tree analysis, the MTBF values need to be estimated. As mentioned, the MTBF of a software program can be estimated by means of extensive testing only and is determined by: MTBF = t n / n When n failures occur while a program is tested during time interval t n. That means that, unless this information can be copied from previous research of software similar to the considered software, testing the integration algorithm is always unavoidable to determine the fault tree analysis of an integrated GNSS receiver. Also in case of a transparent algorithm, like a least square snap shot integration with Loran-C for example, this testing procedure still applies. The outcomes of testing the algorithm should be defined in terms of the probability that: 1. A single GNSS failure passes the algorithm 2. A single failure of the additive sensor passes the algorithm 3. Multiple sensor failures pass the algorithm 4. The algorithm introduces a failure itself (while its environment performs correctly). The MTBF values of the integrated sensor can be calculated then by combining this information with the MTBF values of the sensors in the fault tree. Contrary to a stand-alone GNSS receiver, it usually will be very time-consuming to validate an integrated GNSS receiver by means of a fault tree analysis. The sensors do not form insurmountable problems, but the integration algorithm does. The (dedicated) algorithm must be tested exhaustingly, before its reliability can be established. Since the outcome of the tests should validate the relations of the integration algorithm, the integration algorithm can be seen as a black box during testing. 19

EUROCONTROL GNSS Performance Validation The bottleneck in building a fault tree for an integrated receiver appeared to be the integration (software) part always. Therefore the (im)possibilities of validating this part of the receiver is studied in the most detail. However this study is based on the state of the art of reliability analysis. With the continuous evolution of technology and (software) validation knowledge, it might become less difficult to validate the integration algorithm in the future. Managing the integration part properties at the fault tree analysis of an integrated receiver is illustrated by analysing a generic GNSS/INS receiver by means of a fault tree. 4.7. Recommendations for flight trials for (integrated) receiver performance validation The ultimate check to validate the real navigation system error as experienced by aircraft that use the system for navigation is done by means of flight trials. However, it is very challenging since all the environmental parameters are not under control as they are in the simulation environment. Moreover flight trials are by definition an expensive exercise when pilots and aircraft have to be hired for this reason. Therefore the reasons for recommending flight trials for the validation of navigation system performance must be carefully addressed. Two options exist to minimise the flight trials costs: Perform flight trials on normal commercial flights Minimise the necessity for flight trials. Both options have been examined in this study. With respect to the first option the currently performed data collection program (Eurocontrol SAPPHIRE program) onboard commercial aircraft is found to be very useful. This study reviewed the definition of the program and its usefulness with respect to the validation activities defined in this study. Some recommendations have been made to the program to expand the data collection program to make the data statistically more relevant in terms of geographical and time distribution and to make some additions to the processing of the data. With respect to the second option, the full validation of GNSS navigation system performance by means of flight trials is ideal but not complete as the output does not focus on validation of the different error contributions (multipath/interference/obstacles). Moreover the input of the signal in space is not under control during the trials so anomalies of the SIS cannot be investigated. However from a practical standpoint the flight trials serve as a: 1. Validation of the simulation environment in order to validate the software models used in the simulation. 2. Evaluation to demonstrate the capability of the system to be used for civil aviation. Ad 1: For the stand-alone receiver the validation of the simulator models must focus on the validation of multipath, interference and obstacle models implemented in the simulation environment. For the integrated navigation system also the different sensor models must be subject to validation during the flight trials. Ad 2: The measured navigation system error must be used for validation of the total simulator performance and to demonstrate the NSE characteristics as an input to the determination of the total system performance. Therefore improvements have been suggested for the SAPPHIRE project and specific flight trials have been recommended for the purpose of the validation of the GNSS performance simulator. The huge amount of data collected for SAPPHIRE will allow the derivation of representative statistics on availability and continuity, whereas simulation facilities will allow the validation accuracy and integrity. The two options are complementary. The use of both options provides a quite useful tool in order to validate the performance of the system from the stand alone and integrated receiver point of view. The recommendations for the on going SAPPHIRE project are twofold: The first type of recommendations result from the review of SAPPHIRE. The goal of this review was to assess the consistency of the trials comparing to the initial objectives of the project. The second type of recommendations aims at the extension of the scope of SAPPHIRE. It regroups the tests that are developed in this report for GNSS validation that are applicable to trials on civil airliners. 20

GNSS Performance Validation EUROCONTROL Although some of the SAPPHIRE recommendations represent minor changes in the project and could be very beneficial to the validation process, interference and multipath have been identified as critical for the receiver performance validation. A technique using code carrier measurement differences allows estimating the kind of multipath and interference present in the airborne environment. Since SBAS does not bring anything in that matter, meaningful recording can start right now. 21

EUROCONTROL GNSS Performance Validation 5. TOTAL SYSTEM PERFORMANCE VALIDATION Part 4 of the GPV deliverable addresses the validation methods and procedures and a proposed simulation environment for the validation of the total system performance, including the contribution of the navigation system and flight technical error and their possible correlation. 5.1. Definition of scenarios The main topic of this part is to define test scenarios in the scope of validation of Total System Performance (TSP) of Global Navigation Satellite Systems (GNSS) usage in civil aviation. The scenarios are a combination of simulations and flight trials. The main question that arises is: what are the different elements of the total system performance that need to be validated? Therefore first the elements of TSP are assessed. From a high level perspective, the total system performance is interpreted to be the behaviour of the centre of gravity of the aircraft relative to the desired path. This includes the contributions of all the relevant elements that may impact the execution of the flight. Basically these elements are identified to be: Position determination Route determination Accuracy of adherence to flight plan Pilot workload Apart from these items also the influence of the availability of air traffic control on the aircraft's total system performance is described. However as it is an element outside the aircraft it is not considered an integral element that needs to be validated when considering the total system performance. On basis of the TSP elements, different test scenarios groups are defined, which are: 1. Correlation of adherence to flight plan with other TSP elements such as position determination, pilot workload and aircraft type. 2. Procedures, which are the final baseline of the validation. 3. ATC capabilities, especially related to backup procedures in case of EGNOS long term outages. It is recommended that all these scenarios be executed by simulation first. After that, only the procedures need to be tested by flight trials. Besides that, flight trials are also needed to validate simulation models. By analysing the test outputs, the GNSS performance can be validated against the requirements given by ICAO on accuracy, integrity, availability and continuity of service. The accuracy parameter can be measured directly as a sum of the position determination error and the error in adherence to the flight plan. Availability will be measured at the beginning of a procedure. Outages of the system during the procedure will establish the actual continuity figures. However since the integrity parameter is mainly provided by the EGNOS ground system and specified in terms of 10-7, it is concluded that it is very difficult to assess the integrity requirements by means of simulations only. The fast time simulation needs to be complemented with real time simulations, performance analysis and flight trials to increase the confidence level. The outcome of the test scenarios can also be a basis for a cost/benefit analysis. Using an assessment of the accident risk of aircraft, the minimum separation between parallel routes could be determined. 22

GNSS Performance Validation EUROCONTROL 5.2. Contribution of the NSE to the TSE As mentioned, one of the main contributing parameters in the total system performance is the navigation system performance. Therefore first a theoretical analysis of GNSS NSE is performed, supported with available flight data available as a result of a flight test campaign done in the frame of the EGNOS Early Trials, in June-July 1996. This analysis has focused on the major components of GNSS NSE, which are: UERE of all satellites in view, which comprises a number of error sources such as: - clock and ephemeris errors - receiver noise errors - tropospheric and ionospheric errors - multipath errors Geometric properties (Dilution of Precision or DOP) of the satellite constellation in view. All possible combinations of the existing satellite constellations are presented, i.e. GPS, GLONASS, GPS+GLONASS, GPS+GEO and GPS+GLONASS+GEO. The analysis has focused on the horizontal and vertical NSE (HNSE and VNSE) of the stand-alone receiver. These parameters have been characterised as a function of real data, taking into account the aircraft dynamics (velocity, acceleration and jerk). Theoretical results for 95 percentile of these parameters have been compared with results obtained with flight tests. These show that theoretical formulae provide only an upper bound to HNSE 95% and VNSE 95%. Substantial differences have been found when analysing real HNSE 95% and VNSE 95%. This significant difference is due to the fact that some current assumptions on the characteristics of HNSE and VNSE may not be valid: The Gaussian approximation for GNSS UERE is probably not completely true. GNSS UERE shows a great dependency with elevation. This introduces a sharpening effect on the Gaussian characteristics of UERE. DOP is a time varying function. The result of multiplying DOP by UERE introduces another kurtotic effect on NSE. In addition, worst case conditions for DOP should be replaced by average DOP conditions in the computation of NSE 95%, as they seem to better match theoretical results with actual flight data. As a result VNSE follows a Gaussian-like function, but with a sharpening effect that removes energy from the queues, to be placed around its mean value. Thus the real 95 percentile results will be better than the theoretical predicted one. The effect on HNSE, which follows a Rayleigh-like function, is similar. Finally, conclusions obtained are summarised to provide a set of recommendations for the simulation of NSE within a complete GNSS TSE Simulator. Out of these recommendations it is important to note that: Satellite visibility on board the aircraft depends very much on the antennae configuration deployed on the fuselage. This is an issue that needs to be further investigated in order not to lose track of visible satellites when the aircraft is performing manoeuvres. Accurate simulation of multipath in PA and landing phases of flight requires a flight test campaign to characterise this effect. It is to be noted that each airport environment is likely to be characterised by different multipath profiles. 23

EUROCONTROL GNSS Performance Validation 5.3. Contribution of the FTE to the TSE The second main component of the Total System Error (TSE) is the Flight Technical Error (FTE). The main point of interest for the FTE is the relation to the behaviour of the Navigation System Error (NSE). Both real pilot and autopilot behaviour has been considered in terms of accuracy, because the RNP-parameters integrity, continuity and availability are system-specific. In order to perform the assessment, a literature survey has been performed together with an analysis of the results of already performed flight trials. After the review of available material the gaps and suspected issues with respect to FTE are identified followed by recommendations how to validate those aspects. These have been selected according to the impact of possible mismatches in the common agreed values. This has been done for all flight phases for which GNSS has to be validated: En-route TMA Non Precision Approaches Cat 1 precision approaches The survey resulted into a focused programme of necessary flight/simulation trials and a number of situations, which may require the mandated use of the flight director if flown manually, for instance during non-precision approaches and maybe the TMA flight-phase as well. The following FTE-issues are proposed to be a subject for validation trials: The 2σ value of the FTE during manually flown RNP1 En-route and TMA flights without the use of a Flight Director; The 2σ value of the FTE during manually flown non-precision approaches using Flight Directors; The 2σ value of the FTE during manually flown Cat I Precision Approaches with and without the use of a Flight Director; The possible correlation of FTE and NSE during the spiral mode and the impact on the GPV-activities. The trials for en-route, TMA and NPA can be conducted via a flight simulator using a GPS model. The trials for Cat I precision approaches require at least a very detailed GPS/EGNOS model or a GNSS simulator in combination with an aircraft simulator. At the end of the day these trials have to be confirmed in flight trials using EGNOS prototyping equipment. These tests are necessary due to the potential effect of the NSE on the FTE by correlation of EGNOS error frequencies and eigen-frequencies of the aircraft, which effect can be very sensitive during a precision approach. 5.4. Simulation environment definition The simulation environment (GPV Simulator) to be used for evaluating and validating the total system performance of GNSS is defined. The total system error has to be validated including the different contributions of the navigation system error (NSE) and the flight technical error (FTE) as described in the previous sections. The simulations can involve hardware (i.e. when a flight simulator or the GNSS receiver are in the loop) or software only. For the all software configuration the simulator is based on the simulation environment defined in section 4.5 with additional capabilities and features. 24

GNSS Performance Validation EUROCONTROL As a first step the test scenarios as proposed in 5.1 have been analysed. Based on these test scenarios a number of requirements have been identified for the simulation environment. This collection of requirements is covering the complete simulation environment. The proposed test scenarios have a large impact on the actual facilities needed to perform these tests. Six simulator configurations have been defined, which correspond to the actual physical facilities necessary to run different sets of tests. A reference between the test configurations and the test scenarios including the scenario variables is provided in this document. From a practical point of view, these configurations are based on the hardware and software facilities needed to perform the proposed test scenarios, also taking into account the consideration made in section 4.1. The six configurations are mentioned below. Software Simulator This configuration is capable of running all the tests where an autopilot can be or has to be used instead of a real pilot. This will reduce significantly the required hardware to perform the tests. Software Simulator with Hardware GNSS Receiver This configuration is in principle the same as the Software Simulator, except that a hardware GNSS Receiver is being used in the simulations where a high confidence level of the navigation system performance is required. Flight Simulator This configuration is used to assess the pilot performance (human-in-the-loop) and requires the availability of a high fidelity flight simulator, i.e. including a moving base, a high-resolution visual system, a hardware representation of the aircraft cockpit including the instrumentation and controls, etc. Flight Simulator with Hardware GNSS Receiver This configuration is in principle the same as the Flight Simulator configuration, except that a hardware GNSS Receiver is being used for the high confidence level required in the simulations. ATC Simulator This configuration requires the availability of an ATC simulator, including models to simulate multiple aircraft and enables the validation of ATC capabilities in a GNSS environment. Flight Simulator and ATC Simulator This configuration, connecting the flight simulator and the ACT simulator improves the reliability of the validation scenarios including both the pilot performance and the ATC capabilities by providing pilot and controller with a realistic environment. The six simulator configurations have been defined in detail, including their architecture, software components and hardware interfaces. With the detailed information on the simulator configurations available, it is possible to identify exactly which requirements on the simulation environment are applicable to each of the simulator configurations. Finally a high-level verification and validation plan for the simulation environment has been described. 25

EUROCONTROL GNSS Performance Validation 5.5. Recommended sensor integration Sensor integration is usually performed in order to use the advantages and to overcome the disadvantages of sensors. Although the subject of this project is the performance validation of GNSS, integration of GNSS with other sensors is discussed. The goal of this section is to select an optimal set of sensors and to propose tests to validate the performance of sets. First the requirements for a set of sensors have to be selected. The SIS requirements refer to a fault- free receiver. However, a receiver will never be fault-free in practice. Increasing the number of receivers will result in a nearly fault-free set of receivers, which has to comply with the SIS requirements. Various sensor combinations are compared with respect to accuracy, continuity, integrity, and availability. INS will only comply for a sufficient period to the accuracy requirements of the operation oceanic en-route. GPS+GEO and GPS+GLONASS+GEO are expected to comply to the accuracy requirements up to CAT I, whereas GPS and GPS+GLONASS do not comply with the requirements of the operations CAT I and IPV. The accuracy of an integrated GNSS/INS receiver is only slightly better than the accuracy of a GNSS receiver. Using more than one sensor of the same type does not result in a significant improvement of accuracy. On the contrary, using more than one sensor of the same type is beneficial in most cases with respect to continuity, integrity and availability. By applying two or more GPS+GLONASS receivers or GNSS/INS receiver's continuity and availability requirements are fulfilled. By applying only a number of GPS receivers, continuity and availability requirements are not fulfilled, irrespective of the number of receivers. Using two or more GPS+GEO receivers will result in compliance to the continuity and availability requirements. For the operation NPA those requirements are expected to be fulfilled as well, although this could not be shown due to the lack of some information. Finally, a GPS+GEO or a GPS+GLONASS+GEO receiver is expected to fulfil requirements with respect to integrity. The integrity risk depends on the probability of failure of the equipment, on the probability of failure of the monitoring system and the probability of failures that cannot be detected by the monitoring system by nature. Since the values of the last two probabilities have not been found during a literature survey, a precise evaluation of the performance of a set of sensors with respect to integrity could not be performed. Based on a performance analysis and bearing in mind that the costs should be minimised, the following set of sensors is recommended: The recommended set of sensors consists of at least 2, or in case 3 INS sensors have been installed previously, 3 uncoupled integrated GNSS/INS receivers. The validation of the following elements, which have not been proposed as subject of validation in other sections, is of importance for the validation of a set of sensors: the MTBF of a GNSS receiver, the independence of failures of GNSS receivers, the MTBF of the monitoring system. In case the expected MTBF of a GNSS receiver is to be validated (MTBF = 30000 hour), the IGS network is most suited. In case compliance of the MTBF to the GPS/WAAS MOPS is to be validated (MTBF 5000 hr), the tests proposed in [RD-02] for validation of GNSS SIS integrity should be used. Validation of the independence of failures of GNSS receivers should be performed by observing the number of simultaneous multiple receiver failures in the IGS network for a period of a couple of years. Finally, the MTBF of the monitoring system can be validated by interrupting and corrupting the output of sensors and observing the number of times no warning was given by the monitoring system. 26

GNSS Performance Validation EUROCONTROL 5.6. Recommendations for additional flight trials The two main objectives of the flight trials defined in this document are to validate the GPV TSP simulator and to demonstrate the TSP of a navigation system relying on GNSS. Concerning the validation of the simulator, the strategy adopted is to reproduce a flight trial in simulation and compare the performances with the data obtained in real situation. Since it appeared too challenging to do it with software simulator it has been decided to record the environment during the trials and to use it as input in the simulator. A validation in three steps is proposed. First the modules dealing with the navigation sensors are validated. Then, the close loop simulator is split into one half loop dealing with the NSE and another one dealing with the FTE. Finally procedures aiming at validating the close loop simulator for each phase of flight are defined. This includes the definition of decision criteria specific to each phase of flight for declaring the simulator operational. Once validated, the simulator will allow reproduce many different situations in order to validate the GNSS for civil aviation users. However, simulations are not sufficient for declaring the system fully operational. Some sensitive scenarios have to be performed with real flight in order to demonstrate the capabilities of the system to meet the civil aviation requirements. Those scenarios are described in the report for each phase of flight. However, it is important to point out that, for the time being, those scenarios have been defined from information present in literature. Furthermore certain independence is assumed between NSE and FTE. Those figures and assumption might not be totally suitable to GNSS. Some tests are also defined in this document in order to assess the applicability of those assumptions and information to navigation systems using GNSS. Therefore, the run of simulations will probably bring new elements that will allow adjusting the flight trials aiming at demonstrating the performances of GNSS system for civil applications. However, four additional flight trials are proposed. Since flight trials are very demanding in terms of resources, the ones selected in this document have been optimised to cover the all the objectives previously defined: The validation of the simulator for all the phase of flight The demonstration of GNSS TSP for civil aviation users The validation of assumptions coming from literature for GNSS 27

EUROCONTROL GNSS Performance Validation 6. RECOMMENDATIONS FOR GNSS PERFORMANCE VALIDATION 6.1. Signal-In-Space Performance validation The following recommendations with respect to the validation of the performance of the Signal in Space (SIS) have been identified as a result of this study: Validation of nominal SIS performance should be carried out according to the methodology presented in the GPV study. Within the validation plan, the use of clearly stated definitions is important in order to avoid confusion and mis-interpretations. Primary means of SIS validation are real measurements (static). Where necessary, simulations can be used to cover special conditions and to extend the amount of data. The use of an independent validation network is required. Combination with an existing network like the IGS is possible and advised. Data from different measurement campaigns should be combined as much as possible. The validation activities should start as early as possible, because: This would allow to obtain a large amount of data; Through this study, ionosphere has been identified as a critical point for the SIS GNSS performances, especially for integrity, continuity and availability. The ionospheric activity will reach a periodic peak in 2000/2001. Therefore, the data collection should start as soon as possible in order to take the opportunity to collect real data in the worst ionospheric conditions. Validation of anomalous SIS performance can not be covered by measurements only; therefore simulation and analysis should be used. Failures resulting in unavailability could be validated by means of the RAMS report and simulations. The analysis of the hazardous situation requires special attention, i.e. a safety case. 6.2. Performance validation of receivers and integrated systems The following recommendations with respect to the validation of the performance of the stand-alone and integrated receiver have been identified as a result of this study: Additional analysis should be undertaken in the area of GNSS receiver interference and multipath as these are found to be critical to the receiver performance, especially with respect to the Cat. I operational requirements. The basis for the validation of the stand-alone receiver performance should be taken from the RTCA MOPS Do 229A (RD-02). However, after a critical review of the tests defined in the MOPS, some gaps have been identified specifically with respect to multipath. Therefore additional tests have been defined and recommended for inclusion to EUROCAE/RTCA. It has been found that no internationally accepted documents exist which define the required tests for integrated architectures. As it was identified that inertial navigation systems are the most promising integration solution, specific tests have been recommended for the integrated GNSS/INS architecture. For the validation of the stand-alone receiver performance, the use of a GNSS SIS RF simulator is required. The RF simulator should allow simulation of interference, multipath, aircraft dynamics, etc as required by the defined test scenarios. For the validation of the integrated receiver, additional software models must be developed to simulate the performance of the additional navigation sensor outputs. 28

GNSS Performance Validation EUROCONTROL Real tests such as flight trials are still necessary to validate the simulation facilities. Flight trials should be performed to assess the validity of the software models implemented in the simulator. For stand-alone receivers those models mainly deal with interference and multipath aspects, while for integrated architectures, those models aim at simulating the additional sensor outputs. For the assessment of receiver failures, which is important for the assessment of the NSE availability and continuity, two complementary methods are proposed. The first is to carry out a fault tree analysis of the different receiver architectures. The second one is to use the data collected by the Eurocontrol in the ongoing SAPPHIRE project. The data recorded on SAPPHIRE shall be compared with the outputs of the fault tree analysis. In order to take full advantage of the SAPPHIRE project in the context of GNSS performance validation, recommendations for improving this project have been provided. Especially the characterisation of interference and multipath effects could be incorporated in SAPPHIRE. 6.3. Total System performance validation The following recommendations with respect to the validation of the performance of the total aircraft system have been identified as a result of this study: Both the navigation system and the flight technical error determine the total system behaviour. Changes in both NSE and FTE must be taken into account as well as the possible correlation effects when addressing the implementation of GNSS to existing operations. As for the (integrated) validation, the validation strategy proposed by GPV is to rely on simulation. This has two advantages: simulations are flexible and nevertheless allow a high degree of fidelity (hardware in the loop). They are also cost effective since allow to decrease the need of rather expensive flight trials. Several simulation architectures are proposed in accordance with the required validation tests. The range of architectures varies from an all software simulation environment to a simulation environment which includes a hardware GNSS receiver and a flight simulator. The hardware GNSS receiver is necessary to model the NSE for the validation of Category 1 operations, where interference and multipath are the largest budget error; and The flight simulator would be required to incorporate the effects of pilot guidance using GNSS signals. Although the main part of the validation process relies on simulation, flight trials are still necessary. The objectives of those trials are first to validate the simulation environment for all the phases of flight and secondly to demonstrate the GNSS based TSP to civil aviation users. Although ATC is not a formal part of the total system performance of a single aircraft, it may be interesting to address the capability of ATC to address major GNSS failures. Therefore an architecture which allows an assessment of this capability is also proposed. Based on an assessment of costs, validation activity required and potential benefits in performance, it is recommended to use a receiver architecture that consists of 2 or, in case 3 INS sensors have been installed previously, 3 uncoupled integrated GNSS/INS receivers. 29

EUROCONTROL GNSS Performance Validation 7. REFERENCES [RD-01] "Standards and Recommended Practices for GNSS", draft version 7.0, August 1998. [RD-02] "Minimum Operational Performance Specification for the Wide Area Augmentation System", RTCA, Do 229, 1998. [RD-03] ICD-GPS-200, revision IRN-200C-002, 25 September 1997. [RD-04] GLONASS-ICD, Coordinational Scientific Informational Centre, October 1995. [RD-05] Procedimientos para los servicios de navegacion aerea. OPERACION DE AERONAVES. Doc 8168- OPS/611. Volume I. ICAO 1993. 30

GNSS Performance Validation EUROCONTROL 8. ABBREVIATIONS ATC CAA DOP EGNOS ESA ESTEC FTE GEO GEO GIC GLONASS GMV GNSS GPS GPV HNSE ICAO ICD IGP IGS INS IOD IR MOPS MTBF NAGU NANU NLR NSE PA PLL RAIM RF RNP R x SAR Air Traffic Control Civil Aviation Authority Dilution of Precision European Geostationary Navigation Overlay Service European Space Agency European Space Technology and Research Centre Flight Technical Error Geostationary satellite Geostationary GEO Integrity Channel GLObal NAvigation Satellite System (Russian) Grupo de Mecánica del Vuelo (Spain) Global Navigation Satellite System Global Positioning System GNSS Performance Validation Horizontal Navigation System Error International Civil Aviation Organisation Interface Control Document Ionospheric Grid Point International GPS reference network Inertial Navigation System Issue Of Data Integrated Receiver Minimum Operational Performance Specification Mean Time Between Failure Notice Advisory to GLONASS Users Notice Advisory to NAVSTAR Users National Aerospace Laboratory (the Netherlands) Navigation System Error Precision Approach Phase-Lock Loop Receiver Autonomous Integrity Monitoring Radio Frequency Required Navigation Performance Receiver Stand-Alone Receiver 31

EUROCONTROL GNSS Performance Validation SARPS SBAS SIS SNR SV TSE TSP UDRE UERE UIVE VNSE WAAS WAD Standards And Recommended Practices Satellite Based Augmentation System Signal In Space Signal-to-Noise Ratio Satellite Vehicle Total System Error Total System Performance User Differential Range Error User Equivalent Range Error User Ionospheric Vertical Range Vertical navigation System Performance Wide Area Augmentation System Wide Area Differential 32

Validation des Performances du GNSS Résumé EUROCONTROL VALIDATION DES PERFORMANCES DU GNSS AVANT-PROPOS Le présent rapport constitue une synthèse du rapport final de l'étude consacrée à la validation des performances du Système mondial de satellites de navigation (GNSS). Cette étude a été entreprise dans le but de recueillir des informations de base devant permettre la validation du GNSS pour les applications de l'aviation civile, tâche confiée à EUROCONTROL dans le cadre du Groupe tripartite européen (GTE). Composé de la Commission européenne, de l'agence spatiale européenne et d'eurocontrol, le GTE a pour mission de coordonner le développement et la mise en œuvre d'une composante européenne du GNSS appelée Complément géostationnaire européen de navigation (EGNOS). L'EGNOS se présente sous la forme d'un complément spatial (SBAS) des constellations de satellites de navigation GPS et GLONASS. Trois systèmes SBAS sont actuellement à l'étude, à savoir : le système WAAS, américain, le système MSAS, japonais, et le système EGNOS, européen. Ces systèmes régionaux intègrent tous un segment terrestre qui a pour fonction de surveiller les constellations de satellites de navigation et de transmettre aux usagers les données de correction et d'intégrité par l'intermédiaire de satellites géostationnaires (Figure 1-2). Ces satellites géostationnaires émettent également des signaux de mesure de distance de type GPS. Le rapport GPV propose une stratégie de validation ainsi que des procédures détaillées pour la validation des éléments de base du GNSS - GPS et GLONASS et du complément spatial. Ces méthodes de validation sont structurées de la manière suivante : 1. Analyse des paramètres critiques pour le processus de validation 2. Définition des essais requis 3. Définition d'un environnement de simulation à valider Signal électromagnétique Equipements des utilisateurs, y compris les architectures intégrées Performances des aéronefs L'approche retenue pour la validation des systèmes repose sur une évaluation de ces derniers sous l'angle de leurs utilisateurs. Les procédures de validation des différents niveaux de performance sont systématiquement définies par rapport aux besoins des utilisateurs. La méthode de validation proposée est, pour l'essentiel, générique à l'ensemble du GNSS et fournira de précieux apports pour les travaux futurs de validation du complément au sol ainsi que du GNSS-2. La série complète, en quatre volumes, des rapports finals peut être obtenue, sur simple demande, auprès du Programme GNSS EUROCONTROL. Edward Breeuwer Richard Farnworth Responsables du projet, EUROCONTROL 33

EUROCONTROL Validation des Performances du GNSS Résumé 9. INTRODUCTION Le projet de validation des performances du GNSS (GPV) avait pour objet d'étudier des procédures de validation pour les systèmes de navigation opérant sur la base du Système mondial de satellites de navigation. Les résultats des travaux serviront à la définition d'un futur projet EUROCONTROL de validation opérationnelle du GNSS-1 en région CEAC. Le terme GNSS-1 désigne ici les systèmes de localisation par satellites GPS et GLONASS, augmentés du Complément géostationnaire européen de navigation (EGNOS). Des procédures ont été mises au point pour la validation des paramètres requis du signal électromagnétique ainsi que des performances des équipements des utilisateurs, qui, ensemble, caractérisent les performances globales du système de navigation. 9.1. Objectifs L'objectif de l'étude GPV était de proposer des procédures appropriées pour la validation opérationnelle de l'emploi du GNSS-1 (GPS + GLONASS + EGNOS) pour les besoins de l'aviation civile. Cet objectif de haut niveau a été atteint grâce à : l'analyse des paramètres critiques pour le processus de validation la définition de tests appropriés la définition d'un environnement de simulation pour la validation du signal électromagnétique, des équipements des utilisateurs et des performances des aéronefs 9.2. Stratégie de validation La stratégie de validation appliquée dans le cadre de l'étude consistait à considérer le système sous l'angle de ses utilisateurs. De ce point de vue, il est proposé que la validation opérationnelle du GNSS-1 soit réalisée à différents niveaux (SIS, équipements des utilisateurs et aéronefs), en se référant aux besoins correspondants. Les procédures de validation requises pour évaluer les performances du signal électromagnétique (SIS) du GNSS sont analysées en premier lieu. Dès que le SIS a été validé, il est ensuite nécessaire de valider les performances du récepteur, intégré, le cas échéant, à d'autres capteurs de navigation tels que le système de navigation par inertie. L'étape ultime consiste à tester l'efficacité du système global en introduisant les éléments "aéronefs" dans le champ de la validation. Les procédures de validation mises au point sont une combinaison des méthodes suivantes : essais en vol mesures statiques simulations analyses extrapolation de données Les essais réalisés au moyen de données réelles ainsi qu'en vol revêtent une importance capitale dans la mesure où ils apportent la preuve ultime du bon fonctionnement du GNSS. Malheureusement ces essais sont particulièrement exigeants et onéreux. Aussi bon nombre de performances du système seront-elles validées sur la base de simulations réalisées à l'aide de matériel informatique fiable et de modèles logiciels éprouvés. Ces modèles devront être eux-mêmes validés au moyen de données réelles et d'essais en vol. Une fois validé, l'environnement de simulation défini permet d'étudier un large éventail de scénarios réalistes pour un coût inférieur à celui des vols d'essais. 34

Validation des Performances du GNSS Résumé EUROCONTROL 9.3. Portée du Projet de validation des performances du GNSS Les performances du GNSS doivent in fine être validées par rapport aux besoins des utilisateurs, lesquels sont décrits par l'organisation de l'aviation civile internationale (OACI) en termes de niveaux de qualité de navigation requise (RNP), correspondant, chacun, à des procédures de vol distinctes. La stratégie adoptée consiste à valider le système de navigation en se plaçant au point de vue de ses utilisateurs. Le système peut être décomposé en trois éléments : le signal électromagnétique (SIS) du GNSS, les équipements des utilisateurs (parmi lesquels le récepteur GNSS, mais aussi les autres capteurs de navigation) et le système global, qui inclut les aéronefs. La documentation restante, de même que les sections qui suivent obéissent à cette subdivision. 10. VALIDATION DES PERFORMANCES DU SIS Les performances du signal électromagnétique du GNSS sont déterminées sur la base de quatre paramètres : précision, intégrité, continuité et disponibilité. Ces paramètres sont examinés par référence aux différents éléments intervenant dans l'émission des signaux vers les utilisateurs finaux. Dans le cadre du projet GPV, la validation du SIS s'est articulée en quatre volets : validation des performances du SIS GPS & GLONASS ; validation des performances du SIS GEO ; modes de défaillance du SIS ; validation des performances du SIS GNSS. Le premier volet porte sur la définition de procédures permettant d'évaluer les performances des satellites GPS et GLONASS actuellement en service, qui forment le noyau du futur GNSS-1. L'analyse des performances du SIS GEO est envisagée séparément en raison de la complexité supplémentaire qu'engendre le système d'appoint du GNSS et son mode de fonctionnement. L'évaluation des modes de défaillance du SIS vise à mieux cerner les performances non nominales du système ainsi que les incidences globales, sur les utilisateurs, des performances obtenues en termes de précision et, surtout, d'intégrité, de continuité et de disponibilité. Si les trois premiers volets ne portent que sur les performances des signaux émis par les satellites, considérés individuellement, le volet final, consacré aux performances du SIS du GNSS intégral, combine, lui, les apports des volets précédents, y compris les données relatives aux modes de défaillance. La performance du système y est évaluée en termes d'erreur du système de navigation (NSE), c'est-à-dire le niveau du service fourni à l'utilisateur, en ce compris les aspects géométriques de la constellation de satellites. 10.1. Conclusions et recommandations Les travaux de validation des performances du signal électromagnétique (SIS) ont débouché sur les conclusions et recommandations suivantes : La validation des performances nominales du SIS devrait s'opérer selon la méthode proposée dans l'étude GPV. Dans le cadre du plan de validation, le recours à des définitions claires et précises est primordial pour éviter les confusions ou erreurs d'interprétation. La validation du SIS s'effectue essentiellement sur la base de mesures réelles (statiques), complétées, le cas échéant, par des simulations visant à recréer des situations particulières et à étoffer les résultats des mesures initiales. 35

EUROCONTROL Validation des Performances du GNSS Résumé La validation des performances du SIS commande l'utilisation d'un réseau indépendant. Il est toutefois possible, et même conseillé, d'associer ce réseau à un réseau existant, comme l'igs. Les données en provenance de différentes campagnes de mesure devraient être combinées dans toute la mesure possible. Les travaux de validation devraient débuter dans les meilleurs délais pour les motifs suivants : En agissant de la sorte, on obtiendra un volume de données important. Les travaux d'étude ont révélé que l'ionosphère avait une influence déterminante sur les performances du SIS du GNSS, en particulier pour ce qui est de l'intégrité, de la continuité et de la disponibilité du signal. L'activité ionosphérique devant atteindre un pic périodique en 2000/2001, il y aurait lieu d'entreprendre les opérations de collecte dans les plus brefs délais, de façon à pouvoir ainsi recueillir des données réelles dans les plus mauvaises conditions ionosphériques. La validation des performances anormales du SIS ne pouvant s'opérer uniquement au moyen de mesures, on aura recours aux techniques de simulation et d'analyse : Les défaillances entraînant une non-disponibilité du signal pourraient être validées au moyen du rapport RAMS et de simulations. L'analyse des situations dangereuses requiert une attention particulière, c'est-à-dire l'établissement d'un dossier sécurité. 11. VALIDATION DES PERFORMANCES DU RECEPTEUR (INTEGRE) Le signal électromagnétique étant validé, il convient ensuite de valider le deuxième élément qui influe considérablement sur les performances du système de navigation, à savoir le récepteur GNSS, considéré d'abord isolément, en tant que récepteur autonome (SAR), puis en tant que module d'une architecture intégrée. La validation des performances du système de navigation suite à l'intégration du récepteur GNSS à d'autres capteurs de navigation devient très vite extrêmement complexe. Etant donné la multiplicité des solutions possibles en matière d'intégration des capteurs, on s'est attaché, dans un premier temps, à analyser une série d'architectures intégrées dans le but d'en déterminer l'adéquation. La démarche adoptée pour la validation d'un récepteur GNSS (intégré) s'articule en trois étapes : définition de scénarios de validation ; définition d'un environnement de simulation ; validation de l'environnement de simulation. La première étape consiste à élaborer les scénarios d'essais requis pour la validation des récepteurs, autonome et intégré. Les scénarios de validation du récepteur autonome se fondent principalement sur les essais décrits dans les MOPS [RD-02], lesquels sont préalablement soumis à une analyse critique. Des scénarios d'essais spécifiques sont ensuite définis pour les récepteurs intégrés. La deuxième étape consiste à élaborer, sur la base des scénarios d'essais définis, l'environnement de simulation nécessaire à la réalisation des tests. Cet environnement de simulation doit, lui aussi, être validé afin de s'assurer qu'il reflète de manière précise et fiable l'environnement réel et puisse produire des résultats significatifs. Ce processus se déroule en deux temps : - D'abord, on vérifie que les modèles théoriques inclus dans le simulateur recréent effectivement l'environnement réel du récepteur. Il s'agit, par exemple, des modèles de trajets multiples et d'interférences ou bien encore des modèles d'orbites, d'erreur du satellite ou de disponibilité sélective (SA). - Ensuite, il y a lieu de valider la mise en œuvre de ces modèles dans l'environnement de simulation. 36

Validation des Performances du GNSS Résumé EUROCONTROL La seule manière de procéder consiste à comparer les résultats des simulations avec des données réelles. Selon le modèle, ce processus de validation peut exiger différents types de données réelles, comme illustré à la Figure 1. Les trois options possibles sont la réalisation d'essais en laboratoire au moyen d'un récepteur statique, la réalisation d'essais au moyen d'un récepteur embarqué sur un aéronef statique et la réalisation d'essais à l'aide de récepteurs sur des aéronefs en mouvement. Des recommandations sont formulées quant à la nécessité de procéder à des essais en vol pour valider les installations de simulations. Récepteur statique en laboratoire Modèles Récepteur embarqué sur aéronef statique Récepteur embarqué sur aéronef en vol Figure 1: Options pour la réalisation d'essais à l'aide de données réelles Les performances du récepteur en conditions nominales (sans défaillances) peuvent être validées sur simulateur. Toutefois, étant donné que les performances du GNSS sont définies en termes de précision, d'intégrité, de continuité et de disponibilité, il est nécessaire de prévoir des procédures ayant trait à la disponibilité du récepteur. La méthode d'analyse par arbre de défaillances peut constituer une solution pour ce qui est du récepteur autonome comme du récepteur intégré. 11.1. Conclusions et recommandations Les travaux de validation des performances du récepteur autonome et intégré ont débouché sur les conclusions et recommandations suivantes : Il conviendrait d'analyser plus avant la question des interférences et des trajets multiples au niveau du récepteur GNSS, étant donné que ces deux éléments exercent une influence déterminante sur les performances du récepteur, tout particulièrement en ce qui concerne les exigences opérationnelles Cat. I. La validation des performances du récepteur autonome devrait s'opérer sur la base des tests définis dans les MOPS RTCA Do 229 A (RD-02). L'examen critique de ces tests ayant fait apparaître toutefois certaines lacunes, notamment en ce qui concerne la question des trajets multiples, des essais complémentaires ont été définis et recommandés aux fins d'inclusion dans les prescriptions EUROCAE/RTCA. Il est apparu qu'il n'existait, à l'heure actuelle, aucun document internationalement reconnu qui définisse les scénarios d'essais pour les architectures intégrées. L'étude ayant établi que les systèmes de navigation par inertie offraient les perspectives les plus intéressantes en matière d'intégration, des essais spécifiques ont été recommandés aux fins de validation de l'architecture GNSS/INS. La validation des performances du récepteur autonome commande de recourir à un simulateur GNSS SIS RF. Ce dernier permet en effet de simuler les différents paramètres, tels qu'interférences, trajets multiples ou bien encore dynamique des vols, prévus dans les scénarios d'essais. La validation du récepteur intégré requiert la mise au point de modèles logiciels supplémentaires pour simuler les performances des autres capteurs de navigation. Des essais en conditions réelles, notamment au moyen d'aéronefs d'essais, demeurent nécessaires pour valider les installations de simulation. Ces essais en vol doivent permettre d'évaluer la validité des modèles logiciels intégrés au simulateur. En ce qui concerne les récepteurs intégrés, ces modèles traitent essentiellement des aspects liés aux interférences et aux trajets multiples, alors que, pour les architectures intégrées, ils visent à simuler les paramètres fournis par les autres capteurs. 37

EUROCONTROL Validation des Performances du GNSS Résumé Deux méthodes complémentaires sont proposées pour l'évaluation des défaillances du récepteur, laquelle est importante pour évaluer la disponibilité et la continuité de la NSE. La première méthode est l'analyse par arbre de défaillances des différentes architectures du récepteur. La seconde consiste à utiliser les données recueillies par EUROCONTROL dans le cadre du projet SAPPHIRE, toujours en cours. Les données SAPPHIRE seront comparées aux résultats de l'analyse par arbre de défaillances. Afin de pouvoir tirer pleinement parti du projet SAPPHIRE dans le cadre de la validation des performances du GNSS, une série de recommandations ont été formulées, qui visent à affiner ledit projet. Au nombre des améliorations proposées figure, en particulier, la caractérisation des effets des interférences et des trajets multiples. 12. VALIDATION DES PERFORMANCES DU SYSTEME GLOBAL Ce volet a pour objet principal d'évaluer la portée des travaux de validation des performances globales (TSP) du GNSS dans son application à l'aviation civile. Les exigences en matière de TSP sont déterminées par le concept de qualité de navigation requise (RNP) et définies par l'organisation de l'aviation civile internationale (OACI). Ces exigences sont exprimées en termes de précision, d'intégrité, de disponibilité et de continuité, pour les différentes phases de vol. La NSE n'est pas la seule source d'erreur dont il faille tenir compte dans les performances globales du GNSS- 1. La FTE, c'est-à-dire l'erreur introduite par le système de guidage (pilote ou pilote automatique), revêt également une certaine importance pour les applications de l'aviation civile. La FTE et la NSE déterminent, ensemble, la TSE, qui correspond à la différence entre la position réelle et la position souhaitée de l'aéronef. La FTE est tributaire d'un grand nombre de facteurs ou variables tels que la phase de vol, le système de guidage utilisé, le type d'aéronef, la charge de travail du pilote et l'environnement opérationnel. Etant donné que le GNSS-1 doit satisfaire aux exigences dans tous les cas de figure, toutes ces variables doivent être prises en considération pour la validation du système global. La stratégie de validation des performances du système global comprend les étapes suivantes : identification des différents éléments contribuant à la performance du système global ; définition de scénarios de validation ; définition de l'environnement de simulation ; validation de l'environnement de simulation. La première étape consiste à décrire le processus d'identification des différents éléments contribuant à la performance du système global. La part de l'erreur du système de navigation et de l'erreur technique de vol dans la performance du système global est ensuite caractérisée. Sur la base de ces éléments, on élabore différents scénarios aux fins de validation. La description des scénarios sert de base à la définition des exigences de l'environnement de simulation. En ce qui concerne l'environnement de simulation du récepteur (intégré), des modèles logiciels supplémentaires sont requis, en particulier pour ce qui a trait à la dynamique des vols et aux éléments associés au guidage et au contrôle. C'est la raison pour laquelle différents environnements de scénarios possibles sont proposés. L'étude préconise des vols d'essais supplémentaires pour la validation des autres modèles adjoints. Elle comporte également des recommandations quant aux essais en vol nécessaires pour valider les installations de simulations. 38

Validation des Performances du GNSS Résumé EUROCONTROL 12.1. Conclusions et recommandations Les travaux de validation des performances du système embarqué complet ont débouché sur les recommandations suivantes : L'erreur du système de navigation (NSE) et l'erreur technique de vol (FTE) déterminent, ensemble, le comportement du système global. Toute modification au niveau de la NSE et de la FTE doit être prise en considération, de même que les éventuels effets de corrélation liés à l'application du GNSS aux opérations actuelles. La stratégie proposée dans l'étude GPV pour la validation du système (intégré) se fonde sur le recours à des simulations. De fait, les simulations présentent un double avantage. Elles offrent à la fois la souplesse voulue et un degré de fidélité élevé (matériel en boucle). Elles sont également plus intéressantes du point de vue économique en ce sens qu'elles permettent de limiter le nombre des essais en vol nécessaires, lesquels coûtent relativement chers. Plusieurs architectures sont proposées en fonction du test à réaliser pour la validation des performances. Ces architectures vont de l'environnement de simulation à base exclusivement logicielle à l'environnement intégrant un récepteur GNSS physique et un simulateur de vol. Le récepteur GNSS est nécessaire pour modéliser la NSE aux fins de validation des opérations de catégorie I, où les interférences et les trajets multiples contribuent le plus à l'erreur du système. Le simulateur de vol sera utilisé pour recréer les effets du guidage de l'aéronef au moyen des signaux GNSS. Si l'essentiel du processus de validation peut s'opérer par voie de simulations, des essais en vol demeurent néanmoins nécessaires. Ces essais en vol ont pour objectifs, primo, de valider l'environnement de simulation pour toutes les phases de vol et, secundo, d'apporter la preuve des performances globales du GNSS aux usagers de l'aviation civile. Bien que l'atc ne fasse pas, à proprement parler, partie des paramètres de performance globale d'un aéronef, il peut être intéressant d'évaluer la capacité du contrôle de la circulation aérienne à faire face aux défaillances majeures du GNSS. Une architecture propre à permettre un telle évaluation est donc proposée. L'architecture recommandée, sur la base d'une estimation des coûts, des travaux de validation requis et des avantages potentiels en terme de performances, consiste en deux ou, dans le cas où trois capteurs INS ont été précédemment installés, trois récepteurs intégrés GNSS/INS, non couplés. 39

EUROCONTROL Validation des Performances du GNSS Résumé 13. STRUCTURE DE LA DOCUMENTATION Les résultats de l'étude ont été rassemblés sous la forme d'un dossier comportant quatre parties, chaque partie étant constituée de plusieurs volumes. Le présent rapport donne une vue d'ensemble du dossier. La structure et la teneur des autres rapports sont indiquées ci-dessous. Ces documents sont disponibles sur papier et sur CD-ROM. Ils peuvent être obtenus, sur demande, auprès du Programme GNSS EUROCONTROL. Intitulé du document : Validation des performances du GNSS Partie 1 : Synthèse Partie 2 : Validation des performances du signal électromagnétique du GNSS Volume I: Synthèse des travaux de validation des performances du SIS du GNSS Volume II: Validation des performances du SIS GPS/GLONASS Volume III: Validation des performances du SIS des satellites du complément géostationnaire Volume IV: Modes de défaillance du SIS Volume V: Validation des performances du SIS du GNSS Partie 3 : Validation des performances du récepteur (intégré) Volume I: Synthèse des travaux de validation des performances du récepteur (intégré) Volume II: Définition de l'environnement de simulation du récepteur autonome (SAR) et des expérimentations connexes Volume III: Validation de l'environnement de simulation du SAR Volume IV: Analyse par arbre de défaillances du SAR Volume V: Définition de l'architecture du récepteur intégré (IR x ) Volume VI: Validation de l'environnement de simulation du IR x Volume VII: Analyse par arbre de défaillances du IR x Volume VIII: Recommandations relatives à la réalisation d'essais en vol aux fins de validation des performances du récepteur (intégré) Partie 4 : Validation des performances du système global Volume I: Synthèse des travaux de validation des performances du système global Volume II: Définition de scénarios Volume III: Part de l'erreur du système de navigation dans la performance du système global Volume IV: Part de l'erreur technique de vol dans la performance du système global Volume V: Définition de l'environnement de simulation du système global Volume VI: Mode recommandé d'intégration des capteurs Volume VII: Recommandations relatives à la réalisation d'essais en vol complémentaires 40