EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE

Transcription

1 EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE A SAFE APPROACH TO TRANSITION: KEY ELEMENTS FOR TRANSITION SUCCESS EEC Report No. 405 Project D20/2005 Issued: October 2006 The information contained in this document is the property of the EUROCONTROL Agency and no part should be reproduced in any form without the Agency s permission. The views expressed herein do not necessarily reflect the official views or policy of the Agency.

2

3 REPORT DOCUMENTATION PAGE Reference: EEC Report No. 405 Originator: EEC - SAS (Safety Analysis and Scientific) Security Classification: Unclassified Originator (Corporate Author) Name/Location: EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 F Brétigny-sur-Orge Cedex FRANCE Telephone : +33 (0) Internet : Sponsor: EATM (DAP/SAF) Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency 96, Rue de la Fusée B-1130 Brussels Telephone: TITLE: A SAFE APPROACH TO TRANSITION: KEY ELEMENTS FOR TRANSITION SUCCESS Authors Paul Humphreys, EEC Barry Kirwan, EEC Sven Ternov, Swedavia/LFV EEC Contact Paul Humphreys, EEC Distribution Statement: (a) Controlled by: Head of SRT (b) Special Limitations: None Date 10/2006 Project D20/2005 Pages vi + 24 Figures 1 Tables 4 Task No. Sponsor Annexes -- References 4 Period June-Dec 2005 Descriptors (keywords): Safety management, transition to operations, new ATM systems, future ATM technology. Abstract: This report considers the key safety elements to be considered during the Transition period to operational implementation of an ATM system or sub-system, based on best practice advice gained from a number of ANSPs who have been through major transitions. This Transition period, between the end of the design phase and the O -date, when a new system or sub-system goes operational, is also called Transfer to Operations, and in other industries is sometimes known as Commissioning. In theory, the system has been well and fully designed, and remains to be built, put in place, people trained, interfaced with local data, and switched on. Of course in reality, Transition rarely proceeds so smoothly, and the actual process can take years to get the system ready for actual operation. There is much safety information and guidance written on the design phase, and on the operational and maintenance phase, but little on the Transition phase. Whilst not directly linked to safety, a professional and organised approach to transition provides a better environment in which to monitor, detect, and manage safety issues in a timely and effective manner. This report describes a stepped transition process (called KEEP-TIME ), indicating the roles and responsibilities of the various actors, mapped against the activities which are to be carried out prior to O -date, which may be incorporated into the EUROCONTROL Safety Assessment Methodology documentation. More than fifty practical guidance points are given on what to do and not do, in order to achieve a safe and timely transition process.

4

5 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL TABLE OF CONTENTS LIST OF FIGURES... V LIST OF TABLES... V REFERENCES... VI 1. INTRODUCTION OBJECTIVES CONTEXT TRANSITION MANAGEMENT APPROACH TO BEST PRACTICE GUIDANCE DEVELOPMENT RESULTS SAFETY MANAGEMENT IN TRANSITION DIRECT SAFETY CONSIDERATIONS INDIRECT SAFETY CONSIDERATIONS THE KEEP-TIME TRANSITION FRAMEWORK ROLES AND RESPONSIBILITIES SCHEDULE TO O- DATE CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS RECOMMENDATIONS FRENCH TRANSLATION (TRADUCTION EN LANGUE FRANÇAISE)...21 LIST OF FIGURES Figure 1: Transition Framework LIST OF TABLES Table 1: Staff interviewed for Transition Best Practice Survey... 4 Table 2: Nature of Transition Experience... 4 Table 3: Roles & Responsibilities Table 4: Schedule to transition / CHECKLIST Project D20/2005 EEC Report No. 405 v

6 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success REFERENCES [1] EUROCONTROL (2003) Age, Experience and Automation in European Air Traffic Control. Edition 1.0 HRS/HSP-005-REP-02, EUROCONTROL, Brussels, Belgium. [2] EUROCONTROL (2004) Age, Experience and Automation in European Air Traffic Control Survey in the ECAC Area. Edition 1.0 HRS/HSP-005-REP-05, EUROCONTROL, Brussels, Belgium. [3] EUROCONTROL Safety Regulatory Requirement (2001) - ESARR 4 Risk Assessment and Mitigation in ATM. Edition 1. EUROCONTROL, Brussels, Belgium. [4] EUROCONTROL Safety Assessment Methodology, System Safety Assessment process. vi Project D20/ EEC Report No. 405

7 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL 1. INTRODUCTION 1.1. OBJECTIVES Most safety assurance activities focus on the design phases, and the operational phases in the life cycle of a system. However, there is a phase often known as Transition in which the system is made to be ready for operation (this is also known as the Commissioning phase in other industries such as the nuclear power industry). In a review of the EUROCONTROL Safety Assessment Methodology (SAM) it was noticed that there is room for a safety assurance technique during this phase for new Air Traffic Management (ATM) system developments. For example, a number of new Air Traffic Control Centres (ATCCs) have been developed and implemented, while some States are in the process of gaining new ATCCs, and some that plan to upgrade their ATCCs in the future, e.g. with new Human Machine Interfaces (HMIs). Typically, the time period between finishing the design and design safety assessment, and going operational can be several years. During this period, there may be small but significant changes to the system design and ways of working. Additionally, experience with making the system work may give rise to safety insights, whether these are safety concerns, or ways to operate more safely. The question is how to manage this process of migration to a new system whilst capturing such safety insights and building them or their implications into the Safety Management System (SMS) or safe operational practices for the new centre and its operational workforce. There is therefore a need for an approach which engenders and captures such safety insights, and relates these into the finalising training and procedural systems prior to the system going operational. Such an approach should be integrated with the testing and commissioning processes that are trying to get the system ready for operation, and should build on previous safety case work where appropriate CONTEXT Air traffic management systems worldwide are undergoing a number of changes. There may be new operational centres for air traffic management, with new radar displays and electronic strips, for example, or the introduction of a new tool such as an arrival manager or a medium term conflict detector. In such cases there will have been a lengthy concept exploration period to make sure the concept can add value, then an equally lengthy design and development process with input from many stakeholders, and safety assessment will have accompanied these stages. At the end of this extensive exploration and design process, most people (though maybe not all) will have been convinced it is needed by the business and safety cases, and they want it installed and made operational. But before that happens, it must be built and made ready for use, or in the case of a tool, translated from a prototype into an industrial tool and interfaced with an existing system. This report deals with the period known as Transition, between the end of the design and the O - date, when a new system or sub-system goes operational. This period is also called Transfer to Operations, and in other industries is sometimes known as Commissioning. In theory, the system has been well and fully designed, just needs to be built, put in place, people trained, interfaced with local data, and switched on. Of course in reality, Transition rarely proceeds so smoothly, and the actual process can take years to get the system ready for actual operation. Project D20/2005 EEC Report No

8 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success Any system can be considered in terms of the SHELL model (Software, Hardware, Environment, Liveware, and Liveware 2 [which means teamwork]). During Transition there can be many aspects that may not go smoothly, for example: Software (also meaning procedures) - an issue with SW is that a lot of verification as well as validation cannot be conducted in a simulated environment and therefore a transfer into operations phase is necessary to achieve such V&V. However software bugs that prevent the system from working as planned in large current air traffic control systems these can take months or even years to iron out completely or to an adequate level. Procedures, often developed generically for the to-be-operated new system, will also need significant tailoring to the local needs of the system. Hardware for a new Air Traffic control centre (ACC) for example, the controller workstations, and various supporting equipment and workstations, the general building and its services needs, will take time to get ready and made to work without, for example, intermittent power distribution failures, and getting the workplace ergonomics right. For smaller projects (e.g. a new tool for an existing ACC), locating the hardware and interfacing it without disrupting operations too much are key considerations. Another issue with HW is the need to have some intermediate components to manage the old and the new interfaces, requiring some specific piece of HW. Environment - a typical environment problem that can occur during Transition concerns the ambient lighting and its effects on radar screens, particularly the currently prevalent lighter screens and a more general lighter environment (as opposed to older control centres with darkened rooms and green on black radar displays). Reducing indirect and direct glare and reflections that can affect controller performance in what is a highly visual task requiring visual acuity can be a difficult problem. This difficulty is compounded when different screen types are used (e.g. a touch screen telephone pad) which are at different angles or have different reflectivity. Another environment issue which could be problematic and which should be considered in transition planning, is the location of new installations such as the Operations room or Training establishment. The ease of availability of staff, either for training or for shadow mode trails can be affected by physical location. Liveware controllers must get used to the new system, its look and feel, and adapt it to their working practices and to the local traffic patterns and demands. Above all, they must be able to trust the new system or tool. There will inevitably be problems with any new installation, and so there is a fine choice to be made concerning when to begin training controllers with the new system too soon and they could lose trust in it, and too late and they will not have sufficient time to adapt to it (this is discussed in more detail later). Another aspect to consider from the Human Resources angle, is that older controllers may not adapt as fast to a new system as younger ones. There is a definite effect of age (Rothaug, 2002) in such situations, and it will need consideration to ensure that the total controller workforce is able to adapt to the new system. After all, the older controllers are also the most experienced. Teamwork With a new system there are several teamwork issues. Moving from a conventional ATC radar system to a new computerised one will entail more technical support functions than before, and probably the need for a System Support Centre to ensure the system keeps running properly. The operational organisation will have, and will need, more (or at least different) technical support than it is used to. At the controller working level, the new system or tool may well alter the exact task distribution (and thus the workload distribution) or even roles between tactical and planner controllers. For example, this could be due to a shifting of certain tasks to the planner controller, or it could even be something as simple as a new workplace layout that puts their two principal radar screens further apart, so that they can no longer monitor each other s tasks and synchronise them as they used to. 2 Project D20/ EEC Report No. 405

9 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL Clearly Transition can be an extensive process that can take significant time and resources to get it right before going operational. The schedule to transition checklist, described in chapter 5 is not a formal methodology, but is rather guidance material to support ANSPs in approaching transition in a structured way. The next section considers both direct and indirect safety aspects during this process. Project D20/2005 EEC Report No

10 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success 2. TRANSITION MANAGEMENT The key elements to be considered during the Transition stage are based on best practice advice gained from a number of ANSPs who have been through major transitions. Whilst not directly linked to safety, a professional and organised approach to transition provides a better environment in which to monitor, detect, and manage safety issues in a timely and effective manner APPROACH TO BEST PRACTICE GUIDANCE DEVELOPMENT Literature was reviewed, but in fact little was found that related to this issue. An airline and two nuclear power plant sources were reviewed for their general insights into Transition management. In addition, a total of five ANSPs participated in the project, three answering survey interviews during site visits about their transition experiences, and two giving comment on the developing Transition guidance, the latter helping to consolidate the guidance. One of the ANSPs (ANSP 3) was also used to beta-test the survey questions, before sending the survey to another of its sites. The site visits were carried out during the autumn of 2005, and the consolidation interviews at the end of The range of personnel interviewed and the nature of the transitions are shown in Table 1 and 2 below. Table 1: Staff interviewed for Transition Best Practice Survey INTERVIEWED STAFF ANSP1 Site A ANSP 2 Site B ANSP 2 Site C ANSP 3 Site D Controllers, not involved in project Controllers, involved in project Middle management Senior management Total Table 2: Nature of Transition Experience ORT item Site A Site B Site C Site D Nature of transition Transition to the new system in 2001, which already had been implemented at the countries airports, i.e. a system that was already working. Transition period nine months. Transition to new system in The site is an en-route and approach centre, interfacing with oceanic airspace Transition to new system in 2003, at the same time as site B. Site C is mainly TMA and approach Transition to new system, contract signed 1997, went operational March Stripless system. Site D is mainly en route and TMA, Overall, 31 interviews were carried out, not including beta testing activities. Staff were interviewed individually in two of the sites, either individually or in pairs in another, and in two groups in ANSP 1 (senior management, middle/project management). 4 Project D20/ EEC Report No. 405

11 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL The consolidation interviews involved interviews with personnel from two additional ANSPs for one day each. Personnel at these sites had seen preliminary best practice material based on the prior surveys. Geographically speaking, the five ANSPs in total represented Scandinavia, western Europe and one southern European member state (within the timeframe no central or eastern member state was able to take part in the study). Each participating ANSP had had significant transition experience in recent times. While the interviews focused on operational controllers, operations managers and safety managers, the inclusion of maintenance staff at each stage in a transition process should not be overlooked RESULTS Generally, all the participants understood the point of the study, and what it was trying to achieve, and the two ANSPs who reviewed the initial results found them useful, and added material to the body of knowledge that had been gleaned. What was clear however, was that there was no simple ready-made model of transition, and no obvious One-size-fits all approach. Nevertheless, there were various ideas about pitfalls and how to avoid them, which could facilitate safety management during Transition. What follows is therefore an attempt to construct a general framework for Transition, along with guidance to make such a complex endeavour a success. Project D20/2005 EEC Report No

12 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success 3. SAFETY MANAGEMENT IN TRANSITION 3.1. DIRECT SAFETY CONSIDERATIONS The direct safety aspects of concern relate to Safety Requirements (SRs) developed during the design phases of the system via earlier safety assurance phases (Functional Hazard Assessment [FHA] and Preliminary System Safety Assessment [PSSA]), to ensure the system will perform adequately safely, thus enabling the overall ATM system to reach its target level of safety. These aspects are already addressed in the Safety Assessment Methodology (SAM), via the System Safety Assessment (SSA) process. Put simply, the FHA and PSSA phases have been relatively predictive in nature, and during Transition the design becomes a real system, and it is the job of the SSA to ensure that the Safety Requirements (SR) do their job in the actual system. It may be useful at the end of the PSSA to highlight critical assumptions and safety requirements that would need special attention during transition and post transition monitoring period. Whereas the PSSA will naturally prioritise SRs, it may be useful for the key safety assessors in a design stage to highlight the SRs that warrant most attention. However, there is also a reasonable likelihood that during transition, and the refining of the designed system towards actual operations, it is found that the system must be tailored to the detailed local demands and context. It is also difficult during design to anticipate everything, and all the possible interactions that may happen in all the local scenario variants. There may therefore be a few surprises in terms of how systems actually perform when being implemented and tested. This may mean altering some aspects of the original design intentions to make the system work as intended. Such alterations in turn may in some cases necessitate re-consideration or adjustment of certain Safety Requirements. Such adjustment must be carried out carefully and all considerations recorded transparently, and in such cases it may be necessary to carry out re-assessment (e.g. at the PSSA level) to confirm that the new adjusted SR satisfies the overall safety case given the modified context (operational environment description). If during the design safety assessment assurance levels (procedural, human or software) have been assigned, this will also give general indication of the degree of safety attention required. The general aims of safety at this phase are therefore as follows: 1. Ensure Safety Requirements are effected as they were intended. 2. Analyse deviations or necessary changes or side effects. 3. Ensure prior safety assumptions are upheld and not violated. 4. Analyse risk impacts of any deviations. 5. Monitor safety parameters to see if previous estimations of frequency and severity are upheld if not, re-estimate risk and develop mitigations. 6. Detect and analyse new safety issues that were not identified by the previous safety work. Additionally, there is arguably a seventh aim: 7. Identify safety opportunities where safety could be improved, e.g. by better and more adaptive methods of working. 6 Project D20/ EEC Report No. 405

13 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL This latter aim recognises the potential safety opportunities due to a synergy between the new system design on the one hand, and the ways of working and ingenuity of the controller work system on the other. These aims mean that safety management during the transition period must encompass the following capabilities and functions: 1. A detailed understanding of the prior safety documentation, including the operational environment descriptions, safety assessments, assumptions, uncertainties, sensitivity analyses, and resulting safety objectives and requirements. 2. A testing, tracking and monitoring system for all identified safety requirements. 3. Objective (performance-based) and Subjective (Controller acceptability-based) criteria for each SR test, to ensure it fulfils its designers and safety assessors intent, and also that it fits with the new safety operators (controllers) safety approach. 4. A generalised monitoring system to detect any deviations, anomalies, or other occurrences during system testing and training that could have safety implications. 5. Safety re-assessment capability, for any deviations or changes in assumptions etc., to update the safety case and ensure that changes do not erode previous safety margins nor introduce side-effects nor failure dependencies with other system components. This capability can also take the role of refining safety case calculations, estimates and assumptions, which were significantly uncertain (e.g. having to use expert judgement), particularly where such estimates were rated highly in the safety case sensitivity analysis. 6. Ability to generate new mitigation strategies should these be found necessary (e.g. if a safety requirement test finds a safety measure is lacking, or if new safety issues are discovered). 7. Capability to capture new safety-enhancing ideas from testers, controllers, and others during the Transition period. 8. A transition Safety Policy that clarifies the role and priority of safety during Transition. 9. A Transition Safety Plan, describing in more detail and in more concrete terms for the system/tool being implemented and installed, the sequence and inter-relationships of the various safety activities, processes, outputs and success criteria needed to ensure safety in this critical phase. 10. A sufficiently empowered safety management structure, resources and expertise to enact the safety activities during Transition, integrated with the Transition Management team, and interfacing with the regulatory authorities. 11. A final Operational Readiness Safety Case as part of the SSA, to show that the transition has occurred safely, with all the activities recorded. 12. A safety culture of the organisation that permits sufficient attention to safety matters given the inevitable increasing commercial and time pressures to move to O-date, to the extent that the Project Manager will not bow to such pressure if there are key residual safety concerns. Project D20/2005 EEC Report No

14 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success 3.2. INDIRECT SAFETY CONSIDERATIONS There are also indirect safety aspects to consider during Transition. The first of these, and the most important, relates to inevitable commercial time pressure at this stage of a system s development, and the adverse effects this can have on safety. This time pressure element can be seen as influencing a number of the elements listed above, and can lead to a Get it done or We must meet O-date mentality and culture that puts safety second or even further down the real working agenda during transition, for example 1 : Limited time to test the effectiveness of Safety Requirements, leading to possible shortcuts or inadequate testing criteria or testing resources (e.g. safety finds it hard to gain access to simulation testing due to software testing priorities etc.) This problem can be countered by safety providing clear testing requirements for each SR, embedded within a SR testing schedule. Pressure when classifying new safety issues identified, to de-rate them in terms of their safety classification, so as not to hold up the project (e.g. classifying it as a workload issue rather than a safety issue, in ESARR 4 terms, or giving it a low frequency classification). This can only be avoided by a disciplined approach to safety classification. When finding safety performance is not meeting criteria, pushing the safety emphasis onto future training and procedures, possibly during the operational phase ( they ll be able to manage ), rather than for example tackling hardware or software-related human-machine interface issues that in reality will never be resolved by training. Insufficient resources for safety management, in terms of either people or expertise (including controller expertise), are allocated to safety (e.g. the controller experts that have worked formerly with the system design are all working on transition and training, and new people inexperienced with its intended operation are assigned to safety tasks). Inadequate prioritisation of safety in the transition management structure (e.g. safety is left till last on key meeting agendas; safety representation is poor in key meetings; safety manager feels isolated and unsupported when raising concerns). Failure to capture good safety ideas because no-one has time to listen, and such ideas might cause additional delay. Incomplete safety case evidence, because short cuts had to be taken due to insufficient resource and time available (e.g. we were supposed to do a HMI assessment, but there was never enough time, so a few guys looked at it and said it seemed ok ). Referral to external safety contractors who will sign off certain specific safety tasks without knowing how these safety tasks relate to other aspects of system safety. It can be seen that whilst Transition Management is not obviously within the scope of safety, it can affect the latter s ability to perform its duties. For this reason, this report has focused on factors and measures to try to achieve a successful and measured Transition that is not under undue time pressure. Of course there will always be some time pressure, because these are commercial systems that are being implemented in a commercially competitive environment. But there are pitfalls that can be avoided, and measures that can be taken, that can help to ensure a more stable process, so that there will be sufficient time for safety. These potential keys to successful transition management are discussed in depth in the next section. 1 These examples are anonymous but not fictional they have all occurred either in the ATM or other industries during Transition or Commissioning phases 8 Project D20/ EEC Report No. 405

15 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL 3.3. THE KEEP-TIME TRANSITION FRAMEWORK Figure 1 is a framework developed to structure the guidance gleaned from the various interviews. The six main concepts can fit into an acronym KEEP-TIME [for safety], which is indicative of the overall philosophy of avoiding becoming too time pressured during this phase, so that safety, and indeed other aspects, may suffer. Below is a generic description of KEEP-TIME. The specific application of the framework to an ATM transition is described following Figure KNOW 2. ENGAGE 3. PREPARE 4. TEST 5. INFORM 6. MONITOR the nature of the change (novelty; size; complexity; risks; boundaries), Transition expertise; Safety expertise motivate people; create user groups; ensure regulator interactions. plan and resource, train core group and main group, train to criterion and beyond, fault management training. safety requirements & objectives, assumptions and safety opportunities, trials, shadow-mode, simulations; wild simulations; fault injection; safety recording and analysis; HF testing and debriefing; etc. communication mechanisms, trust; feedback; line ops involvement. professionalism, organisation to monitor whole process; decision-making; safety and performance indicators; issues logs; etc. Project D20/2005 EEC Report No

16 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success Figure 1: Transition Framework The six elements can be applied to an ATM Transition as follows: KNOW (who you need to involve and what you need to do before planning the transition) Contract/Finance Department - the terms of the contract with the new system supplier can be a decisive factor in Transition. There will undoubtedly be changes required to the system during transition. If these suffer heavy penalty clauses, this will significantly constrain the inhouse management team. New systems contracts negotiations must be informed by transition experience, to protect the interests of the service provider (ANSP). Project Management Support - professional project management expertise is crucial to the success of Transition. Project location should be physically located at the site for the transition this keeps the project management much more up to date and keeps people focused remote control and management is therefore not recommended for Transition. Create expert groups, and be sure that their roles are well defined. Define clear roles and responsibilities. Buy in Transition expertise where required, as well as human factors and training expertise if these do not exist sufficiently in-house. Getting the new system up and running will require significant technical expertise and system knowledge buy this in if necessary, but ensure the knowledge is transferred inhouse during transition. 10 Project D20/ EEC Report No. 405

17 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL With major new systems upgrades, there will probably be more technical support than before, which will continue during operation and maintenance. Ops and tech support will need a new level of mutual understanding to enhance optimum system performance, so these groups should interface with each other regularly at an appropriate level. ENGAGE (build and organise your team) Recruit interested controllers from the site early in the process (specifications phase), and use these controllers as trainers later on. Appropriate involvement of airlines/pilots should be arranged if the transfer into operations significantly relies or impacts on them, e.g. datalink, independent parallel approach, etc. Engage other controllers on hands on exercises as soon as possible, but not before the system is sufficiently mature (shadow and simulator). Engage controllers in adapting the new system to working practices, with an emphasis on developing best working practices with the new system. Do whatever else you can to obtain controller confidence in the new system, and thus avoiding rumours. Have a customer. If the project concerns more than one site, be especially careful to bring on board management and staff from both sites. Treat controller feedback promptly, especially where safety issues are raised. PLAN and PLAN MONITORING Develop an operational transition plan/prepare/monitor in concordance with involved expert groups, and simulate it, and refine it, before O-day. Keep on with this on a regular (monthly) basis. Plan for training: o develop a training plan, with clear definitions of trainers training, and their responsibilities, o don t rely too much on Computer-Based Training (CBT), o develop training criteria, so you know when controllers are proficient on the system, and ready to go operational, o once a core training team has been fully trained, final training of all controllers should occur not too far from O-date, particularly if the system is a radical change from current operations. Otherwise there is a risk of negative transfer of training from the new system to the existing one. If unsure, seek training expert advice, o define local management support responsibilities to trainers. Plan for contingency: o develop contingency plans, particularly for final O-date transition to operation, o although there is some debate about this, it may be best to plan O-date itself during a relatively low traffic seasonal workload period, o shadow-mode back-up in the first few weeks of operation may also be considered as a contingency measure, Project D20/2005 EEC Report No

18 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success TEST o a freeze period, e.g. a month or so before O-Date, where no further changes are required, can be a good way to prepare and also to allow the final building of confidence for Transition, o introduction of a tool during non-busy periods (e.g.the night-shift) only for an initial period of time is also another option, o if there are safety concerns, or operational ones, traffic may be reduced (and/or manning increased) for the first few months of operation, o Contingency plan for such phase [ability to revert to a previous state/situation, safely, rapidly and smoothly (that is part of PAL4)]. Carry out all necessary tests 2 and de-bugging activities, including system performance demonstrations (SPDs) to ensure Safety Requirements and Safety Objectives are implemented effectively. o Suppliers may well supply their own test data related to safety and operational aspects. For safety these need to be confirmed, since supplier tests will probably have been tested in laboratory conditions by system engineers the system needs to be tested in realistic local conditions with controllers ( testing in the wild ). o Develop objective (performance-based) and subjective criteria for all tests, develop pass/fail criteria. o For subjective criteria, at least six controllers are needed (less is too small a sample for confidence in their evaluations). o Testing should progress from basic, to nominal mode, to fault mode, degraded mode, and emergency operational scenarios. o Use a representative sample of controllers for testing, this sample increasing in size as O-date approaches. o In later stages of testing (when a reasonable level of system maturity and controller competence in the core training team has been reached), begin fault insertion training to see how the system responds and how the controllers can deal with such faults. Carry out Human Factors checks and audits using standard checklists for workplace layout, environmental adequacy (light, noise, thermal conditions, and in particular lighting and legibility considerations, and speech comprehension / interference from adjacent controller workstations). If earlier Human Factors Cases or Safety Cases have highlighted issues such as potential workload or situation awareness issues, or teamwork issues, these should be tested locally to determine if they are still present and, if so, to develop effective mitigations. Shadow-mode (observing but not handling live traffic) may be used during final testing phases, and live traffic handling with shadow mode back-up may also be considered, the latter particularly for the implementation of new system tools. 2 The level of test should be commensurate with the assessed risk. A test may therefore range from real time simulation or shadow mode/live trials to observation, demonstration and inspection. 12 Project D20/ EEC Report No. 405

19 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL INFORM Inform other controllers as soon as possible concerning the project status. o o Use intranets and weekly briefings to keep people up to date with progress. Maintain up-to-date documentation on Transition progress, tests, results, deviations, mitigation measures, etc. o Let controllers who are testing and adapting the system, and developing training, know the hazards, system failures, human errors and recoveries already identified in the FHA/PSSA, including their likely (estimated) frequencies of occurrence. Safety should be an integrated part of the Project Management, the safety manager reporting directly to the overall Project Manager on safety progress and issues, as well as presenting and participating at all key Transition Project Management meetings. Ensure that you have a regular formal dialogue with Regulator. MONITOR (after transition) Implement and encourage a formal safety reporting process, and define the role of the safety manager in the project. Create a channel for anonymous reporting whether of safety issues or operational concerns. All human error reporting should use a common descriptive language, such as the HERA system developed by EUROCONTROL, or equivalent. Controllers should also be encouraged to record where they encounter problems and how they resolve them, and such latter information may then be fed towards training development to feed into working practice development. Have a continually updated Issues list, with all outstanding issues graded according to importance and urgency, with date of origin and actions placed on personnel. Monitor confidence and trust in the developing system, via both formal (questionnaire, interviews, and group fora discussions) and informal means (e.g. open intranet comments and concerns website ). Monitor training progress if there appear to be difficulties, this may be indicative of a more serious problem in system understanding or design. Monitor the progress of older (>40 years) controllers during training. Offer additional facilities where appropriate (e.g. self-learning, additional time on simulator, partner with a more rapidly advancing controller, tutoring, etc.). Monitor safety-related events during training, to see if safety data from PSSA can be refined. Project D20/2005 EEC Report No

20 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success 4. ROLES AND RESPONSIBILITIES The roles described below are probably typical of most Service Providers. ANSPs with different organisational structures can nevertheless use the model and adapt it as necessary. Table 3: Roles & Responsibilities PERSON/GROUP ANSP Management Board Project Manager (PM) Site General Manager (GM) Safety Director (SD) (if this position exists in a given ANSP s organisation) Site Safety Manager (SM) Site Operations Manager (OM) RESPONSIBILITIES Overall responsibility for Transition Project Overall management and coordination of project, reports to ANSP Management Board The customer for the new system, responsible for on-site management, reports to PM Member of ANSP Management Board. Formally approves all safety documentation, manages site Safety Managers, responsible for dialogue with Regulator. Manages and coordinates all safety activities, prepares, approves local safety documentation, reports to SD. If no SD, has responsibilities above and reports to PM. Responsible for all operational aspects of project on site; procedures, staff, transition training, expert groups. Reports to GM. Site Engineering Manager (EM) Project ATCOs Training Manager Etc. Responsible for all engineering aspects of project; installation, testing. Reports to GM. Participate in expert groups as required; operational, procedures, HMI etc. Prepare and manage training, provide feedback to OM, GM, SM. Develops and implements training plan for the new system Other positions as required/appropriate 14 Project D20/ EEC Report No. 405

21 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL 5. SCHEDULE TO O- DATE The efficient management of the safety activities relies very much on the quality of project management in place for the total project; from contract definition to procurement, to installation and operation. The Table 4 indicates the schedule of project key events, the safety related transition activities at each stage in the process, together with the roles and responsibilities of the various players. Table 4: Schedule to transition / CHECKLIST Note: the time line indicated in this table is appropriate for very large projects. For smaller projects, the scale of the schedule would be adjusted accordingly. Time line/ Keep- Time elements Project Process Transition Activity Roles & Responsibilities 3-4 years Contract with manufacturer ANSP Management Board (MB) KNOW Establishment of project Very detailed project description, with project plan, safety plan and organisational plan, from now on to O-day. Give much consideration to the physical location of the project management in relation to the actual site for the transition. Findings indicate that a major success factor is to locate the project physically in close relation to the site for the transition. When appointing a project manager this person must have (documented) competence for this. This topic is one of the most crucial for success of the project. If this competency is not in-house, buy it in. Project Manager (PM) reporting to MB Safety Manager (SM) reporting to PM or Safety Director (SD) if one exists There will probably be a project management board (PMB) in place including OPS, ENG, HR, Finance & Infrastructure Heads 3 years ENGAGE INFORM Specifications Involve the end user group early, i.e. interested controllers from the site. PM, site General Manager (GM), OPS Manager (OM) 3 years PREPARE Specifications review Involve the end user group early, i.e. interested controllers from the site (R&D controllers: these controllers will be the future trainers). Manufacturers may have limited knowledge about ATM. Specs may be seriously misunderstood. This can result in months sending the specs back and forth. ATCOs organised in groups PM, OM, Eng. Manager (EM) full-time project ATCOs/engineers Project D20/2005 EEC Report No

22 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success Time line/ Keep- Time elements Project Process Transition Activity Roles & Responsibilities Sort things out with the manufacturers during a number of face-to-face meetings. 2 years PREPARE TEST Provisional acceptance (FAT) Mainly manufacturer, together with project management, and R&D controllers. This is a feedback that you and the manufacturer had understood each other during the spec review process. Major misunderstandings should be sorted out by now. PM, OM, EM, ATCOs, SM 1½ years INFORM Site acceptance test (SAT) After FAT, the prototype changes to a product, and is conveyed to the engineering and OPS department at location for further handling. GM, OM, EM, site project teams ENGAGE Make sure that the local ENG and OPS get involved, and that the roles and interfaces are clearly defined between these and the project. site project teams Now the site, represented by ENG and OPS, may be called customer. GM is the customer ENGAGE INFORM Involve Regulator latest at this stage. The safety manager may act as liaison. The Regulator should be presented with the safety plan, and overview of how to handle the safety case. Feedback from the Regulator will be valuable. Note: refer to ESARR1 requirements for Regulator involvement. PM, SM, Regulator (Reg) Regular meetings with the Regulator concerning status reports should be planned. SM, Reg 1 year TEST Validation /Testing Chaired by project management and R&D controllers, but involving other controllers from the site. PM, OM, EM, ATCOs Make detailed scripts for the validation exercises, the why s, what s and how s, so that controllers have a clear picture of what is expected from them. Training ATCOs 16 Project D20/ EEC Report No. 405

23 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL Time line/ Keep- Time elements INFORM ENGAGE Project Process Transition Activity Roles & Responsibilities Give proper feedback, and use follow-up forms for the controllers, where they can write down their concerns, both concerning the planning of the validation exercises, and also concerning safety issues they experience. These controller concerns are swiftly taken care of by line management, and proper feedback to the reporters is given. OM, SM To keep the controllers in the loop is crucial for their motivation working in the new system. Consider the validation exercises as some kind of training. Do not start on this, with non-project controllers, before the system is sufficiently mature. Otherwise, it is pretty certain that you lose controller confidence in the system. SM, Reg PREPARE The dialogue with the Regulator is going on, on a regular basis. The Regulator is provided with continuous information on the progress of the validation, and Regulator requirements are discussed and met. Training ATCOs The project controllers will now start their role as trainers, and will begin to prepare the training programme. These controllers are the only controllers that have sufficient knowledge about the system at this time. GM(s) It is imperative that they get the support from local management for this difficult, and sometimes controversial, role One year TEST MONITOR Training Starts in parallel with the validation phase. The R&D controllers have prepared the training program. Training ATCOs, OM Project D20/2005 EEC Report No

24 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success Time line/ Keep- Time elements Project Process Transition Activity Roles & Responsibilities Their role as trainers are clearly defined by local management and the project, and local management, is continuously following up on trainers progress and concerns, and is actively supporting the trainers. Do not start hands on training in the system, before it is sufficiently mature. Otherwise, you may jeopardise controller confidence. Concerns for the trainers, to be followed up by the line management, can for instance be: What do we do with controllers, who do not show up for sessions? What do we do with controllers, who obviously failed in the former lessons? What is our role in certifying controllers, that they actually meet the requirements for handling traffic in the new system? The value of CBT is limited. Do not overestimate the value of this. Controllers want hands-on training, not a CD. To get confidence in the system, two suggestions turn up, not mutually exclusive: 1. Use many shadow runs, but staff them properly, so that you can train on aircraft transfer between sectors. 2. Use more off-line (simulator) training, with high traffic load; so that they really can feel that the system can cope with this traffic volume. Minus one month MONITOR INFORM Freeze (approx. four weeks before O-day) The system is frozen. No changes are allowed, apart from minor maintenance (defined in a document). This is the last chance to get controller confidence in the system. PM, PMB If major bugs are discovered, the O- day will have to be postponed. PM 18 Project D20/ EEC Report No. 405

25 A Safe Approach to Transition: Key Elements for Transition Success EUROCONTROL Time line/ Keep- Time elements Project Process Transition Activity Roles & Responsibilities The whole site will be involved, shadowing in the new system, and if possible, doing some sharp runs in some sectors during night-time in the new system, shadowed in the old system for safety reasons. all O-day Plus one month MONITOR ENGAGE Contingency period (this can range from four to 24 weeks after O-day) Aberrations are carefully monitored by the project management and R&D controllers. Any concerns from the controllers are carefully monitored and processed. SM, GM, OM/EM, project ATCOs Controllers will have a follow up form to note their concerns. Daily this information will be processed by project management and R&D controllers, to assess the stability of the system. In the first month, on a minute operative basis the watch supervisor has the power to switch the system to contingency mode. This is prepared, and trained for, in detail (and documented). OM delegated to Watch Supervisors (WS) No return date Old system not serviceable anymore MONITOR Normal operations An incident reporting system is already implemented. Controllers will write down, on prepared forms, incidents and events which may disclose safety shortcomings in the new system. Local management will swiftly take care of these observations, and amend them, with proper feedback to the controllers. SM Project D20/2005 EEC Report No

26 EUROCONTROL A Safe Approach to Transition: Key Elements for Transition Success 6. CONCLUSIONS AND RECOMMENDATIONS 6.1. CONCLUSIONS This study has derived guidance for the period known as Transition, both to help manage a smoother transition, and also to keep time for safety management and improvement activities in this difficult period. There is not a one size fits all approach, but it is hoped that the above guidance and considerations will help ANSPs develop a robust, safe and timely transition process. For the safety manager of a transition project, this guidance needs to be considered in association with the available ESARR 4 guidance on System Safety Assessment (Transfer to Operations) RECOMMENDATIONS The KEEP-TIME for Safety approach to transition should be incorporated into the EUROCONTROL Safety Assessment Methodology as guidance material. 20 Project D20/ EEC Report No. 405

27 Sécurité de la Transition : Éléments Clés pour une Transition réussie EUROCONTROL TRADUCTION EN LANGUE FRANÇAISE SYNTHESE Le présent rapport traite des éléments de sécurité essentiels à prendre en compte pendant la période de transition précédant la mise en service opérationnel d un système ou sous système ATM. Il se fonde sur les recommandations en matière de pratiques exemplaires formulées par les ANSP qui ont connu des transitions importantes. La période de transition, également appelée passage à la mise en service opérationnel ou encore, dans d'autres secteurs, «commissioning» ou essais de mise en service, s'étend de la fin de la phase de conception à la date de la mise en service opérationnel d'un nouveau système ou soussystème. En théorie, le système a été conçu dans ses moindres détails et il reste à le réaliser, à le mettre en place, à former les utilisateurs, à assurer l'interface avec les données locales et à le brancher. Bien sûr, dans la réalité, la transition se déroule rarement de manière aussi harmonieuse, et il faut parfois des années avant que le système ne soit prêt à être mis en service opérationnel. Il existe une littérature abondante sur la sécurité des phases de conception, d exploitation et de maintenance, mais très peu de choses sur la sécurité de la phase de transition. Bien qu elle ne présente pas un lien direct avec la sécurité, une approche professionnelle et organisée de la transition génère un environnement dans lequel il est plus aisé de surveiller, détecter et gérer les questions de sécurité de manière rapide et efficace. Le présent rapport propose un processus de transition progressive (appelée «KEEP TIME»), qui définit les rôles et les responsabilités des différents acteurs au regard des activités à mener à bien avant la date de mise en service opérationnel, et qui peut être intégré dans la documentation relative à la méthode d évaluation de la sécurité appliquée à EUROCONTROL. Plus de cinquante recommandations pratiques sont formulées sur ce qu il convient de faire et de ne pas faire pour que la transition se déroule dans de bonnes conditions de sécurité et dans les délais fixés. 1. INTRODUCTION 1.1. OBJECTIFS La plupart des activités d'assurance sécurité sont ciblées sur les phases de conception et d'exploitation du cycle de vie d'un système. Toutefois, il existe une phase, généralement appelée phase de transition, au cours de laquelle sont effectués tous les travaux préparatoires à la mise en service opérationnel du système (dans d autres secteurs, comme celui de l énergie nucléaire, cette phase est également appelée «commissioning» ou essais de mise en service ). Une analyse de la méthode d évaluation de la sécurité appliquée à EUROCONTROL (SAM) a révélé que les techniques d assurance sécurité peuvent également être appliquées pendant cette phase lorsqu il s agit du développement de nouveaux systèmes de gestion du trafic aérien (ATM). À titre d exemple, un certain nombre de nouveaux centres de contrôle de la circulation aérienne ont été conçus et mis en service alors que plusieurs États oeuvrent encore à l acquisition de nouveaux centres ou planifient une mise à niveau des centres existants au moyen, par exemple, de nouvelles interfaces homme-machine (HMI). Or, il s écoule généralement plusieurs années entre la fin des phases de conception et d évaluation de la sécurité de la conception et le début de la mise en service opérationnel. Pendant cette période, il peut arriver que la conception du système et ses modalités de fonctionnement fassent l objet de modifications mineures, mais néanmoins importantes. De plus, l expérience acquise dans le fonctionnement du système permet de tirer des enseignements en matière de sécurité, qu il s agisse de nouveaux sujets de préoccupation ou de modalités de fonctionnement plus sûres. Projet D20/2005 Rapport CEE n