Holistic Safety Analysis for Urban Air Mobility

First published July 2020

This white paper was written in July 2020, before SAE published ARP4754B and ARP4761A. Its references to SC-VTOL and to the standards reflect that date.

Introduction

This paper proposes an approach that uses a holistic safety analysis to derive requirements for an Urban Air Mobility (UAM) aircraft. The approach uses existing and emerging safety analysis techniques, addresses the deficiencies of EASA’s Special Condition for VTOL aircraft (SC-VTOL), modifies probability budgets and Development Assurance Levels (DALs) to suit a UAM aircraft, and identifies compensating factors that ensure FAA and EASA safety assurance objectives are exceeded.

Background

Traditionally, novel technologies and applications of technologies have been accommodated through special conditions. The intent of requirements such as these can be summarized as achieving a societally acceptable level of safety assurance and risk. However, special conditions are often written independently of the safety analysis, which introduces inconsistencies.

It is a misconception that Part 23 Amendment 64 is a performance-based requirements framework. In reality, many of the performance-based requirements in Amendment 62 do not exist in Amendment 64. Instead, it is an abstraction of Amendment 62 that eliminates a requirements tier. It accommodates a wider variety of system architectures by being less prescriptive. However, it leaves a void, because systems engineering and design assurance objectives require traceability from safety assurance objectives, through requirements tiers, to the software and complex hardware.

The proposal is that a holistic safety analysis, similar to that advocated by Moak, L. et al. (2020), is used to derive requirements that bridge this gap and are consistent with safety assurance objectives.

The FAA and EASA have opposing safety assurance objectives for UAM aircraft. The FAA applies a safety continuum that links probability budgets and DALs to the number of passengers. EASA’s SC-VTOL associates probability budgets and DALs with Basic and Enhanced certification categories, linked to the intended type of operations.

SC-VTOL also imposes additional requirements:

• It redefines continued safe flight and landing as “continued controlled flight and landing at a vertiport, possibly using emergency procedures, without requiring exceptional piloting skill or strength.” This levies requirements that do not exist for Parts 23, 25, 27, and 29, and does not accommodate an emergency procedure to land as soon as possible or practicable away from a vertiport to prevent a catastrophic failure condition.
• It does not apply the safety continuum. Therefore, the probability budget and DAL objectives for a UAM aircraft are the same as for a Part 23 airworthiness level 4 aircraft or a Part 25 aircraft.
• Its extension of the single failure concept to “also include aircraft structures” ignores the factor of safety approach traditionally applied to aircraft structures.
• It increases safety assurance objectives “by one level compared to CS-23 due to a higher dependency on systems that are associated with distributed propulsion.” In reality, distributed propulsion is inherently safer than the propulsion systems of Part 27 and 29 aircraft, because of its fault tolerance and redundancy. UAM aircraft should not be penalized for having distributed propulsion.

Consequently, to apply a holistic safety analysis successfully, the safety assurance objectives of the FAA and EASA must be harmonized in a way that is not prohibitive or unreasonably burdensome.

Harmonized safety assurance objectives

The following table summarizes different approaches to assigning probability budgets and DALs, taken from ASTM F3061/F3061M-19a, ASTM F3230-17, and SC-VTOL. Its final row introduces new harmonized safety assurance objectives that support a holistic safety analysis. Each cell gives the Function Development Assurance Level (FDAL) and the probability budget per flight hour.

1. No consideration of the system architecture for a DAL reduction is acceptable.
2. Consideration of the system architecture for a DAL reduction is acceptable.
3. FDAL for the primary system / FDAL for the secondary system.

Part 23 is the starting point for the following reasons:

• Parts 25, 27, and 29 have not been abstracted to accommodate a wider variety of system architectures, unlike Part 23.
• Parts 25 and 29 are in a different risk category to Parts 23 and 27.
• The Part 23 and Part 27 safety continuums serve the same purpose and are similar enough to justify focusing on Part 23 rather than Part 27 (see PS-ASW-27-15).

DAL assignment to members of a functional failure set

ARP4754A prescribes how FDALs are assigned to the members of a functional failure set. For a catastrophic failure condition, a functional failure set with a single member requires FDAL A. With multiple members, either one member is FDAL A, or two members are FDAL B, and any other contributing members are no lower than FDAL C. The same pattern applies one level down for hazardous failure conditions (FDAL B, or two members at FDAL C), major failure conditions (FDAL C, or two members at FDAL D), and minor failure conditions (FDAL D).

The approaches in the table above either do or do not accept architectural mitigation for a DAL reduction. The harmonized safety assurance objectives accept it.

Compensating factors

The harmonized safety assurance objectives are not a compromise between FAA and EASA requirements. Instead, they enable a more succinct safety analysis without undermining the safety assurance objectives of SC-VTOL. This is achieved through compensating factors.

Compensating for a reduced probability budget

The order of magnitude reduction in the probability budget for catastrophic failure conditions is compensated for by imposing conservative Failure Modes and Effects Analysis (FMEA) and Failure Modes and Effects Summary (FMES) assumptions:

• Functional, rather than piece-part, FMEA and FMES.
• Using the full failure rate for components and collections of components, instead of failure mode distributions.

The same order of magnitude reduction can be applied to hazardous and major failure conditions. However, it should be reserved for failure conditions that satisfy the single failure concept.

Compensating for a reduced Development Assurance Level

Requiring DAL B for system architectures with elements that could contribute to a catastrophic failure condition enables the use of safety-enhancing Commercial Off-The-Shelf (COTS) equipment. COTS equipment can generally satisfy the objectives of DAL D, but not DAL A, B, or C. DAL B can be achieved with a system architecture that has a primary member at DAL B and a secondary member at DAL D.

Examples of safety-enhancing COTS equipment include:

• Detect and Avoid (DAA) systems that incorporate neural networks.
• Global Navigation Satellite System (GNSS) equipment.
• Inertial Measurement Unit (IMU) equipment.

Human factors

Historically, safety analyses have taken credit for flight crew actions documented in the aircraft flight manual, and for maintenance actions documented in the maintenance manual, that can reasonably be expected to be performed correctly. However, around 80% of errors can be attributed to human error. Therefore, in line with the recommendations of Moak, L. et al. (2020), the safety analysis must systematically consider human error, by evaluating human performance in response to nominal and off-nominal conditions and tasks. Model-based systems and safety techniques and control theory must also be applied to derive control actions and unsafe control actions. The recommended model-based approach is Systems Theoretic Process Analysis (STPA).

Continued airworthiness

Historically, data of sufficient fidelity has not been available to track failures and perform root cause analysis. A mechanism to track this data must be imposed, and the safety analysis must be revisited as failure rate assumptions change. Maintenance intervals must be updated accordingly. The transition from an open-loop to a closed-loop approach is also expected to enable condition-based maintenance that minimizes unscheduled maintenance.

Endurance testing

Critical parts, whose failure is expected to result in a hazardous or catastrophic failure condition, are expected to undergo endurance testing. Endurance testing approaches have improved dramatically in recent years. Critical parts require an approach that exploits the relationship between Highly Accelerated Life Testing (HALT), Highly Accelerated Stress Screening (HASS), endurance testing, and failure rate.

Factors requiring clarification

Aircraft and system safety assessments

ARP4761A introduces the concepts of an Aircraft Functional Hazard Analysis (AFHA), a Preliminary Aircraft Safety Assessment (PASA), and an Aircraft Safety Assessment (ASA) to analyze integrated systems. The rationale is that aircraft are becoming highly integrated, with fewer federated systems.

Assuming a full transition from federated to integrated systems, the two tiers should be replaced by a single tier of Functional Hazard Analysis (FHA), Preliminary System Safety Assessment (PSSA), and System Safety Assessment (SSA). These should be linked to each other, and to a single tier of FMEA, FMES, and reliability prediction. The links and this flatter structure enable a more iterative safety analysis than traditional approaches such as the V-model. The analysis should be repeated regularly throughout the design and development of the UAM aircraft. For more on the two versions of the standard, see our comparison of ARP4761 and ARP4761A.

Continued safe flight and landing

The redefinition of a catastrophic failure condition from “continued safe flight and landing” to “continued safe flight and landing at a vertiport, possibly using emergency procedures, without requiring exceptional piloting” (SC-VTOL) should not happen. It would have a profound effect on the safety analysis, and decrease rather than increase safety:

• No safety analysis credit for developing systems that enable an emergency landing as soon as possible or practicable away from a vertiport.
• Prohibiting a UAM aircraft in a degraded state from landing as soon as possible or practicable exposes passengers and third parties to increased risk.
• The direct relationship between the safety analysis and societally acceptable safety assurance and risk would be undermined.
• Catastrophic failure conditions would be assessed against the aircraft’s ability to land at a vertiport, rather than against the risk to passengers and third parties.
• It would introduce unprecedented regulatory requirements for UAM aircraft that do not exist for Part 23, 25, 27, and 29 aircraft flying over congested areas.

Multiple instances of a COTS device

Through SWCEH-001, EASA imposes architectural mitigation wherever multiple instances of a COTS device with a common failure mode could cause a catastrophic failure condition. The FAA does not explicitly impose this requirement. However, obtaining the information needed to satisfy design assurance objectives can be difficult for a COTS device.

This is changing, with increasing automation in other industries and wider use of standards such as ISO 26262 and IEC 61508, which have similarities to DO-178C and DO-254. Therefore, instead of requiring architectural mitigation, there should be an option to use the information generated by a manufacturer’s functional safety process for a COTS device. This information, a combination of quality assurance, configuration management, and other lifecycle data, should be used carefully, because it is generated out of context, with assumptions that may not apply. Also, ISO 26262 and IEC 61508 are goal based (not requiring that all goals are met), while DO-178C and DO-254 are objective based (requiring that all objectives are met). The goals that have been met should be examined against the DO-178C and DO-254 design assurance objectives.

Failure condition classification

The following table is used to classify failure conditions. To support a holistic safety analysis that considers everyone who interacts with the UAM aircraft, the effect on the maintenance crew is added.

Compared to passengers, third parties face a lower level of risk from an accident. Of all the fatal accidents involving Part 27 and 29 aircraft between 2010 and 2020, five were over congested areas, and two of these resulted in third-party fatalities: the EC135 T2+ accident on 29 November 2013, and the AW109 G-CRST accident on 16 January 2013. The effect on third parties is therefore implicitly accounted for, and is not added to the table.

SC-VTOL attempts to consider the effect on third parties by assigning safety assurance objectives regardless of the number of passengers. This is not appropriate, because:

• Part 27 and even Part 23 aircraft regularly operate in, out of, and over densely populated areas such as cities, without additional requirements.
• UAM aircraft will operate to and from cities. However, with ranges of around 100 km, most of the flight path is not expected to be over densely populated areas, and the flight path can be chosen to mitigate risk to third parties.
• There are more effective ways of mitigating risk to third parties near densely populated areas, such as automated takeoff and landing aids.

When assigning the effect on the flight crew, the impact of automation, including the effect of a loss of automation and vigilance decrement, must be considered. Vigilance decrement is the “deterioration in the ability to remain vigilant for critical signals with time, as indicated by a decline in the rate of the correct detection of signals” (Parasuraman, 1997, and Mackworth, 1948). The flight crew could be a pilot or a remote pilot, depending on the UAM aircraft. If there is a many-to-many relationship between UAM aircraft and a team of pilots, the most severe effect on any individual pilot must be considered.

If automation can mitigate the effect of a degraded state on the flight crew, aircraft, and occupants, the effect on safety margins may be what separates minor, major, and hazardous failure conditions: whether a failure condition results in a slight, significant, or large reduction in safety margins. Since probability is a function of exposure time and failure rate, a UAM aircraft that reduces its exposure time to a second catastrophic failure by landing as soon as possible or practicable after a first failure can make a case for the first failure condition being minor, rather than major or hazardous:

Probability of a second catastrophic failure condition after the first	Classification of the first failure condition
Less than 1E-5	Minor
1E-5 to less than 1E-3	Major
1E-3 to less than 1E-1	Hazardous

To calculate the probability of the second failure condition after the first has occurred, each cut set contributing to the first failure condition is set, in turn, to a probability of 1. The highest resulting probability sets the classification of the first failure condition.

Further considerations

Aircraft parachute system

An aircraft parachute can mitigate certain UAM aircraft failure conditions. The Cirrus SR20 and SR22 were certified with an aircraft parachute system.

The disadvantages of an aircraft parachute system for a UAM aircraft are:

• Additional mass.
• A minimum deployment altitude of around 500 ft, so it does not mitigate a loss of vertical thrust during the critical takeoff and landing phases.
• The pyrotechnics can fail latently and require scheduled maintenance.

The advantages are:

• If the failure is detected, it could mitigate a loss of vertical thrust during cruise.
• It can be operated with no electrical power.

Dispatch relief

The dispatchability of a UAM aircraft warrants consideration, because takeoff and landing sites are likely to be austere, with few replacement parts. Additional fault tolerance and redundancy can allow the aircraft to be dispatched with a failure. However, it cannot be dispatched if it cannot satisfy the single failure concept. To determine an acceptable dispatch relief interval, every latent failure is given an exposure time equal to the dispatch relief time in the Master Minimum Equipment List (MMEL), and the failure is set to a probability of 1. Under these conditions, it must still be possible to satisfy the probability budgets.

Neural networks

There has been much deliberation about how to certify safety-enhancing neural networks. The biggest challenge is the structural coverage and low-level requirements objectives of DO-178C, which apply to DAL A, B, and C. However, there are no structural coverage or low-level requirements objectives for DAL D. Also, following the functional failure set rules above, a primary DAL B member and a secondary DAL D member, or two independent DAL D members, can satisfy FDAL C objectives. Therefore, with architectural mitigation, an approach to certifying safety-enhancing neural networks already exists.

Our AI variant applies the same principle of architectural mitigation: its edge inference board is developed to DAL C and paired with a DAL A safety-critical board, supporting DAL A at the equipment level.

Operating and environmental limitations

The safety analysis is valid as long as the UAM aircraft remains within its operating and environmental limits. Most regulatory requirements, including SC-VTOL, exist to ensure these limitations are set for a particular aircraft type. They should be developed together with the safety analysis.

The holistic safety analysis framework

Compared to ARP4761 and ARP4761A, the holistic safety analysis framework:

• Adds STPA to address human factors, identify missing failure conditions and hardware and software requirements, and validate those that already exist.
• Flattens the structure of the safety analysis.
• Shows the relationship between reliability prediction, endurance testing, Single Event Effects (SEE) analysis, and the other safety analysis activities.

STPA is chosen because of its relative maturity and regulatory acceptance compared to other model-based systems and safety techniques.

Flattening the structure to one tier, instead of two for ARP4761 and three for ARP4761A, streamlines the safety analysis, allows more frequent iterations, and can shorten the design and development lifecycle. This is possible because of the relative complexity of a UAM aircraft compared to a traditional Part 25, or even Part 23, aircraft. It is also necessary because of the level of systems integration, which means that, more than ever, the effect of a failure is expected to appear at the aircraft level rather than the system level. A flatter structure also makes the safety analysis proactive, informing the hardware and software design processes rather than reacting to them.

SEE analysis has in recent years been applied to Part 25 aircraft flying at high altitudes and high latitudes. Atmospheric particles, mainly neutrons and protons, have become more of a risk as device feature sizes have shrunk and the number of devices per aircraft has grown. Devices that use technology immune to atmospheric particles are preferred. Per SAE AIR6219, an aircraft at around 40,000 ft, such as a Part 25 aircraft, is exposed to ten times the relative atmospheric particle flux of an aircraft at around 10,000 ft, such as a UAM aircraft. However, the following factors increase a UAM aircraft’s risk from atmospheric particles:

• The number of devices is expected to be high compared to a Part 25 aircraft.
• It may not be possible to select technology that is immune to atmospheric particles.
• Mitigations such as derating power electronics may not be possible.

Definitions

• Single failure concept: the objective of this design concept is to permit the airplane to continue safe flight and landing after any single failure. Protection from multiple malfunctions or failures should be provided when the first malfunction or failure would not be detected during normal operations of the airplane, which includes preflight checks, or if the first malfunction or failure would inevitably cause other malfunctions or failures (AC 23.1309).
• Land as soon as possible: a landing, as a result of a hazardous failure condition, at a site that exposes persons or property to the least possible level of risk.
• Land as soon as practicable: a landing, as a result of a major or minor failure condition, at the nearest vertiport, or at a site known to be clear of persons or property. At the nearest vertiport, the UAM aircraft is expected to be prioritized.
• Continued safe flight and landing: the airplane is capable of continued controlled flight and landing, possibly using emergency procedures, without requiring exceptional pilot skill or strength. Upon landing, some airplane damage may occur as a result of a failure condition (AC 23.1309).

Applicable industry standards

• SAE ARP4754A, “Guidelines for Development of Civil Aircraft and Systems”, recognized by AC 20-174 as an acceptable means of compliance.
• SAE ARP4761, “Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment”.
• ASTM F3061-19a, “Standard Specification for Systems and Equipment in Small Aircraft”.
• ASTM F3230-17, “Standard Practices for Safety Assessment for Systems and Equipment in Small Aircraft”.
• SAE AIR6913, “Using STPA During Development and Safety Assessment of Civil Aircraft”.
• ASTM WK60748, “New Guide for Application of Systems-Theoretic Process Analysis to Aircraft”.
• SAE AIR6219, “Incorporation of Atmospheric Neutron Single Event Effects Analysis into a Safety Assessment”.
• SAE ARP5150, “Safety Assessment of Transport Airplanes in Commercial Service”.
• SAE ARP5151, “Safety Assessment of General Aviation Airplanes and Rotorcraft in Commercial Service”.
• ASTM F3254, “Aircraft Interaction of Systems and Structures”.
• RTCA DO-178C, “Software Considerations in Airborne Systems and Equipment Certification”.
• RTCA DO-330, “Software Tool Qualification Considerations”.
• RTCA DO-331, “Model-Based Development and Verification Supplement to DO-178C and DO-278A”.
• RTCA DO-332, “Object-Oriented Technology and Related Techniques Supplement to DO-178C and DO-278A”.
• RTCA DO-333, “Formal Methods Supplement to DO-178C and DO-278A”.
• RTCA DO-254, “Design Assurance Guidance for Airborne Electronic Hardware”.
• RTCA DO-160G, “Environmental Conditions and Test Procedures for Airborne Equipment”.

References

• AC 23.1309-1E, 2011. “System Safety Analysis and Assessment for Part 23 Airplanes.”
• Mackworth, N. H., 1948. “The breakdown of vigilance during prolonged visual search.” Quarterly Journal of Experimental Psychology.
• Moak, L. et al., 2020. “Official Report of the Special Committee to Review the Federal Aviation Administration’s Aircraft Certification Process.”
• Parasuraman, R., 1997. “Humans and Automation: Use, Misuse, Disuse, Abuse.” Human Factors and Ergonomics Society.
• PS-ASW-27-15, 2017. “Safety Continuum for Part 27 Normal Category Rotorcraft Systems and Equipment.”
• RIAC, 2015. “RIAC’s Reliability Prediction Methodology.” Reliability Information Analysis Center.
• SC-VTOL, 2019. “Special Condition for Small-Category VTOL Aircraft.”
• SWCEH-001 Issue 1, Revision 2, 2018. “Development Assurance of Airborne Electronic Hardware.”