The Price of Safety

 

A recent paper by Chao Chen, Genserik Reniers, Nima Khakzad, and Ming Yang discusses safety economics.  Safety economics is concerned with the costs of safety measures, and an important objective is to minimize the sum of two costs:  (1) the expected cost of the harms due to accidents in the future and (2) the current and future cost of safety measures.  Safety economics is a tool for making "decisions that are as good as possible (or 'optimal')" in order to both optimize the use of resources and maximize safety.  The paper discusses the importance of cost modeling, which includes direct costs, indirect costs, and "non-economic costs" that need to be monetized.  The value of statistical life and willingness to pay are mentioned in this context.

A common approach in safety economics is risk-based safety optimization, which is a type of risk management process.  This includes hazard identification, risk analysis, risk evaluation (i.e., is the risk acceptable), and risk mitigation.  The last step is accomplished by safety cost optimization, which evaluates the costs of the different safety strategies and selects the one with minimal cost.

The paper also discusses the minimal total safety cost approach (which considers both the safety strategy cost and the potential accident cost), cost-benefit analysis, cost-effectiveness analysis, multi-objective optimization, and game theory approaches.

To me the variety of approaches suggests that one must first engage in metareasoning to decide which decision-making process should be used.  Moreover, all of the approaches require human input in the form of setting thresholds (for risk acceptance criteria or cost-effectiveness ratios), weighing criteria, and making tradeoffs.  In practice, as with many decision models, a "decision calculus" (Little, 1970) may emerge in which the decision-maker asks the analyst to "find the solution," but these two people iterate as the decision-maker asks "what if?" in response to the results that the analyst generates.

Finally, the paper's focus on minimizing costs suggests that safety economics is based on substantive rationality, in which a decision-maker should choose the optimal alternative (Stirling, 2003).  Because bounded rationality better describes human decision-making, approaches that focus on finding satisfactory (not necessarily optimal) solutions may be more practical (Simon, 1981).

Cited sources:
Chen, Chao, Genserik Reniers, Nima Khakzad, and Ming Yang, "Operational safety economics: Foundations, current approaches and paths for future research," Safety Science, Volume 141, 2021.
Little, John D.C., “Models and managers: the concept of a decision calculus,” Management Science, Volume 16, Number 8, pages B-466-485, 1970.
Simon, Herbert A., The Sciences of the Artificial, second edition, The MIT Press, Cambridge, Massachusetts, 1981.
Stirling, Wynn C., Satisficing Games and Decision Making, Cambridge University Press, Cambridge, 2003.
 

Image source: https://www.gov.uk/government/news/venues-required-by-law-to-record-contact-details

Too Many Warnings?

At 5:35 P.M. on September 17, 2018, the University of Maryland Police Department (UMPD) sent a tornado warning by email and text message to university students, faculty, and staff: "A Tornado Warning has been issued for the UMD campus. The sirens will be activated. Seek shelter immediately, avoid windows."  This message did not mention that the warning was issued by Accuweather.  The National Weather Service (NWS) did not issue a tornado warning.  At 6:04 P.M., the UMPD sent the following message: "The Tornado Warning that was issued by AccuWeather is now cancelled as of 6:00 PM."

Part of Accuweather's business is generating severe weather alerts for its clients. Its website touts the benefits of Accuweather's warnings and its "null tornado notifications" that a NWS tornado warning will not affect a client's facility.  The website discusses the benefits of knowing that a tornado warning is not a direct threat, so a shutdown is not necessary, and claims that Accuweather's false alarm rate is lower than the NWS false alarm rate (10% to 80%).

When one organization issues a warning while the other does not, the inconsistency could decrease trust in warnings, increasing the likelihood of ignoring warnings.
An article in The Washington Post about the false alarm (Angela Fritz and Sarah Larimer, "Red flags at U-Md. over false alarm for tornado," September 19, 2018) quoted Gary Szatkowki, a former NWS meteorologist, who asked "What do we want to do about a weather organization issuing a tornado warning when the Weather Service does not?"

Although the University of Maryland is a large institution (approximately 40,000 students), its cost of a false alarm is smaller than the same cost for a factory (where a shutdown directly impacts throughput and revenue).  Moreover, many students and others are outdoors, and the university has a strong desire to be viewed as a safe place, so the potential impact is large.

In that case, the university should be willing to tolerate more warnings.  Too many warnings?  No, in this case.

UAV Operations and Failures

Earlier this month, the FAA's Micro Unmanned Aircraft Systems Aviation Rulemaking Committee (ARC) issued its recommendations in a final report: http://www.faa.gov/uas/publications/media/Micro-UAS-ARC-FINAL-Report.pdf

The recommendations include classifying UAVs (drones) into four categories, based on the risk that they pose to people underneath them.  If the UAV fails, it will crash and could cause a serious injury. 

If the mass of the UAV is less than or equal to 250 grams, then it would be in Category 1, which would have no additional restrictions (beyond those already in place).

UAVs more likely to cause a serious injury would face more restrictions.  For instance, Category 2 UAVs "must maintain minimum set-off distances of 20 feet above people’s heads, or 10 feet laterally away from people, and may not operate so close to people as to create an undue hazard to those people."  Category 3 UAVs would not be allowed to fly over crowds or dense concentrations of people.  A Category 4 UAV, on the other hand, could do that if it complied with a documented, risk mitigation plan.

There are many interesting details about the ARC, its risk attitude, how the ARC developed its recommendations, and other factors.  In particular, the ARC did not consider the likelihood of a UAV failure or the likelihood that it would hit someone if it failed; it considered only the distribution of the consequence (the chance of a serious injury) if it hit someone:  "Specifically, the ARC recommends that a small UAS be permitted to conduct limited operations over people ... if that UAS presents a 30% or lower chance of causing [a serious] injury upon impact with a person."

Keeping a Pipeline Safe

The risk associated with the 62-year-old pipelines under the Straits of Mackinac in northern Michigan was the subject of an article by Steve Friess in The Washington Post on Sunday.

After a different Enbridge pipeline in Michigan failed in 2010 and released about 20,000 barrels of oil, the state appointed a task force to study the oil pipelines throughout Michigan, include those under the Straits of Mackinac.  The task force report made four recommendations about the Straits pipelines and nine others for the whole state.  The task force recommended that the Straits pipelines should not transport heavy crude oil.  Enbridge has stated that the pipelines carry only light crude oil and light synthetic crude and natural gas liquids, including propane.  See, for instance, its Operational Reliability Plan.  The Enbridge website has more information about the pipelines and their plans to keep it safe; see http://www.enbridgeus.com/Line-5.aspx

Everyone agrees that a failure of the Straits pipelines could cause severe environmental damage. 
Enbridge, of course, also has a financial risk; they would lose revenue if the pipeline fails and has to be shutdown.  An Enbridge spokesperson stated, “Every day we’re out repairing pipelines and shutting down due to release, we’re not moving product. It’s in our interest as a pipeline company to keep it in the pipe.”

The Michigan Petroleum Pipeline Task Force website also has some interesting documents about the pipeline construction, including the 1953 engineering analysis (http://michigan.gov/documents/deq/Appendix_A.2_493980_7.pdf), which describes the selection of the location, the construction of the pipeline, and the analysis of the stresses involved.  In general, it is a good example of risk assessment and mitigation.  It acknowledges both the environmental and financial risks.  The pipeline elsewhere in Michigan has only one pipe, but two pipelines were used at the Straits, "for purposes of extra flexibility, extra strength, and a greater factor of safety against possible damage," according to this report.  If there are two pipes, then a leak in one pipe should release less oil, and the other pipe can continue to operate, which minimizes the financial and operational disruptions.  The report mentions the hazard from a ship's anchor and describes why this is unlikely in general and how the pipeline design will reduce this risk.  It also mentions that "any possible contamination of the waters caused by oil spillage from the pipeline crossing is considered remote in comparison to the amount and possibility of spillage from oil tankers."

This last point remains extremely relevant: given that people in Michigan use oil from Canada, all of the transportation options have risks, which the task force report acknowledged. 
For example, trains transporting oil had accidents in Quebec and Virginia.

Is a Self-Driving Car Safe?

Matt McFarland's article about the safety of robots, drones, and self-driving cars highlights the need for testing of all types: software tests, hardware tests, simulations, and operations on test tracks and real roads.

As the article mentions, the big question: how can one know if the robotic system is safe?

Safety must be relative to an acceptable risk threshold, and setting that threshold will be an important conversation.  Testing will have to show that the likelihood of an accident is sufficiently low and that the damage, if an accident occurs, is acceptably low.

In their draft requirements, the California Department of Motor Vehicles suggested that a third-party testing organization should verify that the vehicle is safe.




Links:
https://www.washingtonpost.com/news/innovations/wp/2015/12/18/the-billion-dollar-robot-question-how-can-we-make-sure-theyre-safe/

http://www.dmv.ca.gov/portal/dmv/detail/pubs/newsrel/newsrel15/2015_63

Small pilots, big risks

On October 24, 2015, The Washington Post reported that testing of the F-35 Joint Strike Fighter has shown that the ejection seat system poses a risk of whiplash to pilots.  In particular, pilots who weigh less than 136 pounds face a "high" risk of danger.  "Mid-weight" pilots face a "serious" risk.
The Post also reported that the mass of the pilot's helmet increases the risk because it is too heavy.
Until the risk can be mitigated, the F-35 program has restricted "lighter-weight" pilots from flying the plane, which will be used by the the U.S. Air Force, Navy, and Marine Corps, the Royal Air Force, and other U.S. allies.  The 34th Fighter Squadron at Hill Air Force Base was the first operational Air Force unit to fly combat-coded F-35s.

Of course, in normal operation, a pilot does not eject; using the ejection seat is a contingency plan for worst-case scenarios. But the story highlights the need to consider the potential problems (risks) that can occur during a contingency plan.

It also shows different levels of risk acceptance: in 2011, the Operational Test and Evaluation office had "serious concerns" about conducting training flights with the ejection seat, but the F-35 program office accepted the risk and went ahead with training.

Mitigating the Risk of Equipment Maintenance

Earlier this month I led a course on engineering risk management for a group of engineers and managers at a manufacturing firm that does sheet metal work and makes a variety of air distribution systems and components.  They have numerous machines that use multiple sources of power, which makes equipment maintenance more challenging.  They use lockout and tagout (LOTO) procedures (https://www.osha.gov/SLTC/controlhazardousenergy/index.html) but were interested in a systematic procedure for managing the risk associated with equipment maintenance.  While covering the process of risk management, the associated activities, and the fundamentals of decision making, we discussed how they could apply these steps to make their equipment maintenance operations safer.  The discussion included the potential problems of their current lockout procedures.

The bottom line: establishing and documenting lockout and tagout (LOTO) procedures are useful steps, but they don't replace a systematic risk management process that assesses, analyzes, evaluates, mitigates, and monitors the risks of equipment maintenance.  Look for the potential problems, identify the root causes, put in place safeguards that prevent them, and have contingency plans that can react promptly to keep a problem from getting worse.

P.S. I would like to thank the IIE Training Center (http://www.iienet2.org/IIETrainingCenter/Default.aspx) for the opportunity to lead this course.  Please contact them if you're interested in a short course on engineering decision making and risk management.

Managing the Risk of Fireworks

The Fourth of July is a great opportunity to talk about risk management.  Setting off fireworks at home is a popular entertainment, but it is dangerous, as the press reminds us every year: http://www.washingtonpost.com/blogs/govbeat/wp/2015/07/03/here-are-photos-of-all-the-horrific-ways-fireworks-can-maim-or-kill-you/

After assessing the risk, how can one mitigate it?  Here are the basic approaches:  (1) avoiding the risk by abandoning the planned action or eliminating the root cause or the consequences, (2) reducing the likelihood of the root cause or decreasing its consequences by modifying the planned action or performing preventive measures, (3) transferring the risk to another organization, or (4) assuming (accepting) the risk without mitigating it.

How would these apply to fireworks at home?
1. Avoid the risk: don't do it.  Go to a fireworks show or watch one on TV or find something else to do.
2. Reduce the risk: stick to sparklers and party poppers and follow safety guidelines (like these from http://www.cpsc.gov/safety-education/safety-education-centers/fireworks/): keep fireworks away from brush and other substances that can burn, don't let children play with fireworks, keep a bucket of water handy to douse the fireworks or anything that catches fire.
3. Transfer the risk: hire a professional (or other trained expert) to do a fireworks show at your place, or let a neighbor run the show while you and your family watch from a safe distance.
4. Accept the risk: indulge in the tradition!

The relative desirability of these options depends upon how much you like fireworks and how much risk you're willing to accept.

Have a Happy Fourth of July!