Testing generates information that can be used to make a decision. Testing can occur at any stage in the development of a product or system; it can test a specific attribute or overall performance; it can test a component, a subsystem, a system, or a system-of-systems.
Planning a test requires making crucial decisions: Which item to test? Which test to perform? How many tests to perform?
The test plan determines the value of the information gathered and the time and cost of testing. In general the key tradeoff is that gathering more valuable information requires more time and cost.
My students and I have developed some techniques for making test planning decisions.
Which facility to use? In some cases, a system (such as a military vehicle) needs to be used in an operational environment, but there are no existing test facilities like that environment. Instead of building a new test facility, one could use a combination of existing facilities to replicate the new operational environment. We developed an optimization model to find the test plan that used the best combination of existing facilities examples. For military vehicle applications, it specified the time and number of miles that the test vehicles should run on each existing test facility. Reference: http://www.isr.umd.edu/~jwh2/papers/IEST.pdf
Which configuration to test? A system-of-systems (SoS) consists of relatively independent systems. For example, a missile defense systems has control stations, radar locations, and rocket launchers. If the reliability of the SoS is unknown because the reliability of the systems is unknown, then testing is needed, but testing a full-scale configuration is expensive. We developed a simulation technique to predict the results of tests with smaller configurations under different scenarios and estimate the expected error of the tests. With this information, the test planning decision-makers could evaluate the tradeoffs of cost and expected error and determine the best test configuration. Reference: http://www.isr.umd.edu/~jwh2/papers/Tamburello-Herrmann-JRR-2015.pdf
Which attribute to measure (test)? In a multiattribute decision making situation, measurements of the attributes are valuable for knowing which alternative is best. If these measurements have error and the total budget for measurements is limited, then it is crucial to measure the attributes in a way that provides the most valuable information and increases the likelihood of selecting the truly best alternative. We developed and tested rules for determining which attributes should be measured how many times and showed that better rules can significantly increase this likelihood. Reference: http://www.isr.umd.edu/~jwh2/papers/Leber-Herrmann-ISERC-2015.pdf
Which test to perform? Demonstrating the reliability of a complicated system (such as a liquid rocket engine) requires testing the system and its components and subsystems. These tests and the associated hardware are expensive and require time at scarce test facilities. We developed a multi-objective test plan optimization approach to determine the best test plan that meets the demonstrated reliability. Reference: http://www.isr.umd.edu/~jwh2/papers/Strunz-Herrmann-CEAS-Space-2011.pdf
Monday, November 30, 2015
Saturday, November 21, 2015
How to React to an Earthquake
One cannot prevent an earthquake, but one can protect people and property from its impact. Traditionally, this means designing and building structures that will survive the earthquake without damage (preventive actions). Now, because of earthquake warning systems, contingency planning is becoming feasible.
Earlier this month, The Washington Post's Eric Niiler wrote about technology that can tell that an earthquake is about to happen. The November 3 article ("Last-minute warnings may make quakes less destructive") states that a combination of ground sensors and satellite-based instruments can give a warning a minute or two before the earthquake occurs.
The technology works by detecting the P-waves that precede the earthquake's shock (the sound waves are an earthquake precursor). The ShakeAlert earthquake early warning system for the west coast of the United States was developed by the United States Geological Survey (USGS) and university partners. According to Niiler, Chile, Mexico, China, Taiwan, Turkey, and Israel are developing similar systems.
Although a minute is not enough time to evacuate large buildings or a city, it is enough to shut down (or put into a safe mode) critical systems such as trains, gas lines, elevators, tunnels, and bridges. Unfortunately, it is not enough time to send text messages to millions of people simultaneously, so earthquake alerts go only to emergency agencies, utilities, and similar high-priority organizations.
For more about ShakeAlert, see the fact sheet at http://pubs.usgs.gov/fs/2014/3083/pdf/fs2014-3083.pdf
Tuesday, November 3, 2015
Small pilots, big risks
On October 24, 2015, The Washington Post reported that testing of the F-35 Joint Strike Fighter has shown that the ejection seat system poses a risk of whiplash to pilots. In particular, pilots who weigh less than 136 pounds face a "high" risk of danger. "Mid-weight" pilots face a "serious" risk.
The Post also reported that the mass of the pilot's helmet increases the risk because it is too heavy.
Until the risk can be mitigated, the F-35 program has restricted "lighter-weight" pilots from flying the plane, which will be used by the the U.S. Air Force, Navy, and Marine Corps, the Royal Air Force, and other U.S. allies. The 34th Fighter Squadron at Hill Air Force Base was the first operational Air Force unit to fly combat-coded F-35s.
Of course, in normal operation, a pilot does not eject; using the ejection seat is a contingency plan for worst-case scenarios. But the story highlights the need to consider the potential problems (risks) that can occur during a contingency plan.
It also shows different levels of risk acceptance: in 2011, the Operational Test and Evaluation office had "serious concerns" about conducting training flights with the ejection seat, but the F-35 program office accepted the risk and went ahead with training.
The Post also reported that the mass of the pilot's helmet increases the risk because it is too heavy.
Until the risk can be mitigated, the F-35 program has restricted "lighter-weight" pilots from flying the plane, which will be used by the the U.S. Air Force, Navy, and Marine Corps, the Royal Air Force, and other U.S. allies. The 34th Fighter Squadron at Hill Air Force Base was the first operational Air Force unit to fly combat-coded F-35s.
Of course, in normal operation, a pilot does not eject; using the ejection seat is a contingency plan for worst-case scenarios. But the story highlights the need to consider the potential problems (risks) that can occur during a contingency plan.
It also shows different levels of risk acceptance: in 2011, the Operational Test and Evaluation office had "serious concerns" about conducting training flights with the ejection seat, but the F-35 program office accepted the risk and went ahead with training.
Subscribe to:
Posts (Atom)