Testing generates information that can be used to make a decision. Testing can occur at any stage in the development of a product or system; it can test a specific attribute or overall performance; it can test a component, a subsystem, a system, or a system-of-systems.
Planning a test requires making crucial decisions: Which item to test? Which test to perform? How many tests to perform?
The test plan determines the value of the information gathered and the time and cost of testing. In general the key tradeoff is that gathering more valuable information requires more time and cost.
My students and I have developed some techniques for making test planning decisions.
Which facility to use? In some cases, a system (such as a military vehicle) needs to be used in an operational environment, but there are no existing test facilities like that environment. Instead of building a new test facility, one could use a combination of existing facilities to replicate the new operational environment. We developed an optimization model to find the test plan that used the best combination of existing facilities examples. For military vehicle applications, it specified the time and number of miles that the test vehicles should run on each existing test facility. Reference: http://www.isr.umd.edu/~jwh2/papers/IEST.pdf
Which configuration to test? A system-of-systems (SoS) consists of relatively independent systems. For example, a missile defense systems has control stations, radar locations, and rocket launchers. If the reliability of the SoS is unknown because the reliability of the systems is unknown, then testing is needed, but testing a full-scale configuration is expensive. We developed a simulation technique to predict the results of tests with smaller configurations under different scenarios and estimate the expected error of the tests. With this information, the test planning decision-makers could evaluate the tradeoffs of cost and expected error and determine the best test configuration. Reference: http://www.isr.umd.edu/~jwh2/papers/Tamburello-Herrmann-JRR-2015.pdf
Which attribute to measure (test)? In a multiattribute decision making situation, measurements of the attributes are valuable for knowing which alternative is best. If these measurements have error and the total budget for measurements is limited, then it is crucial to measure the attributes in a way that provides the most valuable information and increases the likelihood of selecting the truly best alternative. We developed and tested rules for determining which attributes should be measured how many times and showed that better rules can significantly increase this likelihood. Reference: http://www.isr.umd.edu/~jwh2/papers/Leber-Herrmann-ISERC-2015.pdf
Which test to perform? Demonstrating the reliability of a complicated system (such as a liquid rocket engine) requires testing the system and its components and subsystems. These tests and the associated hardware are expensive and require time at scarce test facilities. We developed a multi-objective test plan optimization approach to determine the best test plan that meets the demonstrated reliability. Reference: http://www.isr.umd.edu/~jwh2/papers/Strunz-Herrmann-CEAS-Space-2011.pdf
No comments:
Post a Comment