Dynamic detection of COTS component incompatibility

The development of COTS-based systems shifts the focus of testing and verification from single components to component integration. Independent teams and organizations develop COTS components without referring to specific systems or interaction patterns. Developing systems that reuse COTS components (even high-quality ones) therefore presents new compatibility problems. David Garlan, Robert Allen, and John Ockerbloom reported that in their experience, integrating four COTS components took 10 person-years (rather than the one planned person-year), mainly because of integration problems. According to Barry Boehm and Chris Abts, three of the four main problems with reusing COTS products are absence of control over their functionality, absence of control over their evolution, and lack of design for interoperability. Traditional integration testing techniques incrementally validate the integration of the modules that compose the system being tested. Test designers can derive test cases from both the source code and the specifications. Unfortunately, COTS components are often distributed without the source code and with incomplete specifications. Moreover, developers can use them in ways that the original design didn’t predict. So, testers must test them in the contexts in which they’re used. Our proposed technique, called behavior capture and test, detects COTS component incompatibilities by dynamically analyzing component behavior. BCT incrementally builds behavioral models of components and compares them with the behavior the components display when reused in new contexts. This lets us identify incompatibilities, unexpected interactions, untested behaviors, and dangerous side effects.