Topic: Six Aspects of Testing Complex Embedded Systems

Message

   Depending on the type of the granule, and according to common usage in the industry (we will discuss this usage later in this section), it is common to consider testing from six different aspects to evaluate whether the application's behavior, performance, and robustness match expected criteria. These aspects are:

   1. Software unit testing
   2. Software integration testing
   3. Software validation testing
   4. System unit testing
   5. System integration testing
   6. System validation testing

What follows is a discussion of each of these six aspects in relation to testing complex embedded systems.

1. Software Unit Testing

The GuT is either an isolated C function or a C++ class. Depending on the purpose of the GuT, the test case consists of either:

    * Data-intensive testing: applying a large range of data variation for function parameter values, or
    * Scenario-based testing: exercising different C++ method invocation sequences to perform all possible use cases as found in the requirements.

Points of Observation are returned value parameters, object property assessments, and source code coverage. White-box testing is used for testing units, meaning that the tester must be familiar with the content of the GuT. Unit testing is thus the responsibility of the developer.

Since it is not easy to track down trivial errors in a complex embedded system, every effort should be made to locate and remove them at the unit-test level.

2. Software Integration Testing

The GuT is now a set of functions or a cluster of classes. The essence of integration testing is the validation of the interface. The same type of Points of Control applies as for unit testing (data-intensive main function call or method-invocation sequences), while Points of Observation focus on interactions between lower-level granules using information flow diagrams.

As soon as the GuT starts to be meaningful, that is when an end-to-end test scenario can be applied to the GuT. First, performance tests can be run that should provide a good indication about the validity of the architecture. As for functional testing, the earlier the better. Each forthcoming step will then include performance testing.White-box testing is also the method used during that step. Software integration testing is the responsibility of the developer.

3. Software Validation Testing

The GuT is all the user code inside a component. This includes all the code developed by the team, but only the code itself. This aspect of testing can be considered one of the activities that occur toward the end of each software integration or build cycle. Partial use-case instances -- also called partial scenarios -- begin to drive the test implementation. The test implementation is less aware of and influenced by the implementation details of the GuT. Points of Observation include resource usage evaluation since the GuT is a significant part of the overall system. Again (and finally), we consider this step as white-box testing. Software validation testing is still the responsibility of the developer.

4. System Unit Testing

The GuT is now a full system component -- that is, the user code as tested during software validation testing plus all RTOS- and platform-related pieces that allow the code to run at minimum on a simulated target: tasking mechanisms, communications, interrupts, and so on. The Point of Control protocol is no longer a call to a function or a method invocation, but rather a message sent/received using the RTOS message queues, for example.

The similarities in their message-passing paradigms implies that the distinction between test drivers and stubs can, from the perspective of the GuT, be considered irrelevant at this stage. We will call them Virtual Testers because each one can replace and act as another system component vis-?is the GuT. "Simulator" or "tester" are synonyms for "virtual tester." Virtual tester technology should be versatile enough to adapt to a large number of RTOS and networking protocols. From now on, test scripts usually: bring the GuT into the desired initial state; then generate ordered sequences of samples of messages; and validate messages received by comparing (1) message content against expected messages and (2) date of reception against timing constraints. The test script is distributed and deployed over the various virtual testers. System resources are monitored to assess the system's ability to sustain embedded system execution. For this aspect, grey-box testing is the preferred testing method. In most cases, only a knowledge of the interface (the API) to the GuT is required to implement and execute appropriate tests. Depending on the organization, system unit testing is either the responsibility of the developer or of a dedicated system integration team.

5. System Integration Testing

The GuT starts from a set of components within a single node and eventually encompasses all system nodes up to a set of distributed nodes. The PCOs are a mix of RTOS- and network-related communication protocols, such as RTOS events and network messages. In addition to a component, a Virtual Tester can also play the role of a node. As for software integration, the focus is on validating the various interfaces. Grey-box testing is the preferred testing method. System integration testing is typically the responsibility of the system integration team.

6. System Validation Testing

The GuT is now a complete implementation subsystem or the complete embedded system. The objectives of this final aspect are several:

    * Meet external-actor functional requirements. Note that an external-actor might either be a device in a telecom network (say if our embedded system is an Internet Router), or a person (if the system is a consumer device), or both (an Internet Router that can be administered by an end user).
    * Perform final non-functional testing such as load and robustness testing. Virtual testers can be duplicated to simulate load, and be programmed to generate failures in the system.
    * Ensure interoperability with other connected equipment. Check conformance to applicable interconnection standards.

Going into details for these objectives is not in the scope of this article. Black-box testing is the preferred method: The tester typically concentrates on both frequently used and potentially risky or dangerous use-case instances.

Author	Message
Ankita_buie Newbie Joined: 03May2007 Online Status: Offline Posts: 1	Topic: Six Aspects of Testing Complex Embedded Systems Posted: 03May2007 at 11:09pm
	Depending on the type of the granule, and according to common usage in the industry (we will discuss this usage later in this section), it is common to consider testing from six different aspects to evaluate whether the application's behavior, performance, and robustness match expected criteria. These aspects are: 1. Software unit testing 2. Software integration testing 3. Software validation testing 4. System unit testing 5. System integration testing 6. System validation testing What follows is a discussion of each of these six aspects in relation to testing complex embedded systems. 1. Software Unit Testing The GuT is either an isolated C function or a C++ class. Depending on the purpose of the GuT, the test case consists of either: * Data-intensive testing: applying a large range of data variation for function parameter values, or * Scenario-based testing: exercising different C++ method invocation sequences to perform all possible use cases as found in the requirements. Points of Observation are returned value parameters, object property assessments, and source code coverage. White-box testing is used for testing units, meaning that the tester must be familiar with the content of the GuT. Unit testing is thus the responsibility of the developer. Since it is not easy to track down trivial errors in a complex embedded system, every effort should be made to locate and remove them at the unit-test level. 2. Software Integration Testing The GuT is now a set of functions or a cluster of classes. The essence of integration testing is the validation of the interface. The same type of Points of Control applies as for unit testing (data-intensive main function call or method-invocation sequences), while Points of Observation focus on interactions between lower-level granules using information flow diagrams. As soon as the GuT starts to be meaningful, that is when an end-to-end test scenario can be applied to the GuT. First, performance tests can be run that should provide a good indication about the validity of the architecture. As for functional testing, the earlier the better. Each forthcoming step will then include performance testing.White-box testing is also the method used during that step. Software integration testing is the responsibility of the developer. 3. Software Validation Testing The GuT is all the user code inside a component. This includes all the code developed by the team, but only the code itself. This aspect of testing can be considered one of the activities that occur toward the end of each software integration or build cycle. Partial use-case instances -- also called partial scenarios -- begin to drive the test implementation. The test implementation is less aware of and influenced by the implementation details of the GuT. Points of Observation include resource usage evaluation since the GuT is a significant part of the overall system. Again (and finally), we consider this step as white-box testing. Software validation testing is still the responsibility of the developer. 4. System Unit Testing The GuT is now a full system component -- that is, the user code as tested during software validation testing plus all RTOS- and platform-related pieces that allow the code to run at minimum on a simulated target: tasking mechanisms, communications, interrupts, and so on. The Point of Control protocol is no longer a call to a function or a method invocation, but rather a message sent/received using the RTOS message queues, for example. The similarities in their message-passing paradigms implies that the distinction between test drivers and stubs can, from the perspective of the GuT, be considered irrelevant at this stage. We will call them Virtual Testers because each one can replace and act as another system component vis-?is the GuT. "Simulator" or "tester" are synonyms for "virtual tester." Virtual tester technology should be versatile enough to adapt to a large number of RTOS and networking protocols. From now on, test scripts usually: bring the GuT into the desired initial state; then generate ordered sequences of samples of messages; and validate messages received by comparing (1) message content against expected messages and (2) date of reception against timing constraints. The test script is distributed and deployed over the various virtual testers. System resources are monitored to assess the system's ability to sustain embedded system execution. For this aspect, grey-box testing is the preferred testing method. In most cases, only a knowledge of the interface (the API) to the GuT is required to implement and execute appropriate tests. Depending on the organization, system unit testing is either the responsibility of the developer or of a dedicated system integration team. 5. System Integration Testing The GuT starts from a set of components within a single node and eventually encompasses all system nodes up to a set of distributed nodes. The PCOs are a mix of RTOS- and network-related communication protocols, such as RTOS events and network messages. In addition to a component, a Virtual Tester can also play the role of a node. As for software integration, the focus is on validating the various interfaces. Grey-box testing is the preferred testing method. System integration testing is typically the responsibility of the system integration team. 6. System Validation Testing The GuT is now a complete implementation subsystem or the complete embedded system. The objectives of this final aspect are several: * Meet external-actor functional requirements. Note that an external-actor might either be a device in a telecom network (say if our embedded system is an Internet Router), or a person (if the system is a consumer device), or both (an Internet Router that can be administered by an end user). * Perform final non-functional testing such as load and robustness testing. Virtual testers can be duplicated to simulate load, and be programmed to generate failures in the system. * Ensure interoperability with other connected equipment. Check conformance to applicable interconnection standards. Going into details for these objectives is not in the scope of this article. Black-box testing is the preferred method: The tester typically concentrates on both frequently used and potentially risky or dangerous use-case instances. Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged

bothasaichander Newbie Joined: 01Jun2008 Location: India Online Status: Offline Posts: 1	Posted: 01Jun2008 at 2:12am
	hi, this is sai from hyderabad.. am in sech of job on embedded systems can anyone help in getting a job...
	Thanks & Regards saichander [email protected] (+91-9290804806)
	IP Logged

puranamravinder Newbie Joined: 26Jun2008 Online Status: Offline Posts: 8	Posted: 27Jun2008 at 2:26am
	You can look for: 1. "keona software" - Near to Fortune Katriya Hotel, somajiguda, Hyderabad. 2. "eXpertonet Systems", Bangalore 3. CDAC 4. "Emertxe Information Technologies (P) Ltd", Bangalore And Many others..... Check which is the best one for you ..... you need to work hard for some days to get in to Embedded Systems side.... That is not so easy... All the best, Ravi
	http://earninglinkspuranam.blogspot.com
	IP Logged


One Stop Testing Forum : Types Of Software Testing @ OneStopTesting : Mobile, PDAs & Embedded Systems Testing @ OneStopTesting