This is the next article from my series Advanced Software Testing Techniques where I talk about various software testing techniques that I learned from school and my previous projects. In this article, I will present an interesting way of visualising software requirements by transforming our application into a directed graph.

🔍 Overview

Structural testing is another important part of the software development process, as it helps to ensure the quality and reliability of a software system. This type of testing focuses on the internal structure of the software and verifies that it is designed and built according to industry standards and best practices.

By evaluating the various components of the system and the relationships between them, structural testing can detect potential weaknesses and defects that could cause problems down the line. This article will provide an overview of structural testing, including its benefits, techniques, and challenges.

🔑 Key Features

• The test data is generated based on the implementation (program), without taking into account the specification(s) of the program;

• To use structural testing methods, the program can be represented as a directed graph;

• The test data is chosen to go through all the elements (instruction, branch, or path) of the graph at least once. Depending on the type of elements chosen, different measures of graph coverage will be defined: Instruction level coverage, branch level coverage or path level coverage;

📍 Converting an application into an oriented graph

• For a sequence of instructions, a node is inserted:

  • if c then s1 else s2:

    

  • while c do s:

    

  • repeat s until c:

    

💻 Example

We will use the same example of the application from the previous Functional Testing article.

import java.util.Scanner;

public class CharacterSearch {

    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);
        String inputString;
        char searchChar;
        String searchAgain = "y";

        while (searchAgain.equals("y")) {
            // get the input string
            System.out.print("Enter a string of no more than 20 characters: ");
            inputString = scanner.nextLine();
            while(inputString.length()>20){
                System.out.println("String is too long, please enter a string of no more than 20 characters:");
                inputString = scanner.nextLine();
            }
            // get the character to search for
            System.out.print("Enter a character to search for: ");
            searchChar = scanner.nextLine().charAt(0);

            // search for the character in the string
            int charIndex = inputString.indexOf(searchChar);
            if (charIndex == -1) {
                System.out.println("The character '" + searchChar + "' was not found in the string '" + inputString + "'.");
            } else {
                System.out.println("The character '" + searchChar + "' was found at position " + (charIndex + 1) + " in the string '" + inputString + "'.");
            }

            // ask the user if they want to search for another character
            System.out.print("Search for another character? (y/n): ");
            searchAgain = scanner.nextLine();
        }
        System.out.println("Thank you for using the Character Search program!");
    }
}

The graph representation will be:

🌍 Coverages

In structural testing, coverage refers to the extent to which the source code is executed during testing. Various coverage criteria can be used to measure the degree of code coverage achieved during testing.

One approach to measuring coverage is to use a control flow graph, which is a graphical representation of the program's control flow. Based on the control flow graph, different coverage criteria can be defined, such as:

• Statement coverage: each instruction (node in the graph) is executed at least once;

• Branch coverage: each branch in the graph is executed at least once;

• Path coverage: each path through the graph is executed at least once;

📍 Statement coverage

Statement coverage is a metric used in structural testing that measures the percentage of individual statements in a program that have been executed during testing. In other words, it refers to the degree to which the code has been exercised by the test cases.

In the context of structural testing, achieving statement coverage means that every statement in the code has been executed at least once during testing. This is typically considered the minimum level of coverage that should be achieved through structural testing.

To achieve statement coverage, testers need to focus on those statements in the code that are controlled by conditions, which correspond to the branches in the control flow graph. By testing both the true and false branches of each condition, testers can ensure that all statements in the code have been executed at least once.

💻 Example

Let's say we have a Java program that calculates the average of two numbers. Here's the code:

public class AverageCalculator {
    public static double calculateAverage(double num1, double num2) {
        double sum = num1 + num2;
        double average = sum / 2;
        return average;
    }
}

To achieve statement coverage, we need to ensure that each statement in the code is executed at least once during testing. In this case, that means we need to test both the assignment statement for the sum variable and the assignment statement for the average variable.

Here's an example of a JUnit test case that achieves statement coverage:

import org.junit.Test;
import static org.junit.Assert.assertEquals;

public class AverageCalculatorTest {
    @Test
    public void testCalculateAverage() {
        double num1 = 10;
        double num2 = 20;
        double expectedAverage = 15;
        double actualAverage = AverageCalculator.calculateAverage(num1, num2);
        assertEquals(expectedAverage, actualAverage, 0.01);
    }
}

In this test case, we're passing in the values 10 and 20 for num1 and num2, respectively, and we're expecting the calculated average to be 15. This test case ensures that both the sum and average statements are executed, thus achieving statement coverage.

🔹 Statement coverage weaknesses

While statement coverage is a useful metric for measuring the thoroughness of testing in some cases, it does have certain weaknesses:

1. Limited effectiveness in detecting faults: Achieving 100% statement coverage does not necessarily mean that all potential faults in the code have been detected. It is possible to achieve high statement coverage while still leaving certain faults undetected. This is because statement coverage does not take into account the behaviour of the code under different inputs or conditions.

2. Difficulty in achieving complex programs: For large and complex programs, achieving high statement coverage can be difficult and time-consuming, since there may be many possible execution paths and combinations of inputs to consider.

3. Focus on execution, not correctness: Statement coverage focuses on the extent to which code has been executed, rather than whether it has been executed correctly. In other words, statement coverage can tell us which parts of the code have been executed, but it does not provide any guarantee that the code is working correctly.

4. False sense of security: Achieving high statement coverage can give testers a false sense of security, leading them to believe that the code is free of faults when in fact some may still be present.

5. Lack of guidance on test case selection: Statement coverage does not provide guidance on which test cases to select to achieve the desired coverage, which can make it difficult for testers to develop effective test suites.

📍 Decision coverage

Decision coverage, also known as branch coverage or ramification coverage, is a metric used in structural testing that measures the degree to which each decision point in the code has been exercised during testing. It is an extension of statement coverage that focuses on the control flow of the program, rather than just the individual statements.

To achieve decision coverage, testers need to ensure that every possible branch in the code has been executed at least once. This means generating test cases that exercise each decision point in the code in both the true and false directions. This includes branches that are not explicitly covered by if/else statements, such as null branches and default cases in switch statements.

💡 Remarks

• The decision means any branch in the graph, even when it does not appear explicitly in the program.

• For example, for the construction of for i:= 1 to n from Pascal, the default condition is i<=n.

📍 Condition coverage

To achieve condition coverage, testers need to ensure that each condition within a decision has been evaluated as both true and false. For example, if a decision takes the form of c1 || c2 or c1 && c2, then condition coverage is achieved by testing each condition c1 and c2 in both true and false directions.

Condition coverage is useful in cases where certain conditions within a decision are more critical or complex than others, as it allows testers to focus their efforts on these conditions. However, it should be noted that achieving 100% condition coverage does not guarantee that all potential faults in the code have been detected, as it does not take into account the interaction between different conditions or inputs.

💻 Example

Using the code example with CharacterSearch from the beginning...

• (n, x, c, s) = (0, , , _)

• (n, x, c, s) = (25, , , _)

• (n, x, c, s) = (1, a, a, y)

• (n, x, c, s) = (_, _, b, Y)

🔹 Condition coverage weaknesses

While condition coverage can help detect certain faults, it is not a foolproof method and has its weaknesses. One such weakness is that it may not necessarily achieve branch coverage, as it only focuses on individual conditions within a decision. For example, the data used in a condition coverage test may not result in the program executing all possible branches of a decision.

For instance, consider the code snippet below:

while (response == 'y' || response == 'Y') {
    // some code here
}

If we only focus on condition coverage and test with data where the condition (response == 'y' || response == 'Y') evaluates to true, the loop will continue indefinitely and we may miss testing the exit condition. In this case, a more comprehensive testing approach that includes decision or branch coverage may be required to ensure all possible code paths are executed.

To overcome this weakness, testers can use other testing techniques in conjunction with condition coverage, such as a decision or branch coverage, to ensure more comprehensive coverage of the code. By using multiple testing techniques in combination, testers can increase their chances of detecting faults in the code.

📍 Condition/decision coverage

Generate test data so that each condition in a decision must take both a true value and a false value (if possible) and every decision must also take both a true value and a false value.

• (n, x, c, s) = (0, _, _, _)

• (n, x, c, s) = (25, _, _, _)

• (n, x, c, s) = (1, a, a, y)

• (n, x, c, s) = (_, _, b, y)

• (n, x, c, s) = (_, _, B, n) produces the false value for the remaining global condition ((response=='y') ||(response='Y')

📍 Multiple condition coverage

Multiple condition coverage is a structural testing technique that aims to test all possible combinations of truth values for the individual conditions within a decision. This testing technique ensures that all possible combinations of truth values for the conditions are tested, thus providing more thorough coverage than other testing techniques like condition coverage.

For example, if a decision has three conditions, A, B, and C, then multiple condition coverage requires testing all eight possible combinations of true and false values for each condition: A=true, B=true, C=true; A=true, B=true, C=false; A=true, B=false, C=true; A=true, B=false, C=false; A=false, B=true, C=true; A=false, B=true, C=false; A=false, B=false, C=true; A=false, B=false, C=false.

While multiple-condition coverage provides more thorough coverage than other testing techniques, it can result in a large number of test cases, especially when the number of conditions is large. This is because the number of possible combinations grows exponentially as the number of conditions increases. Hence, there may be a trade-off between coverage and the practicality of generating and executing the required number of test cases.

📍 Modified condition/decision (MC/DC) coverage

Modified Condition/Decision Coverage (MC/DC) is a structural testing technique that ensures each condition in a decision statement independently affects the outcome of the decision. MC/DC requires that each condition be tested with all possible combinations of the other conditions and that each condition should be evaluated to be both true and false at least once.

In MC/DC, each condition should be evaluated in such a way that it changes the outcome of the decision statement, and the other conditions should be held constant. This technique helps to identify and test the unique decision points within a program.

MC/DC is commonly used in safety-critical applications such as aviation and medical devices, where errors can have serious consequences. This technique provides a higher level of assurance that the software is functioning correctly and meets the required safety standards.

A test set satisfies MC/DC coverage when:

• Each condition in a decision takes both a true value and a value fake;

• Each decision takes both a true and a false value;

• Each condition independently influences the decision from which side face;

There are several advantages to using Modified Condition/Decision Coverage (MC/DC) in structural testing:

1. Thorough Testing: MC/DC requires that each condition be tested with all possible combinations of the other conditions, ensuring that all decision outcomes are tested. This technique provides a thorough and rigorous testing approach that can identify potential errors or bugs.

2. Effective Fault Localization: MC/DC can help in localizing the faults by identifying the specific condition(s) that caused a failure or error. This can reduce the time and effort needed to locate and fix the issue.

3. Better Quality Assurance: MC/DC is commonly used in safety-critical applications such as aviation and medical devices, where errors can have serious consequences. By using this technique, the software can be tested to a higher level of assurance and meet the required safety standards.

4. Reduced Maintenance Costs: MC/DC testing can help identify issues earlier in the development cycle, reducing the cost and effort required to fix issues in later stages of the software development lifecycle.

5. Improved Test Efficiency: By focusing on the unique decision points within a program, MC/DC testing can help reduce the number of tests required while still providing thorough coverage. This can improve test efficiency and reduce testing time and effort.

💻 Examples of test cases

1. Using AND operator in MC/DC:

t1 and t3 covers C1

t1 and t2 covers C2

1. Using OR operator in MC/DC:

t2 and t3 covers C1

t1 and t3 covers C2

1. Using XOR operator in MC/DC:

t2 and t3 covers C1

t1 and t2 covers C2

💻 And a more complex example: C = C1 ∧ C2 ∨ C3

📍Conclusion

In conclusion, structural testing plays an essential part in ensuring the quality and reliability of software systems. The techniques described in this article allow for a thorough analysis of the internal structure of the software, ensuring that all possible paths and decision points are tested.