Strides Interactive

Software Quality - Part 11Filed Under: Weekly Tuesday Dose of goodness

Hi all,

This week we’ll talk about stubs in greater detail. We’ve seen how stubs can be a useful helper and there’re many types of stubs as well.

We’ll also need to cover some hidden dangers of stubs and mock objects as well.

So what’s so dangerous about stubs and mock objects anyway? Let’s find out.

Introduction

In this article we’ll talk about several aspects of stubs and how we can make full use of them to our full advantage.

First of all, as explained in my previous posts, stubs and mock objects are temporary assets which help bridge functionalities or mock state representation of a certain module.

We’ll talk about how much efforts should we invest in such temporary assets, what types of stubs and mock objects do we have and the dangers involved in using such assets.

Efforts vs Effects

Looks familiar isn’t it? This is the name of one of the post that was published a long long time ago.

When it comes to manual processes, stubs and mock objects stand in a single line. That’s because, if I can implement a mock object, I might as well do the stubs inside the mock object so that I can save time for others.

That’s a pretty standard and likely clever way of doing things. The only problem is - the code has to know of the existence of your mock objct.

Thus the amount of efforts come not just from coding the mock object and integrating it but also from maintaining the mock object integrations.

For example, you’ll have to account for how many integrations to this mock object is done so that you’ll know where to remove them when the project is finally ready for SIT or UAT. Very likely SIT (System Integrated Testing).

When you actually remove the mock objects from integration, you’ll begin to feel that it was such an inefficient use of time. Objects that are coded with care are now poised for removal from project, possible complete deletion once the project finishes.

And the effects? Well - it served it purpose more than less.

One more thing we’ll have to take in mind is that if we’re doing C++, then extra care must be taken when coding this object so that it doesn’t cause memory leaks or stub-triggered memory problems.

It’s like - nursing a pet from a young till its teen years and then slaughter it for no apparent reason or usage. That hurts.

Stubbing/Mock Object Framework vs Manual Efforts

Naturally, a stubbing or mock object framework can prove to be very useful in terms of helping the stub/mock object programmer to create such assets with ease. However, for such a framework to be viable, it must at least have the following qualities:

1) Ease of use
2) Visibility of what’s stubbed, what’s not
3) Ease of management
4) Auto-integration
5) Does not TAINT the original source codes

Let’s talk about all the 5 qualities.

Ease of Use

Naturally, this is one of the expectations when it comes to stubbing frameworks. However, the notion of “ease” can be subjective. For one, “ease” could be a code snippet generator, while for others, “ease” could be as complicated as a stub management system.

Visibility

Anyone who does stubs will be concerned as to which method is being stubbed out or which objects are passed in as mock objects. This is to be able to know the exact ingredients used for a particular white-box test case so that the end results will be correct still.

Ease of Management

Naturally, once the locations of all the stubs and mock objects are known, the next step will be the ease of accessing or removing them in a list. Thus removing the need for stub programmers to write/remove/search for declarations and definitions manually.

Auto-integration and taint-free source codes

This is one of the hardest things to do in stubs. Auto-integration means that quite a lot has to be done. There’re several approaches here but in my opinion, the best auto-integration will be a seamless one, ie, integration without any changes to the original source codes.

This is only possible with instrumentation.

Stub Types

Now, let’s put mock objects down and go into stubs itself. As we know, stubs by definition are nothing but methods returning a dummy value.

However, even so, there’re several types of stubs. Namely:

1) Original
2) User
3) Safe
4) Virtual

So what are they?

Original - Not stubs. It’s really just original source codes
User - User-defined stubs, the ones that you’ll write
Safe - Functions that are deemed to be dangerous by the stubbing framework will automatically stub them out for you
Virtual - It’s to indicate that this stub is based on a virtual method call

A good stubbing framework will allow several types of tests to be done. They’re namely:

1) Backdoor testing via circumvention
2) Specific return value via circumvention
3) Controlling who gets the stub value, who gets the real value
4) Exception testing

The word ‘circumvention’ appeared twice. So what exactly is circumvention? For the benefit of those whose command of the English language may not as good as others, circumvention essentially means going around.

So what are we going around? Well, the question is answered by why are we doing stubs in the first place. The answer? White Box Test Cases!

Let’s take a look at the 4 types of testing.

Backdoor Testing

I’ve mentioned these type of testing many times in my previous articles. The reason for this type of testing to exist is because of the inherent limitations of white box testing. As we know, white box testing primarily involves methods that allows arguments to be passed in.

Thus, methods that pull values using the Pull Approach will be nearly impossible to test.

The solution to this problem is backdoor testing and the ingredient to this solution is stubs.

Specific Value via Circumvention

This method is closely tied to backdoor testing with the exception that the former is done because there’s no other choice. The former is done because the original codes cannot reproduce the values required for the test.

Such scenarios are common among pull approaches that pull values from the hardware. A good scenario will be the commercial jetliner autopilot system. This system makes use of readings from the instruments to make certain decisions, ie, to ascend or descend.

Thus, if there’s a test to test such a condition, it’ll be impossible to just simply power the hardware on and hope that the plane can be simulated at 15000 feet. This is provided that the hardware even comes with a simulator or simulation device.

Remember here, we’re testing codes, not integration here - thus if real hardware could be avoided, they should be avoided.

Thus, circumvention comes in handy.

Stub Return Value Control

One of the most important aspects of stubs is to be able to return a different return value based on a certain predicate. In terms of white box testing, the most direct predicate will be the test case name. If registered correctly, the test case name can be used to check if the test runner is running a particular test case.

Once this condition is fulfilled, the stub will then be able to return the desired value.

It can also reconnect to the original source codes and return the original value instead if desired.

Exception Testing

One of the hardest aspects in unit testing is to trigger an exception. As we know, exceptions are thrown for unexpected behavior. Therefore, there’re 2 ways to generate this unexpected behavior.

1) Create the scenario to generate this behavior, ie, data reading from thumb drive and you pull it out to trigger an IO Exception.
2) Use stubs

Naturally, the use of stubs is a more enticing proposition than to physically test the code via manual means. This usage is naturally evolved from specific return values and stub control.

Based on test case predicates, the stub can actually check and throw exceptions at certain test cases to simulate an exception scenario where the catch handling code comes into play.

On top of that - it allows you to see how the exception is being handled and how much coverage is derived from it.

Conclusion

Again, it has been a lot of information this week. I’m afraid I’d have to leave the dangers of using stubs behind till next week.

One thing I’d like to note. The stubs we’ve been talking so far all belong to the same stub approach known as Method-Specific Stubs.

There’s another stub approach known as Test-Specific Stubs.

The differences are -
1) Method specific stubs allows pipe-blockage of all the method calls to the method it stubs out
2) Test specific stubs only work for the test case it’s tied to

We’ll however, not talk abt test specific stubs.

That’s it for this week. Please have a great weekend ahead!

Signing off,
Jeremy

Series Navigation

«Software Quality - Part 10Software Quality - Part 12»