Skip to content
Science, Maths & Technology
Author:

Testing Programs

Updated Thursday 25th June 2015

Find out about some good ways to test your software, and why testing is important.

Testing your code

Computer program Copyrighted image Icon Copyright: Dreamstime After writing or modifying a program, you should always test it to see whether it (still!) works. Even when making very small changes it's easy to accidentally introduce errors into code that was working! You can't possibly run a program for every possible input value (there are, for example, infinitely many numbers), so it's important to choose a set of test inputs that are likely to find any errors. For if statements, for example, you should test with at least two values, one which makes the condition true, and one which makes it false. This will ensure that you test both possible routes through the program: the route by which the statement(s) within the braces are executed, and the route by which they are ignored.

Activity 1

When you change your computer password, the program usually requires you to enter a new password, and then retype it (to ensure that you haven't made typing errors in your original entry). If the two entries don't match, the password isn't changed.

(a) Open testing_activity_1.html in your preferred text editor and modify it so that the program prompts you to enter a password, and then prompts you to re-enter the password. It then checks whether the two entries match. If they don't, the program displays Your password is unchanged. Whether or not they match, it displays Thank you on a new line on the screen. The program is similar in structure to the bank program, but you will need to change most of the details. There are some hints following part (b) if you need them.

(b) How would you test whether your program works?

Hints

  • What do you need to do before the if statement?
  • What is the required Boolean expression in the if statement?
  • What do you need to do if it evaluates to true?
  • What do you need to do afterwards?
  • Before the if statement, you need to prompt the user to enter a password, and then re-enter it.
  • You will need to declare two variables, one for the original password entry and one for the retyped password.
  • If the values of the two variables don't match, you should display Your password is unchanged. You may have to look at the table of operators (Table 3.1) to remind yourself of the JavaScript symbol for 'not equal to'.
  • You need to finish off the program with a further line of output.

If you're still stuck, see the solution for our version of the code and check how it differs from your own.

Solution

(a) Here is our code (yours might differ in details).

var password,repeatPassword;
password = window.prompt('Please enter your password','');
repeatPassword = window.prompt('Please re-enter your
password','');
if (password != repeatPassword)
{
    document.write('Your password is unchanged' + '<BR>');
}
document.write('Thank you');

 

You will find a working solution in testing_solution_1.html.

(b) In order to test your code, you would need to run the program at least twice, once when the original password and the retyped password don't match, so (password != repeatPassword) is true, and once when they do, so that the condition is false.

Testing Methods

Suppose you have written a program. All syntax errors have been eliminated and the program runs, accepts inputs and produces output which as far as you can see is right. But this is absolutely no guarantee that the program is correct. For all you know it may work fine most of the time but fall down spectacularly for certain particular input values. The only way you can have a reasonable level of assurance that the program is correct is to do some properly designed and systematic testing.

When you test a program what you are looking for is evidence that the program does what it is supposed to and doesn't do anything it isn't supposed to. The basic idea is to choose some suitable test data and work out by hand what results the program ought to give for these values. Then you run the program, input the test values, and see if the results are what you expected.
Note that the form of testing we are describing is based entirely on the specification – what the program is meant to do – not on how it does it. This is often called black-box testing, because it looks at the program from the outside, without considering the internal details of the code. We'll look at doing some black-box testing in Activity 2.

It's seldom possible to test a program with all possible inputs. There are usually far too many possibilities. For example if a JavaScript program accepts an input which is an integer then it can be anything between – 9007199254740992 and 9007199254740992. At one test per second it would take approximately 300 million years to work through them all.

So the tests we actually do will only represent a tiny sample and it's important that as well as trying some 'well-behaved' inputs – ones which are unlikely to cause problems – we also systematically choose ones that are likely to expose any flaws in the code. We need to think what could go wrong and then deliberately pick values that might cause the program to fail.

The design of software tests is an entire subject in its own right and we only have space here for a very brief discussion. We shall consider two types of test: using boundary values and extreme values.

Boundary values

When a program is supposed to change its behaviour at a threshold value – a boundary value – the programmer can easily go wrong when coding this. For example they might choose the wrong comparison operator and use >= where it should be >.

To detect errors of this kind we should test at the actual boundary and also at the two closest values on either side.

Extreme values

Extreme values are inputs which are valid and ought to be dealt with correctly but which are in some way a special case and may cause the program to fail just at that value. For instance if a program accepts an input which is a string, an extreme value would be an empty string. We might also choose values which are valid but fall a long way outside the normal operating range, just to provide some assurance that if users input unusual values the program will still function as it should.

Activity 2

We have written a program that is supposed to meet the following specification.

''The postal service in the country of Erewhon has two prices for letter post.
Letters up to and including 50g cost 25 Erewhon cents.
Letters over 50g but not over 100g cost 35 Erewhon cents.
Any item exceeding 100g counts as a package.
The program should accept an input from the user, which can be assumed to represent a valid number that is not negative. This is the weight in grams of a postal item.
The program should output a suitable message stating the price the item will cost to send as a letter, or if the weight limit for letters is exceeded inform the user of this fact and request that they refer to the rates for sending packets.''

However we have included a deliberate mistake. Your task is to run blackbox tests that will detect the fault.

Download and open up testing_activity_2.html in your browser. Next, carry out the tests in the table below in order to try and ascertain what's not working properly.

Test NumberType of TestInputExpected Result1Extreme0252Well-behaved25253Boundary49254Boundary50255Boundary51356Well-behaved75357Boundary99358Boundary100359Boundary101Refer user to packet post10Extreme1000Refer user to packet post

Notice that the specification does not say the user cannot input 0, even though it is not a very sensible value, so the program should be able to handle this case and we have included it as an extreme value. We have also included an extreme value of 1000, as an example of an unusual value that a user could theoretically input and which the program ought to deal with correctly.

When you have run these ten tests you should have detected a fault and be able to work out its cause. See if you can correct the program code and re-run the test that failed to check that the output is now as expected.

Solution

In the original version, entering '100' as the input returns a referral to packet post, instead of the 35 cents we were expecting. This is because the if statement in the original program is checking for

 if (weight < 100)

instead of

 if (weight <= 100)

You should now be much more familiar with a few of the different methods available to you to test and fix your programs.

if (familiarWithIfStatements = true)
{
    document.write('Move onto our next article on the if...else Statement');
}

... or recap on some Simple Conditional Statements.

 

Author

Ratings

Share

Related content (tags)

Copyright information

For further information, take a look at our frequently asked questions which may give you the support you need.

Have a question?