Wednesday, December 5, 2018

Testing Python code

Unfortunately, mistakes are unavoidable and there is always some bug that sneaks into our code. But we definitely want our code to be as predictable as possible. What we don't want is to have a surprise, our code behaving in an unpredictable way. Therefore we need to test our code, we need to check that its behavior is correct, that it works as expected when it deals with edge cases, that it doesn't hang when the components it's talking to are down, that the performances are well within the acceptable range, and so on.

In this post we're going to explore testing, including a brief introduction to test-driven development (TDD). Next we'll focus on testing our code using tools in Python’s unittest module, learn to build a test case and check that a set of inputs results in the output we want. Thus we will learn to test functions and classes, and try to understand how many tests to write for a project.

Testing your application

There are many different kinds of tests, to start making an initial classification, we can divide tests into two broad categories: white-box and black-box tests.

White-box tests are those which exercise the internals of the code, they inspect it down to a very fine level of granularity. On the other hand, black-box tests are those which consider the software under testing as if being within a box, the internals of which are ignored. Even the technology, or the language used inside the box is not important for black-box tests. What they do is to plug input to one end of the box and verify the output at the other end, and that's it.

There are many different kinds of tests in these categories, each of which serves a different purpose. Here's a few:

• Front-end tests make sure that the client side of your application is exposing the information that it should, all the links, the buttons, the advertising, everything that needs to be shown to the client. It may also verify that it is possible to walk a certain path through the user interface.

• Scenario tests make use of stories (or scenarios) that help the tester work through a complex problem or test a part of the system.

• Integration tests verify the behavior of the various components of your application when they are working together sending messages through interfaces.

• Smoke tests are particularly useful when you deploy a new update on your application. They check whether the most essential, vital parts of your application are still working as they should and that they are not on fire. This term comes from when engineers tested circuits by making sure nothing
was smoking.

• Acceptance tests, or user acceptance testing (UAT) is what a developer does with a product owner (for example, in a SCRUM environment) to determine if the work that was commissioned was carried out correctly.

• Functional tests verify the features or functionalities of your software.

• Destructive tests take down parts of your system, simulating a failure, in order to establish how well the remaining parts of the system perform. These kinds of tests are performed extensively by companies that need to provide an extremely reliable service, such as Amazon, for example.

• Performance tests aim to verify how well the system performs under a specific load of data or traffic so that, for example, engineers can get a better understanding of which are the bottlenecks in the system that could bring it down to its knees in a heavy load situation, or those which prevent scalability.

• Usability tests, and the closely related user experience (UX) tests, aim to check if the user interface is simple and easy to understand and use. They aim to provide input to the designers so that the user experience is improved.

• Security and penetration tests aim to verify how well the system is protected against attacks and intrusions.

• Unit tests help the developer to write the code in a robust and consistent way, providing the first line of feedback and defense against coding mistakes, refactoring mistakes, and so on.

• Regression tests provide the developer with useful information about a feature being compromised in the system after an update. Some of the causes for a system being said to have a regression are an old bug coming back to life, an existing feature being compromised, or a new issue being introduced.

Testing is an art, an art that we don't learn from books but from experience.When you are having trouble refactoring a bit of code, because every little thing you touch makes a test blow up, you learn how to write less rigid and limiting tests, which still verify the correctness of your code but, at the same time, allow you the freedom and joy to play with it, to shape it as you want.

When you are being called too often to fix unexpected bugs in your code, you learn how to write tests more thoroughly, how to come up with a more comprehensive list of edge cases, and strategies to cope with them before they turn into bugs.

When you are spending too much time reading tests and trying to refactor them in order to change a small feature in the code, you learn to write simpler, shorter, and
better focused tests.

Define a test.

Before we concentrate on unit tests, let's see what a test is, and what its purpose is.

A test is a piece of code whose purpose is to verify something in our system. It may be that we're calling a function passing two integers, that an object has a property called donald_duck, or that when you place an order on some API, after a minute you can see it dissected into its basic elements, in the database.

A test is typically comprised of three sections:

• Preparation: This is where you set up the scene. You prepare all the data, the objects, the services you need in the places you need them so that they are ready to be used.

• Execution: This is where you execute the bit of logic that you're checking against. You perform an action using the data and the interfaces you have set up in the preparation phase.

• Verification: This is where you verify the results and make sure they are according to your expectations. You check the returned value of a function, or that some data is in the database, some is not, some has changed, a request has been made, something has happened, a method has been called,
and so on.

How to write good tests

In order to write good tests, here are some guidelines:

• Keep them as simple as possible: It's okay to violate some good coding rules, such as hardcoding values or duplicating code. Tests need first and foremost to be as readable as possible and easy to understand. When tests are hard to read or understand, you can never be sure if they are actually making sure your code is performing correctly.

• Tests should verify one thing and one thing only: It's very important that you keep them short and contained. It's perfectly fine to write multiple tests to exercise a single object or function. Just make sure that each test has one and only one purpose.

• Tests should not make any unnecessary assumption when verifying data: This is tricky to understand at first, but say you are testing the return value of a function and it is an unordered list of numbers (like [2, 3, 1]). If the order in that list is random, in the test you may be tempted to sort it and compare it with [1, 2, 3]. If you do, you will introduce an extra assumption on the ordering of the result of your function call, and this is bad practice. You should always find a way to verify things without introducing any assumptions or any feature that doesn't belong in the use case you're
describing with your test.

• Tests should exercise the what, rather than the how: Tests should focus on checking what a function is supposed to do, rather than how it is doing it. For example, focus on the fact that it's calculating the square root of a number (the what), instead of on the fact that it is calling math.sqrt to do it (the how). Unless you're writing performance tests or you have a particular need to verify how a certain action is performed, try to avoid this type of testing and focus on the what. Testing the how leads to restrictive tests and makes refactoring hard. Moreover, the type of test you have to write when you concentrate on the how is more likely to degrade the quality of your testing code base when you amend your software frequently.

• Tests should assume the least possible in the preparation phase: Say you have 10 tests that are checking how a data structure is manipulated by a function. And let's say this data structure is a dict with five key/value pairs.

If you put the complete dict in each test, the moment you have to change something in that dict, you also have to amend all ten tests. On the other hand, if you strip down the test data as much as you can, you will find that, most of the time, it's possible to have the majority of tests checking only a partial version of the data, and only a few running with a full version of it. This means that when you need to change your data, you will have to amend only those tests that are actually exercising it.

• Test should run as fast as possible: A good test codebase could end up being much longer than the code being tested itself. It varies according to the situation and the developer but whatever the length, you'll end up having hundreds, if not thousands, of tests to run, which means the faster they run,
the faster you can get back to writing code. When using TDD, for example, you run tests very often, so speed is essential.

• Tests should use up the least possible amount of resources: The reason for this is that every developer who checks out your code should be able to run your tests, no matter how powerful their box is. It could be a skinny virtual machine or a neglected Jenkins box, your tests should run without chewing up too many resources.


Testing a function

The first requirement is to have a code to test this we fulfill by making a display_name() function which takes in a first and last name, and returns a neatly formatted full name. The code is shown below:

def display_cname(fname,lname):   
   
    cname = fname +' '+ lname
    return cname.title()


Save this file as display_name.py in your working directory. Now let’s make a program that uses this function. The program complete_name.py lets users enter a first and last name, and see a neatly formatted full name. The code is shown below:

from display_name import display_cname

first_name = input("Enter your first name: ")

last_name = input("\nEnter your last name: ")

complete_name = display_cname(first_name,last_name)

print("\nYour complete name is " + complete_name)


This program imports display_cname() from display_name.py. The user can enter a series of first and last names, and see the formatted full name that is generated:



Now what if we have people with middle names. We need to modify our display_cname() to handle middle names as well as make sure it handles names that have only a first and last name. One way is to modify our display_cname() function and test by running display_name.py by entering a name or we can automate the testing of a function’s output. First approach will be a tedious one hence we'd prefer the second one.

If we automate the testing of display_cname(), we can always be confident that the function will work when given the kinds of names we’ve written tests for. This is where unit testing comes into picture.

Unit testing

Python standard library provides the module unittest which has the tools for testing your code. A unit test verifies that one specific aspect of a function’s behavior is correct.

To write a test case for a function, import the unittest module and the function you want to test. Then create a class that inherits from unittest.TestCase, and write a series of methods to test different aspects of your function’s behavior.

Here’s a test case with one method that verifies that the function display_cname() works correctly when given a first and last name:

import unittest

from display_name import display_cname

class CnameTestCase(unittest.TestCase):
   
    def test_first_last_name(self):
       
        complete_name = display_cname('veevaeck','swami')
        self.assertEqual(complete_name,'Veevaeck Swami')
       
unittest.main()



Our program starts with importing the unittest module and the function wewant to test, the display_cname(). Next we create a class called CnameTestCase, which will contain a series of unit tests for display_cname(). This class must inherit from the class unittest.TestCase so Python knows how to run the tests you write.

CnameTestCase contains a single method that tests one aspect of display_cname(). We call this method test_first_last_name() because we’re verifying that names with only a first and last name are formatted correctly.

Any method that starts with test_ will be run automatically when we run test_display_name.py. Within this test method, we call the function we want to test and store a return value that we’re interested in testing. In this example we call display_cname() with the arguments 'veevaeck' and
'swami', and store the result in complete_name.

Now comes the key part of the program, we use one of unittest’s most useful features: an assert method. Assert methods verify that a result you received matches the result you expected to receive. In this case, because we know display_cname() is supposed to return a capitalized, properly spaced full name, we expect the value in complete_name to be Veevaeck Swami. To check if this is true, we use unittest’s assertEqual() method and pass it complete_name and 'Veevaeck Swami'.

The last line unittest.main() tells Python to run the tests in this file. When we run test_display_name.py, we get the following output:




The dot on the first line of output tells us that a single test passed. The next line tells us that Python ran one test, and it took less than 0.000 seconds to run. The final OK tells us that all unit tests in the test case passed.

This output indicates that the function display_cname() will always work for names that have a first and last name unless we modify the function. When we modify display_cname(), we can run this test again. If the test case passes, we know the function will still work for names like Veevaeck Swami.

Let's modify our display_cname() so it can handle middle names, but we’ll do so in a way that breaks the function for names with just a first and last name, like Veevaeck Swami.

The modified display_cname() is shown below:

def display_cname(fname,mname,lname):   
   
    cname = fname +' '+ middle +' '+ lname
    return cname.title()



This version should work for people with middle names, but when we test it, we see that we’ve broken the function for people with just a first and last name. This time, running the file test_display_name.py gives this output:



The first item in the output is a single E which tells us one unit test in the test case resulted in an error. Next, we see that test_first_last_name() in CnameTestCase caused an error. The standard traceback, reports that the function call display_cname('veevaeck','swami') no longer works because it’s missing a required positional argument, which is the middle name.

Finally, we see an additional message that the overall test case failed and that one error occurred when running the test case.

Fixing the code

When a test fails, don’t change the test. Instead, fix the code that caused the test to fail. Examine the changes you just made to the function, and figure out how those changes broke the desired behavior.

In our case display_cname() used to require only two parameters: a first name and a last name. Now it requires a first name, middle name, and last name. The addition of that mandatory middle name parameter broke the desired behavior of display_cname(). So what shall fix this function so that it can handle both two parameters and three parameters? Just make the middle name optional so that our test for names like Veevaeck Swami should pass again, and we should be able to accept middle names as well.

See the modified display_cname() which makes middle names optional.

def display_cname(fname,lname,mname=''):
   
    if mname:
       
        cname = fname +' '+ middle +' '+ lname
       
    else:
       
        cname = fname +' '+ lname
       
           
    return cname.title()


To make middle names optional, we move the parameter mname to the end of the parameter list in the function definition and give it an empty default value. We also add an if test that builds the full name properly, depending on whether or not a middle name is provided. Now the display_cname() should work for both kinds of names. This we can verify by running the file test_display_name.py which gives this output for Veevaeck Swami :



The output shows that the function works for names like Veevaeck Swami again without us having to test the function manually. Fixing our function was easy because the failed test helped us identify the new code that broke existing behavior.

Now let’s write a second test for people who include a middle name. We do this by adding another method to the class CnameTestCase for which the code is shown below:

def test_first_last_middle_name(self):
      
        complete_name = display_cname('veevaeck','swami','kumar')
        self.assertEqual(complete_name,'Veevaeck Kumar Swami')



The new test_first_last_middle_name() runs automatically when we run test_display_name.py. To test the function, we call display_cname() with a first, last, and middle name and then we use assertEqual() to check that the returned full name matches the full name (first, middle, and last) that we expect. When we run test_name_function.py again, both tests pass as shown in the output below:



What will be the output if my code snippet for second test is:

def test_first_last_middle_name(self):
      
        complete_name = display_cname('veevaeck','kumar','swami')
        self.assertEqual(complete_name,'Veevaeck Kumar Swami')



Incorporate this in your program and see the output. Also make few more functions and create test cases for them. Here we end today's discussion and in the next post we'll focus on testing a class and the remaining topics that were supposed to be covered today. So till we meet next keep practicing and learning Python as Python is easy to learn!










Share:

0 comments:

Post a Comment