Online Course – Unit Testing and Test Driven Development in Python

Although there’s a plethora of articles and books going over the basics of unit testing and Test Driven Development, I’ve found that there’s a lack of quality examples of how to apply these disciplines in practice.  This causes many developers to struggle with how to start using them in their every day coding.

With that in mind I’ve created another course on the Udemy platform for unit testing and test driven development with Python (the second most used language on my team). The course goes over unit testing in general, the TDD workflow, PyTest, mocking with unitest.mock, and TDD best practices.  I also provide several hands on coding sessions to try and show how TDD works when you’re really writing code in Python.

If you’d like to take a look here’s a link with a coupon code at a discounted price of $9.99 (regularly $30): Unit Testing and Test Driven Development in Python

If you do take a look please let me know what you think and if you have any suggestions for improvements.  You can also provide that feedback and any other review comments on Udemy.

Here’s the link to my previous post on Unit Testing and TDD in C++: Online Course – Beginning Test Driven Development in C++

The Benefits of Unit Testing and Test Driven Development

Unit testing can be one of the most effective ways for a software developer to increase the quality of their code.  Test driven development is a coding practice that helps ensure developers are writing good unit tests but also helps with design and makes coding faster in general.  In the video below I go over these benefits and others.

FizzBuzz Test Driven Development Code Kata in C++

I’ve posted a youtube video of a real Test Driven Development coding session of the FizzBuzz code kata using C++.  This is a pretty simple code kata and serves as a gentle introduction to the process of TDD.  I used the Eclipse CDT with Google Test as the unit testing library.  Please let me know if you’d like to see more videos like this (other katas or other programming languages).  Thanks!

Online Course – Beginning Test Driven Development in C++

I’ve been pushing my team to use TDD in their day to day coding work for a while now, but it’s only recently come to light that most of the developers on my team don’t really feel they understand how to practice TDD and could really use some training.  To that end, I’ve put together an online introductory course on Test Driven Development with C++.  Here’s the link if you’d like to take a look: https://www.udemy.com/beginning-test-driven-development-in-c/learn/v4/overview

The course goes over unit testing in general, the TDD workflow, Google Test and Google Mock, and TDD best practices.  I also provide several hands on coding sessions to try and show how TDD works when you’re really writing code.

If you do take a look please let me know what you think and if you have any suggestions for improvements.  You can also provide that feedback and any other review comments on Udemy.

How Code Coverage Helps Validate Your Unit Tests

What Is Code Coverage?

Unit testing is an important addition to your development process to ensure that your production code is high quality and bug free.  But how do you know your unit tests are adequately validating the production code?  How do you know that you’ve tested all of the positive and negative test cases?  How do you know that there isn’t an uncaught exception for an obscure error case?  While not a complete answer to the quality of your unit tests, code coverage tools can be a huge help by telling you how much of your production code the unit tests are actually exercising.  There are several types of code coverage analysis and this article we’ll go over a few of the most common.

Line Coverage

The simplest type of code coverage analysis is line coverage.  This type of code coverage tells you if a specific line in the production code was executed or not during the test.  Here’s an example of line coverage:

    int doSomething( string str ) {
1        int a, b;
2        a = std::stoi(str);  b = a + 5;
3        if( b == 10 )
4            return 0;
5        else
6            return 1;
    }
// 6 lines, 6 covered = 100% coverage 

Statement Coverage

A slight variant to this is known as statement coverage.  Statement coverage takes things a step further to tell you if each statement was executed.  This type of code coverage will identify multiple statements on a particular line and will validate that each statement is executed.  The example below is the same code as above but the code coverage tool recognizes that there are two statements on the second line and validates that they are both executed.  

      int doSomething( string str ) {
1         int a, b;
2, 3      a = std::stoi(str);  b = a + 5;
4         if( b == 10 )
5             return 0;
6         else
7             return 1;
      }
// 7 statements, 7 covered = 100% coverage 

Branch Coverage

Another common type of code coverage is branch coverage.  This type of coverage tells you if all of the decision paths in the production code have been taken.  These are things like if/else,  or switch/case statements.  Arguably, this can also be determined from line and/or statement coverage by visually verifying that each section is executed.  The benefit of branch coverage is that it automatically tells you if all the control blocks in your code have been executed. Also, you can easily have a high line/statement coverage with a relatively low branch coverage.  Consider the  following example:

    int doSomething( string str ) {
1       int a = 0;
2       int b = 0;
3       try
        {
4           a = std::stoi(str);  
5           b = a + 5;
6           std::cout << “b = “ << b << std::endl;
        }
7       catch( const std::bad_alloc& e )
        {
8           return -1;
        }
9       if( b == 10 )
10          return 0;
11        else
12          return 1;
    }
// 12 statements , 11 covered = 92% coverage 

This code is being analyzed by statement coverage and has 92% coverage.  This doesn’t sound to bad.  But look at the same code analyzed with branch coverage:

    int doSomething( string str ) {
1       int a = 0;
        int b = 0;
        try
        {
2           a = std::stoi(str);  
            b = a + 5;
            std::cout << “b = “ << b << std::endl;
        }
        catch( const std::bad_alloc& e )
        {
3           return -1;
        }

        if( b == 10 )
4           return 0;
        else
5           return 1;
    }
// 5 control sections, 4 covered = 80% coverage 

 With branch coverage only four out of the five control sections are executed resulting in 80% coverage which is not nearly as good and would be a much more significant indicator that the unit tests are not testing the code thoroughly.

Additional Types of Coverage

 There are many additional types of code coverage that can be implemented:

  • Function Coverage – Gives the percentage of the functions in the production code that are executed during the testing.
  • Loop Coverage – Have all the loops in the code been executed at least once
  • Entry/Exit Coverage – Have all entries/exits of a function been executed
  • Parameter Value Coverage – Verify code execution with common range of values for the input parameters
  • Conditional Coverage – Verify each boolean statement is evaluated to both true and false.

 Many of these additional types of code coverage are typically only used with safety critical software as the the additional overhead and cost of the tools can outweigh the benefits they provide.  But if you’re not using any type of code coverage tools in your current development process then even simple line coverage would be a huge benefit in verifying that your unit tests are at least exercising the production in all the necessary test cases.  

Conclusion

 Unit testing along with other essential software development best practices (peer code reviews, object oriented design, static code analysis) is essential in making sure you have high quality and bug free production code.  Using code coverage analysis tools can help you ensure your unit tests are actually doing a thorough job and give you confidence in that safety net catching any bugs in the production code.

5 Things Your Software Development Team Can Do To Ensure Code Quality

Having high code quality is critical to your business.  If you have bugs regularly showing up in your production code you’ll be hurting your reputation with your customers.  In addition the development team be spending time that had been planned on implementing new features fixing those bugs which hurts your business.  Here are five things your development team can do to help ensure high code quality.

Write Automated Unit Tests

Unit tests validate the code is correct at the function level.  They test that each function in the production code behaves properly in all positive and negative test cases.  Code coverage tools which show the percentage of the code that’s been executed during a test run can help ensure that all the positive and negative test cases have been covered.  Automating the unit tests enables you to run those tests whenever there is a change checked in to your code base to ensure that change hasn’t broken anything.  Automated unit tests are the first level of safety nets for catching any bugs in your software before they get out to the field.

 

Practice Test Driven Development

If writing automated unit tests is critical then the process you use for writing the tests is just as critical.  Test Driven Development has the developer write a failing unit test before writing the production code.  Then just enough production code is written to make that unit test pass.  Once the tests are passing the developer goes back and refactors the test and the production code to make sure they are clean and following best practices and standards.  This is a short 2-3 minute cycle of changes that ensures that every line of production code has unit test code to back it up.  It also ensures as the developer is working that no other unit tests have been broken by any changes made to the production code.  Lastly, it helps drive to a clean and minimal implementation by having only enough code written to pass the currently failing test and refactoring to make things clean as you go.

Use Static Code Analysis Tools

Static code analysis tools should be run by the developer on all code changes before they are checked in to the baseline.  Static code analyzers can identify things like memory leaks, exceeding array bounds, and uninitialized variables which may be missed by the unit tests.  Some teams go so far as not allowing anything to be checked into the baseline that doesn’t have a clean run of the static analysis tools.

Automated Integration Tests

Automated integration tests verify that the code works properly when integrated into the real system.  This integration can happen at multiple levels.  At the component level where the classes and functions are integrated into a single executable binary or library and at the system level where a set of components are combined to make a system.  This can then be layered into systems of systems for complex products.  The automated integration tests should verify all the inputs and outputs at all of these levels.  The tests should mainly be verifying that things are “wired up” properly.  At the component level this ensures that class and function pointers are created and assigned properly.  At the system level this verifies that network and other communications paths between components have been setup properly.  After new features have been successfully integrated in the overall system the automated integration tests become the automated regression tests to validate that future changes haven’t broken anything in the integrated system.

Peer Code Review

The final thing that should be done is a code review of the changes by other members of the team.  This should include the code for the unit tests, the production code, and the integration tests.  The code reviewers should be checking to make sure the code meets the team’s standards for formatting and best practices, looking for common errors like memory leaks and uninitialized variables, and verifying the code implements the intended logic for the feature.  Code reviews are a very important tool in ensuring the quality of your code base as there’s nothing better than having another set of eyes look through the changes.  And it also helps keep other members on the team in sync on design issues and best practices.

Conclusion

These five items are what I’ve found have been the biggest contributors to maintaining code quality in the software projects I’ve worked on.  I’ve always found that the biggest pressure is to just get the code out the door as fast as possible.  But if you take short cuts and skip practices that ensure code quality you’ll end up with bugs showing up in front of your customer which can be worse than a feature being late.  If you establish these practices in your team and account for them in your initial schedule estimates you’ll have a lot more confidence when the production code goes out to the field that everything will go smoothly.

 

Test Driven Development – Best Practices

 

Test Driven Development is an essential discipline to follow to ensure you are writing code that works all the time.  But there are several things that can impede and even block you from being successful with it.  Below I have some best practices that can help you with following TDD and making it a worthwhile effort.

Always do the next simplest test case

 Even if you think you know exactly what the code is going to end up looking like for the functionality you’re implementing always try to keep yourself from jumping ahead in the testing.  Try to think about what the next simplest test case is and implement that first.  You want to build the functionality incrementally.  If you go to fast and try to skip some of the simpler and more obvious test cases you can find yourself at a place where you have to write a large chunk of code all at once to get the desired functionality.  Writing that big chunk of code all at once can be difficult and can lead to having a more complicated implementation than is necessary as you miss all the incremental refactoring that comes with implementing the simpler test cases first.

Use Descriptive Names for Your Test Cases

Making the code you write readable is hugely important as the code is likely going to be read 1000 times more than it’s going to be modified.  This is even more true for you unit test code as it is the best documentation for what you were trying to do with your production code.  One of the first steps in making your test code readable is having good and descriptive names for your test cases.  The name of the test suite should specify the class or function under test and the individual test names should describe in easily readable text what functionality is being tested.

One Logical Test Per Test Case

This goes hand in hand with well named test cases.  It can be difficult to write a good name for a test case if you’re trying to test many things in that one test.  Tests are generally broken up into three sections: arrange, act, and assert.  You set things up for what needs to be tested, you execute the code under test, and you assert the expected results.  If you go through more than one arrange, act, assert sequence in your test then you should see if it can be broken up into separate tests.

Use Code Coverage Tools

Code coverage tools generate a report from a unit test run telling you what parts of the production code were touched during the test run.  This can quickly tell you if you’ve missed any test cases.  If you don’t have 100% test coverage then there are test cases that you’re missing.  I’m not a proponent for doing unit testing on one line getter and setter methods, but those methods in a class that contain real logic should be tested with 100% code coverage.  100% code coverage doesn’t necessarily mean you’ve written all the tests you need for a particular method, but if you don’t have 100% code coverage then you know you haven’t.

Don’t Rely Solely On the Code Coverage Tools

Just because you have 100% code coverage that doesn’t necessarily mean that your testing is complete or correct.  Your test cases may be getting each line of the production code to execute, but that doesn’t ensure that your test is validating expected outputs or calls to other methods correctly.  You need to ensure that the logic implemented in your test for arranging, acting, and asserting is correct as well. This is one of the reasons why peer code reviews are so important.  

Keep Your Tests Fast

Test Driven Development is all about feedback.  It creates a very short feedback cycle so that for every little change you make to your code base you have immediate feedback that the change hasn’t broken anything.  If it takes 5 minutes to run your unit tests then you’ve lost that feedback loop as no one is going to wait 5 minutes to make sure they didn’t break anything after each small code change.  One easy approach to keep things fast is to only run the unit tests that are necessary (i.e. only the tests for the component or class that you’re working on rather than all of tests in your entire system).  Most testing frameworks provide command line options for filtering which tests should be executed for each test run.  In addition you should always make sure your tests run fast.  The set of tests you run as you’re going through the red, green, refactor cycle should execute in just a few seconds.  Things you do to keep your tests running fast are:

  • Keep console output to a minimum.  Each print statement slows things down and if you’re running hundreds or even thousands of tests each millisecond can count.  Just let the test framework which tests are failing.
  • Mock all system calls.  You shouldn’t do any system calls that will slow things down or even potentially block.  These are things like file system calls, timers, sockets, or database connections.  All of these should be mocked out (both to keep the tests fast and to be able to control the behavior of these calls for different test scenarios).

Run Your Tests Multiple Times and In Random Order

Most unit testing frameworks now have command line options for running the tests many times and for running the tests in a randomized order.  Doing this periodically helps ensure that your tests don’t have any dependencies between each other and also helps to identify any “flaky” tests that may fail intermittently.  One of the most useful things about having a comprehensive suite of unit tests for your code base is being able to run that test suite whenever your baseline changes to ensure that those changes didn’t break anything.  If you have flaky tests that are failing intermittently it can be hard to realize this benefit as you never know if a failure is because of a new code change or one of those flaky tests.

Conclusion

Test Driven Development can help you and your team go faster by always ensuring that your code is working and giving you the confidence to make changes to your code base.  But like all things it must be done properly to ensure you realize all the benefits.

Test Driven Development – The Basics

The most fundamental thing a developer can do to verify that the code they wrote is working properly is to write a unit test for it.  In this article I’m going to talk a little bit about what a unit test is and one of the best disciplines a professional software engineer can follow for writing unit tests: Test Driven Development (or TDD).

What are Unit Tests?

  So what is a unit test?  It seems to be an often confused term as I have many times heard co-workers and managers discussing “unit tests” that were really integration or end to end regression tests.  Unit tests are small, specific bits of code that are meant to test individual functions in isolation.  They are often organized into suites of tests that verify all the functions in a class.  They should verify that a particular function gives the proper outputs based on given known inputs.  This should be done for both positive and negative test cases.  If necessary unit tests can also verify that the code calls the correct underlying libraries with the correct inputs and in the correct order.

Running Unit Tests

 Unit tests are not tests that are written to be run in the target environment and test the inputs and outputs of your actual release builds.  Unit tests are compiled into a specific test component that will execute in your development environment.  It should have minimal dependencies so that it compiles and executes quickly.  The process of compiling and executing the unit tests for an individual components should happen in seconds (if not fractions of a second).  Things like disk IO, databases, and network connections should be “mocked out” (i.e. replaced with stubs of varying functionality).

So What is Test Driven Development?

 TDD enforces the writing of unit tests by requiring that the developer write the unit tests for the production code before writing the production code.  It’s a bizarre notion when you first read it.  But this doesn’t mean that TDD wants you to write ALL the tests and then write ALL the production code.  Instead it’s a cyclic process where:

  • You write a failing unit test.
  • You write some production code to make the test pass.
  • You refactor to make the code clean.

 And that’s it at it’s core.  If you know those three rules then you know the essentials of what TDD is.  But actually following the practice can require some more explanation.  Which tests should you write first?  How much of the production code should any one test verify?  How do I start?  I think Robert Martin’s (Uncle Bob’s) rules for TDD from his book Clean Code explain it best:

  • You may not write production code until you have written a failing unit test.
  • You may not write more of a unit test than is sufficient to fail (and not compiling is failing).
  • You may not write more production code than is sufficient to pass the currently failing test.

 If you follow these rules you will be stuck in a small tight loop of only a few minutes long where you write a small, failing unit test to test a little bit of functionality.  Then you write a little bit of production code where you make that test pass.  Then you refactor both the unit test and the production code to make them clean (removing duplication, following good design and naming conventions, etc).  

How Do I Start?

 What test should you write first?  The simplest test.  This is the simplest bit of functionality that you can think of that needs to be implemented.  If the first thing you need to do is instantiate an instance of a new class in the production code then write a test that verifies you can instantiate the class.  And when you’ve made that test pass you write the next simplest test.  And the next and the next.  In that process you incrementally add functionality until the production code you are writing meets all of its requirements.  At that point you will have automated tests that tell you the production code is working and will tell you in the future if anything ever happens that breaks it.  This way you can move forward to the next piece of the production code with confidence that your unit tests will catch any errors you may introduce as you go.

How Test Driven Development Helps Your Business

Test Driven Development (or TDD) is the practice of writing unit tests before you write your production code in a cycle of write a test, write just enough production code to make the test pass, then refactor the code to make it clean.  This website provides many articles on how to follow this practice from it’s basics to advanced aspects and language specific details.  But before you invest a lot of time and effort learning this practice and adjusting your workflow to use it, it makes sense to understand how it will benefit you and your business.  Because there’s really no point in adopting a practice if it doesn’t ultimately help you with reaching your business goals.  The list below describes several ways that TDD does this.

Documents the Code

Your unit tests provide a compilable and executable example of how the production code is expected to work.  This can help new developers immensely when trying to dive in an understand what the code is doing and how it works.  If new developers can understand the code base more quickly and easily then they’ll start helping the business more quickly.

Keeps the Design Simple

One of the goals of TDD is to only write enough production code to make the currently failing test pass.  This helps to drive to the simplest implementation possible.  Simple designs are easier to understand, maintain, and modify in the future all of which help the business to put our high quality software faster.

Improves Code Quality

Bugs in your production code can be a serious problem for your business’s reputation.  Unit testing should the be the first safety net to catch bugs in the production code.  Every developer should be writing unit tests to verify that their code is working as expected for both positive and negative test cases.  If you aren’t writing unit tests at all then you’re missing a huge opportunity to catch bugs in your code early and in ways that may be difficult to reproduce outside of unit tests.  

Implement Features Faster

Getting new products and features out fast is critical for everyone’s business.  Once you have the TDD workflow in place you will work faster.  You will find bugs with your code, be able to back out the changes that have created those bugs, and then get back on the right track more quickly.  The TDD cycle is a short 2-3 minute cycle of writing a test, writing the code to make the test pass, and refactoring.  If any changes break any of the tests you’ll know very quickly and be able to back those changes out.  You can’t break to much in 2-3 minutes.  This is a much faster cycle than compiling an entire component, deploying it to the target environment, testing it, and trying to dig through a days worth of changes to figure out any problems.

Increase the Maintainability of Your Code Base

Business is always has a need for software products to be creating new features and capabilities.  When you have working unit tests for all the code in your code base then you have a lot more confidence to make changes in that code base because your unit tests will immediately tell you if you’ve broken anything.  Implementing unit tests first also helps drive your code to have a modular and object oriented design which helps with implementing changes later.