Archive for August, 2009
1.- Write a test case for every condition.
There’s a common misunderstanding- some developers translates “one test case for every condition” to “one test case for assertion”, that’s just plain wrong, a condition can contain one or more assertions, check the following test case.
As you can see, only one condition is tested and it’s composed of many assertions. When you are picking the condition to test, it is also a good idea to focus on the boundaries of your system, so if you are testing a range, try testing with the last element of your range, the first element…
2.- Test only one thing .
If you don’t write your tests to only test one thing, you may find the following problems.
- Over complicated tests. Tests which are difficult to understand.
- Overlapping tests. Different tests testing the same functionality, this will carry a lot of overhead when you will have to maintain them.
- Low code coverage. Is easier to have higher code coverage if you write simple test cases for all the elements of your code.
- Difficulty to track an error. When your test cases fail, ideally it should be pretty straight forward to tell where they are failing. When you are testing several things in every test case is hard to tell what the source of the errors is.
To test only one thing is sometimes hard because of the dependencies in your system. You may want to write a test case for a class that indirectly uses a database, or a third class, in this case, the best solution is to use a mock, stub or a fake object.
3.- Make the test case describe how your system works.
Test cases should be used as well as documentation on how your system works, they will tell the user how can you use the interfaces of the different classes and what kind of errors can you expect. A few tips to help creating test cases that self document your code are:
- Write your own asserts, for instance, you can see the custom assert I wrote for the chess test case example above.
- Make clear the 4 stages of every test case: Setup, Exercise, Verification and Tear down
- Make your test code as clean as your production code.
4.- Organize your test cases consistently.
There are several ways to organize your test cases, by class, functionality… Whatever the structure you pick, just keep the consistency so it will be easy to locate and add new test cases in the future.
5.- Make “FIRST class” test cases (Fast, Independent, Repeatable, Small and Transparent)
- Fast. Your test cases should be very fast to execute, every time you want to run all of them it shouldn’t take more than a few seconds for an small application.
- Independent. You should be able to run your test cases in any order.
- Repeatable. The result of the test case should be always the same, no matter how many times you have executed it before.
- Small. Small test cases are easy to understand and change, are also likely to be faster.
- Transparent. It should be clear what the purpose of each test case is.
I’ve just read xUnit Test Patterns – Refactoring Test Code by Gerard Meszaros, this is a book from the “Martin Fowler signature series”, meaning that he has reviewed and advised on the contents of the books, as a result, you will find that the structure of this book is pretty similar to Refactoring: Improving the design of existing code.
These are the main points I’ve taken from the book.
- Unifies terminology and provides with some vocabulary to help promoting open discussions about testing. Some of the common terminology provided in the book is:
- SUT: System under test, the component you are testing.
- DOC: Depended-on component, component which depends from the SUT, like a database or a third party library.
- Test Double: A component that substitutes a DOC to help us testing the SUT isolated and that will make the testing easier.
- Fixture: The set of data that needs to be prepared for each test.
- Highlights the fact that every test consists on 4 steps.
- Setup: Step where the fixture for the test is prepared.
- Exercise: The fixture is passed to the SUT and the functionality to be tested is executed.
- Verification: Ensures that in the previous stage, Exercise, everything happened as expected.
- Tear Down: The fixture is removed.
- Introduces test smells: Test smells are indicators that your tests are not clean and that you may have maintenance problems with them in the future, you are encouraged to refactor them. Some of the most important test smells are.
- Buggy tests: Tests where is usual to find bugs.
- Frequent debugging. If you have a good set of test, debugging shouldn’t be necessary, if you find yourself debugging your code, you may want to write more tests or refactor the ones you have.
- Test code duplication. Same as DRY, don’t duplicate code in your tests either…
- Encourages to have clean test code.
- Sets a few goals and principles for the tests, some of them are:
- Expressive tests. Test should be very easy to read and should be written in such a way that they can be used as documentation on how to use the system
- Fast tests. Tests should always be fast to execute.
- Independent tests. The result of a particular test should not depend in the order of its execution.
- Introduces some patterns to deal with common problems when writing tests. Some of them are:
- Test with doubles. How to make sure that the SUT behaves as expected when it has dependencies with other components as databases or complex elements. The four types of doubles are Mock, Stub, Dummy and Fake.
- Test organization patterns. Patterns to help you decide how to structure your tests.
- Fixture setup patterns. Patterns to help you create your fixtures.
What I liked from this book
This is a great book, as far as I know is the deepest book on how to write good tests, it is not too technical, which is really good as it can be useful no matter what your programming language is. It is very well structured and I’m sure will be used in the future as reference to discuss subjects related with testing.
What I didn’t like from this book
Even thought, as I said, I think this a great book, there are a couple of things I would have changed.
- The title. They call the book xUnit Test Patterns, and they actually use JUnit for most of the examples in the book, but the scope of the book is much bigger.
- The length. Is too long, I believe they could have fit the same concepts and ideas in maybe half of the pages they use.
If you want to improve your unit tests, no matter if you use xUnit or TestNG or any other framework, buy it, it’s a great book, what I have explained of this review is actually just an small portion of all the patterns, goals, and test smells that the book explain.
The simpler the better
The simpler the better, that’s something that we usually forget when we are developing software, lots of times we are tempted to use design patterns/frameworks/technologies/tools… without considering other simpler and less fancy options. If there are two ways to implement something and if they are equally functional and don’t add any repetition to your system, pick the simplest solution.
Considerations for not adding complexity to your project
If all you have is a hammer, everything looks like a nail
This is a common bad habit in developers that have just learn something new and they are anxious to apply it. Suddenly everything fits this new technology/tool/framework… Here are some examples:
- ESB: “Let’s communicate the backend layer and the UI layer of this desktop application through the ESB!!!
- Spring: “I have to create the class car, and it has 4 doors, I will better assign that dependency through Spring!!!!”
Because you can, it doesn’t mean you should
This is common in senior developer that are eager to show how advanced their designs can be. Example:
- Abstraction: “The board class for this chess game software is too rigid. I will change it so that it will be a dynamic 2D grid which will allow us to create chess boards of 12*12, 4*6, or any other arbitrary dimension”
The more things that can fail, the more things that will fail.
There’s something that every developer must always remember. We always make errors. The number of errors we make is proportional to the number of things we develop and their complexity. So the safest path is not to write code at all or not to add complexity.
The more prepared is your code for change, the more difficult is to change your code.
This is a paradox which I believe I read in a Kent Beck book, (I think it was in “Xtreme programming explained”, if someone could please confirm this in a comment I will appreciate it very much).
It is a common scenario when a developer proudly shows some code and explains that because of his design, any future changes related with his class can be handled through configuration. Three months later, the first change request not only shows that the current code needs to be changed, but because of over-engineering, extra effort in refactoring is going to be necessary.
Add complexity if… makes things easier!
I hope I am not sending the wrong message with this post, I’m not saying that using frameworks/tools/new technologies… is bad, I’m saying that if not used appropriately the complexity they bring takes over any flexibility they may offer. So the key is to know when to use them. Here you can find three tips that may help you.
Use them to get rid of some duplication you have in your code.
This is one of the most legitimate causes to add a tool/framework/technology to your project, duplication makes your code maintenance a nightmare.
Use them because of the flexibility they provide
Spring, Hibernate and other frameworks are examples of frameworks that are great when you need flexibility.
Use them because is easier to use them than to implement their functionality by ourselves.
If you need a unit testing, don’t write your own, use any of the xUnit or TestNG, for example.
Code reviews are one of the most valuable engineering practices.
- Code reviews improve the quality of the code through the suggestions from the code reviewer.
- Code reviews are a great tool to indirectly teach other developers parts of the system they may have to maintain in the future.
- Code reviews encourage people to learn best practices from other developers.
- Code reviews can be used as a validation of the clarity and simplicity of the system.
You may have noticed that I have omitted from the list that code reviews help finding bugs and enforce coding standards and that’s because:
- Code reviews SHOULD’T be performed to find errors in the code.
- Code reviews SHOULD’T be performed to enforce coding standards.
10 years ago these previous two points would have made sense in your code reviews, but not anymore, this should be covered by your automated tests and by some tool which should enforce your coding standards. All this said, it doesn’t mean that is bad to raise a bug or a coding standard defect in a code review, it means that to found them is not the purpose of the code review.
From this perspective then, these are my five tips to make good code reviews.
1.- Make your code reviews often.
I have suffered in my own experience from late code reviews, I used to work in a company where a huge code review of the application was supposed to be the last stage of the development, and they were simply useless, having late code reviews have the following disadvantages.
- The more code to review, the more likely that the developer won’t accept a suggestion of refactoring.
- The longer since the developer coded, the more likely that the developer is personally attached to the code. I would like to remind here one of the biggest problems of software developers, their ego, if a developer designs something and makes it work, no matter how crappy it is, there are a lot of chances that his ego will interfere with any suggestion to improve the design.
- The closer to release date, the less importance is given to enhancements to the code.
My own approach is to make code reviews every time I have just finished coding something non trivial, and that usually means several times a day.
2.- Make your code reviews informal and short.
Forget about checklist, turn around and ask for 5 minutes of attention to any colleague you have beside, if you have a checklist, either one of this 2 things will happen.
- Only what is in the checklist will be reviewed during the code review.
- Code reviews will become formalities, the people will sit together for as short time as possible and will just pretend that they care about the code.
Having short and informal reviews will help you to engage in open conversations where no one will feel as if they were reviewed, but advised.
3.- Make your code reviews with different reviewers.
Is a good idea, if possible, to not use always the same person as code reviewer, there are three main reasons to look for different reviewers every time you want to review your code.
- Is good to get different perspectives
- More people will be able to maintain your code in the future.
- It is a team building activity.
4.- Keep a positive attitude.
Again, remember one of the worst problems of the developers, their ego, having a positive attitude will help to make the code review much more pleasant, and all participants will be more likely to accept the suggestions, and remember: you are not the code!
5.- Learn to enjoy the code reviews.
This is the most important tip, if you are in a team where people enjoy code reviews, you will enter in a dynamic where everyone will be committed to deliver quality code that not only looks good to them, but to the whole team.
Testing was considered, (and still is considered in some companies), the last stage of software development, developers would hope that they will build from scratch their applications without errors, so the testing phase would be just a formality where only minor errors would be expected, thankfully we are moving from that utopian parading to a new concept where testing is an integrated part in the development process. What follows is a few “best practices” to take the most from this new test paradigm.
1.- Test code is as important as production code.
One of the most common reasons to fail for teams starting TDD is that their test code is not clean, so it will eventually cost more to maintain the tests than the actual production code, to avoid this scenario, it is important to keep in mind that having clean test code is as important as having clean production code.
Reuse fixtures. If you need some data to be available for different tests, make it reusable and make it available in a single place.
Write you own asserts. If you are testing some data in 5 different scenarios, and to prove that everything works fine for each of the scenarios you need 5 different asserts, write your own assert that performs those other 5 ones and call it from your code instead of writing 25 different asserts.
In general, use the same principles you apply to write good production code: In this other post you can find some more directions to write good code: 10 commandments for creating good code.
For further reference, I highly recommend you to read the following book: xUnit Test Patterns: Refactoring Test Code
2.- Test as much as possible, test as soon as possible, test as often as possible.
As I said on my previous post, I am a convinced TDD developer, but maybe you are not, what is important is that no matter if you write the test before or after, you don’t go forward with new code until you have a test that proves that the code you just implemented works as expected. This attitude of testing as much as possible, as soon as possible and as often as possible will prevent you from entering new bugs into the system, and for this reason it will also make you more confident and productive.
3.- Test at different levels.
Don’t try to cover your application with tests that only cover both the UI and the backend, they take too long to write, are hard to maintain and slow to execute, it is also very difficult to test all the different elements of your application if you don’t separate them, the best approach is to test at different levels. Consider having at least the following set of tests:
Unit tests. Tests that check all your components by themselves, without taking into consideration their interaction with other components.
Integration tests. These tests are not identified by the product owner in his/her acceptance tests but are developed either by QA or development to compliment them.
UI Tests. UI testing is slow, also the UI is likely to be changing during development and so do its respective tests, by keeping the UI tests separated from the backend tests we won’t be affecting them every time the UI changes.
4.- Testing as an integrated part of the development process. Continuous integration
Testing has to be an integrated part of the development process, something is not done until is tested, forget about having a different stage where someone else takes care about the testing, everyone is responsible and accountable for testing, from development to QA to the product owner. Ideally you will also apply continuous integration, which goes further of the testing scope, for more information on continuous integration check this excellent article from Martin Fowler.
5.- Use testing tools and frameworks.
Luckily there is a set of tools and frameworks developed lately to help us testing our applications, a selection of these tools are:
Fitnesse. A tool to create acceptance tests, non technical people can write their own test cases.
xUnit. Framework to develop unit tests, integrated with the most popular IDEs and programming languages.
CodeCover. A code coverage plugin for Eclipse, useful to know what parts of the code we are testing, and what parts we are not.
Jira. A reporting and tracking tool, mostly use to track bugs.
Mercury. Popular testing suite, includes different tools for different types of testing, including UI testing.
Selenium. UI testing tool for browser applications.
Hudson. Continuous integration tool, used to ensure that the code is always ready to build and that it passes all the different tests.
I am a convinced TDD developer, TDD makes you more productive and helps you to create robust and simple code, but it has a flaw, TDD doesn’t help you deciding where to start coding, you have to decide what test you want to write first, and them you keep building from there, this may lead you to a dangerous path where you are actually going forward, but you are taking the wrong direction.
Fitnesse can help you to overcome this problem, Fitnesse is a testing tool that helps you to write acceptance tests as tables in a wiki page. Practically it means that you can first write the tests that will wire all the functionality you want to implement.
Creating the fitnesse test.
The following is a screen capture of a simple acceptance test created in fitnesse to test a chess pawn movement. See that Fitnesse allows you to write tests in such way that they should be self explanatory.
Creating the xUnit test for the fitnesse test and using TDD from there.
Now that we have the acceptance test created and implemented, we can write a xUnit test for it and start applying TDD (note that the implementation details for the fitnesse test are outside of the scope of this post, you can check the fitnesse documentation for reference).
At this stage we will have a failing test which we can make pass very easily using the simplest solution possible, then we can extend the fitnesse test, for example we can add a second case where we try to move two white pawns in a row so that the second pawn movement should be illegal (is the black player turn). We always repeat the same steps until all the desired functionality is implemented: write an acceptance test in fitnesse – write a xUnit test for that acceptance test – apply TDD from there.
Advantages of using Fitnesse.
- It gives you an starting point to code from.
- It sets a clear goal of what you need to implement.
- When you finish your implementation it validates that it works end to end.
- It provides a friendly UI so that even non technical people can write their own test cases.