Making Good Software

Not dealing with uncertainty efficiently is one of the main causes for software development projects to fail. Traditional approaches assert that uncertainty can be defeated by designing and planning ahead, but that’s wrong. Even in a small development, uncertainty is so high that to discover all of it up front is impossible. That’s why classic approaches, as waterfall, fail in dealing with uncertainty, and that’s why, in my opinion, they also fail in dealing with change.

Change and uncertainty are at the core of any software development, as Heraclitus said: “Change is the only constant”, and software development is not exception. That’s why dealing with uncertainty is so important. Some of the most common consequences of not dealing properly with uncertainty are: false expectations and bad estimations.

False expectations. Uncertainty is going to cause change, and change needs to get fed back to all the stake holders, but is very easy to leave information and people outside the loop, if this happens then the expectations are going to be different across the parties involved in the project causing that some of them will have false expectations.

Bad estimations. As I’ve already said before in this blog, I strongly believe that big planning up front is a waste of time, and that’s mainly because uncertainty. The cone of uncertainty is a very well known diagram which graphically shows this.

Source: http://www.codinghorror.com/blog/archives/000623.html

These are my 3 advices in order to deal with uncertainty.

Small steps

It is better to take many small steps in the right direction than to make a great leap forward only to stumble backward.

Source: http://atriskliving.blogspot.com/2008/09/goodbye-baby-step-1.html

It is impossible to get rid of all the uncertainty, so the best way to deal with it is to take as less uncertainty as possible at a time, your estimation then is going to be more accurate and the expectations across all the stakeholders in the project are going to be aligned. Small steps will also provide with quick feed back, so you can correct the direction in your project as soon as is necessary, and it will help you to reduce the total remainder amount of uncertainty.

As a rule of thumb, I don’t like to have tasks longer than 2-3 days. These tasks should cover a whole end to end scenario in your application and should have a clear acceptance criteria, an example of task for a online book store could be: “Add the option of payment with credit card to the checkout page”.

Iterations

Can’t See the Forest for the Trees

Small steps need to have some sort of higher purpose, if not it would be like trying to climb a mountain by never looking further than 5 meters, just always taking the steepest path, but that is usually not the best path to climb a mountain.

Iterations are short time boxed periods that wraps small steps, their purpose is to serve as control points to demo functionality to the product manager and ensure that the direction of the project is correct.

Communication

Good communication plays a primary role when dealing with uncertainty, it is key, that all the parties involved in software development are aware on how uncertainty develops.

What follows is a summary of my own high level approach for testing software, this strategy is based in facts and principles.

Facts

1. It is impossible to detect all the bugs from an application.

The closer you get to 100% coverage, the harder it is to find the remaining bugs.

2. The more important bugs are in the core layer and in the integration layer (backend).

That’s where the testing needs to be focus on. Core layer bugs and integration layer bugs are the most important because they create a cascade effect causing several parts of the application to fail.

3. Using UI automated tests makes harder the detection of bugs.

Even though they are still very popular, UI automated tests are not very effective finding bugs because they test the core layer and the integration layer indirectly. Testing indirectly the backend makes difficult to exercise it, and makes hard to tell where an error is coming from. It is also important to notice that UI automated tests are also slow and expensive to maintain.

4. Manual testing is still necessary.

There are some important bugs that can only be detected through manual testing, that’s the case of the bugs that can be found doing usability testing and exploratory testing.

5. Testing is worthless if is not executed in a continuous basis.

What’s been proven correct through testing now is going to change very soon so it will have to be proven right again. If this feedback is not fast enough, new changes won’t get proven and eventually new bugs will be entered into the system.

Principles

1. Prioritize what’s going to be tested.

Never have a test strategy that expects to cover 100% of the application.

2. Have as much automated tests as possible.

From your previous prioritization, automate as much as you can.

3. Schedule time for the necessary manual testing to be performed on the project.

Two of the main manual testing activities that are necessary to perform are usability testing and exploratory testing.

4. Use preferably backend automated tests instead of UI automated tests.

I am privileged to work on something which I actually enjoy. It is like being a professional sportsman; I get paid to do stuff that I love.

Actually, based on the majority of programmers I know, I would say that most of us feel the same. We are basically a bunch of geeks trying to prove to each other who is a better programmer, we somehow see the day to day in the office as a grown up version of…

<geekstuff>
     <choices>
          <rts>Civilization</rts>
          <mmorpg>World of warcraft</mmrpg>
          <role game>Lord of the rings</ role game>
          <movie>Star wars</movie>
     </choices>
</ geekstuff >

Going to the office is then like a game, and as in any game, if you are not having fun, what´s the point? So, are you still having fun? Having fun at work is, in my opinion, one of the differences that can make a great software developer.

If you are not having fun, you won´t probably be motivated, and if you are not motivated, you are going to do a poor job, so to try to help you, let me present my four golden rules to keep it fun at work!

See your colleagues, and show yourself as a friend, not as a competitor.

Try to avoid getting too emotional when you have arguments with your colleagues as Dale Carnegie said “The only way to get the best of an argument is to avoid it.”

Change your mind set from “what can I do to show they are wrong” to “what can I do to help my colleagues”.

Look for challenges.

Doing easy and repetitive stuff is simply boring.

Don´t take it too seriously, it is only a job.

At the end is only a job, don’t get too stressed if you don’t want to eventually see yourself with an anxiety attack in the office.

If still is not fun, just find another job.

To me not having fun is critical, we expend a huge amount of time at work, so don’t waste it, if you are not having fun just find another job, even if the money is not as good!!!

Continuous integration is in my opinion the most important practice to successfully deliver good software. Continuous integration should go beyond a simple set of practices and tools to build your source code every now and then; doing so is continuous compilation, not continuous integration.

Continuous integration should drive the way the software gets build and how the different actors from the project interact with each other.

The five main characteristics of continuous integration

In my opinion these following five characteristics are the main characteristics of an effective continuous integration.

1.- QA and development work together.

Having QA and development working together mainly means two things:

1. There are no handovers between development and testing. Coding and testing are not two separate processes, they are performed in parallel, and one can’t be completed without the other one and vice versa.
2. Developers and Testers have a common goal, they don’t perform different activities, developers are not just focused on developing and testers in testing. They share a common goal, produce quality software in coordination with the product owner expectations.

2.- QA act as a business representative for the developers.

The integration with the business requirements is critical to successfully develop quality software, failing to do so can have catastrophic consequences. QA should take special care special of the integration with the business requirements, they should make sure at every time that all the tests and developments are aligned with the business requirements.

3.- The build is the most important asset in a software development.

The build is the image of all the previous activities; it contains the software which is already been developed.

1. The build should never be broken
2. If the build breaks, fixing the build is everyone’s top priority

4.- Continuous feedback and inspection.

Knowing at every time what is the status of the project is also critical, there are a few things to be considered:

1. Automatically check the build every time someone checks code in.
2. Run full regression tests every few hours.
3. Every time a build runs, automatically generate reports including, code coverage, code quality metrics, new lines of code…

5.- Communication is king

Everything is about communication, developers talking to testers and business, and testers doing the same. An important rule of thumb for communication is called “power of three”, in every single important discussion there should be at least, a developer, a tester and a representative from the business, if all the rest works, but communication doesn’t, the project is doomed to fail.

Top recommended actions to improve your continuous integration.

1. For every story, first, do a high level design of what needs to be tested. Prepare this design with QA, development and the product owner. Drive your development with that test design.

2. Only consider two states for your stories: “in process” or “completed”. Don’t use intermediary states as “in QA” or “60% completed”. If you do so you will never have QA and development working together, but against each other.

3. Have testers pairing with developers every now and them. This will enhance the communication between them, and will help to understand what needs to be tested.

4. Make QA people work very close with the product owner. Make sure they keep him well informed of what is going on in the sprint and show him if possible any progress in the stories.

5. Make developers responsible for breaking the build. Developers should perform two steps before committing any code:

1. Update all source code.
2. Build locally the whole application. (If the complete build takes too much time, split it and make the developers run all the unit tests before committing any code).

6. Make your CI continuous integration server to run the build every time someone checks in any code into your CSM. This will make sure that no one checks in broken code.

7. Run full regression tests as many times as possible during the day. All your tests should be automated, hook them if with the build which gets trigger when anyone commits code, if they take too much time, schedule them to run as many times during the day as possible.

8. Add a few visual devices in the office to tell when the build is broken. There are many visual devices that can be configured with almost any continuous integration server, place them in the office strategically, so whenever they turn on, everyone will realize that the build is broken, some examples of this devices are: traffic lights, a red light, a lava lamp…

9. Include an inspection report for every build. In modern continuous integration servers is easy to add different plug-ins so for each build you can obtain additional information, as test code coverage, compliance with coding standards, lines of code duplicated…

Lots of people confuse “test first methodologies” with TDD, it is very common to listen comments like “TDD is just about writing your tests first”, which are completely wrong, these kind of affirmations are not describing TDD at all, they are talking about test first development.

The main reason for confusing TDD and test first development is its own name: “test driven development”. If someone that doesn’t know about TDD would had to guess based on the name what TDD is, would probably guess that is just a test first methodology. But is not!

TDD as invented by Kent Beck, (who also invented Xtreme programming and Junit), goes beyond that. In the core of TDD there is a process to follow, which makes it already different from a simple test first approach.

Image source: Wikipedia

This is also known as red (make your test fail), green (make it pass) and refactor . Where this may be seen as a small difference from a test first approach, if combined with some other agile engineering practices and development philosophies, makes TDD quite different from any other test first approach, it actually switches the focus from testing to design.

TDD is a design practice, is more related with emergent design than with testing. TDD looses much of its potential if it is not combined with other agile engineering practices, or agile philosophies as YAGNI and KISS, in TDD having a large set of tests is a nice side effect, not just its purpose.

To know if you are doing TDD the right way, look at your code, TDD code should look simple and lean, TDD also usually generates more testing code than production code, and you should feel that it helps you to design your code.

So, next time you say you are doing TDD make sure you don’t mean that you are doing test first development.

Creating services is quite an abstract subject, everyone has his own ideas and preconceptions, so is worth clarifying what I mean by services in this article. A service is a component that holds some business logic which can be easily reused anywhere else in the application no matter where the service is located, or what the communication mechanism is used between the client and the service. The following code samples detail my own personal approach on how to implement services in Java.

First thing is to design the service itself through its interface

public interface ServiceA {
	public void doThis();
	public void doThat();
}

Next step is writing the implementation of the service. Is also important to remind that the implementation should just read business logic, it shouldn’t be aware of if is going to be called through a web service, jms…

public class ServiceAImp implements ServiceA{

	protected ServiceAImp (){};

	@Override
	public void doThat() {
		System.out.println("do that");
	}

	@Override
	public void doThis() {
		System.out.println("do this");
	}

}

Now it comes the tricky part, I like to make the services look as if they were local to whoever is using them, it makes them easier to use and because the communication code is hidden and doesn’t need to be written every time a service is invoked they are also less prone to bugs. For this purpose I like to use the proxy pattern.

public class ServiceAProxy implements ServiceA {
	ServiceA implementation;

	protected ServiceAProxy(ServiceA imp) {
		this.implementation = imp;
	}

	public void doThat() {
		implementation.doThat();
	}

	public void doThis() {
		implementation.doThis();
	}
}

Next thing is to deal with the communication issue, now we have a proxy that makes the service look as local to the client, but there isn’t an implementation of a communication mechanism between the client and the service, to do so, we create as many different “stubs” as communication mechanisms we want to implement. For this example, we are going to support a remote WebService call, and a remote JMS call.

The WebService stub.

public class ServiceAWebServiceClient implements ServiceA{
	ServiceA webService;
	public ServiceAWebServiceClient() {
		//webService = code to get the reference to Service A from the webservice call
	}

	@Override
	public void doThat() {
		webService.doThat();
	}

	@Override
	public void doThis() {
		webService.doThis();
	}
}

The WebService implementation could be something like:

@WebService
public class ServiceAWebService implements ServiceA{
	ServiceA imp = new ServiceAImp();

	@Override
	public void doThat() {
		imp.doThat();
	}

	@Override
	public void doThis() {
		imp.doThis();
	}
}

The JMS stub

public class ServiceAJmsClient implements ServiceA{
	ServiceA jmsObject;

	public ServiceAJmsClient() {
		//jmsObject = code to get the reference to Service A from the jms call
	}

	@Override
	public void doThat() {
		jmsObject.doThat();
	}

	@Override
	public void doThis() {
		jmsObject.doThis();
	}
}

Now the service is almost ready, everything is kept separated so changes to the business logic are only done in the ServiceAImp, and changes to the actual transportation mechanism are implemented in their own stub, also we provide with a proxy so that whoever is using the service will use it transparently, the only problem left we have, which is a big one, is that this is quite a collection of classes we are asking our clients to understand in order to use our service, to fix this, I like to use a factory.

public class ServiceAFactory {
	private ServiceAFactory() {}

	public static ServiceA getLocalService(){
		return new ServiceAImp ();
	}

	public static ServiceA getRemoteServiceUsingJms(){
		return new ServiceAProxy (new ServiceAJmsClient());
	}

	public static ServiceA getRemoteServiceUsingWebService(){
		return new ServiceAProxy (new ServiceAWebServiceClient());
	}
}

The previous factory is the only thing our customers need to know about us! To put all the pieces together, let’s look at how a client code using our service will look like.

public class MainClass {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		//Calling doThis with a local instance of the ServiceA, very useful for testing!!!!
		ServiceA serviceA1 = ServiceAFactory.getLocalService();
		serviceA1.doThis();
		//Calling doThis with a remote instance of the ServiceA using Web services.
		ServiceA serviceA2 = ServiceAFactory.getRemoteServiceUsingWebService();
		serviceA2.doThis();
		//Calling doThis with a remote instance of the ServiceA using Web services.
		ServiceA serviceA3 = ServiceAFactory.getRemoteServiceUsingJms();
		serviceA3.doThis();
	}

}

[EDIT] Now you can make the client even less aware of the service location, you can use a dependency injection framework as Spring to set the implementation of your service.

`
public class MainClassUsingSpring {
	public static void main(String[] args) {
		ServiceA serviceA = loadFromSpring();
		serviceA.doThis();
	}

	private static ServiceA loadFromSpring() {
		ApplicationContext context = new ClassPathXmlApplicationContext ("beans.xml");
		return (ServiceA) context.getBean("serviceAProxy");
	}

}

Then you can have a spring file configuration similar to this.

By changing your spring configuration file now you are able to change the implementation of your service and its location without even changing your source code!

The research to become a better software developer is core to any good software developer. Being a better software developer reflects directly in the code you produce, the better software developer you are, the less bugs, easier to read and easier to maintain code you will produce.

What follow are the four rules I use on my day to day work to help me become a better software developer.

Rule number 1:  My code is crap.

All code is crap, and that includes yours, (and mine, of course). No matter how brilliant a piece of code you think it is, there will be always better ways to implement it, and several programmers who will hate it.

Even if you are sure your code happens to be good, force yourself into “my code is crap” mode, and ask around about improvements, be open minded and you will see soon how you will find issues in your code that you never have thought about.

Rule number 2: Even if it’s crap, I care about my code.

It is not about creating the best code upfront, is about making it better every time you visit it. When you finish some functionality, you are not done, I haven’t met any programmer that produces acceptable code on their first attempt, you need to polish it. Refactoring is then one of the main activities you should be performing.

Rule number 3: My opinion about my own code is wrong.

We are so egocentric that we end up believing that what we have produced, no matter how bad it is, is the best solution as possible. Never trust your own opinion, show your code to as much developers as possible and listen to them.

Rule number 4: My manager doesn’t care about my code, and he pays me.

This rule regulates the other three, it is easy to forget that you actually get paid to get things done, your manager should also care about the code, but he is more focused in completing things on time with an acceptable quality, so be careful not to expend too much time just making sure your code is not that crap. The key is being able to balance getting things done and their quality.

Bonus: A few tips to help you with these rules.

• Code reviews.
• Keep it simple.
• Test.
• Refactor.
• Non technical factors.

I used to believe that the most important quality of code is its extensibility. The code had to be prepared for future changes so I would code lots of things not necessary now. But after realizing that I never was right upfront guessing what the future changes will be, and after having quite a few maintenance and handover nightmares, a few a years ago I decided that I will take the opposite approach to what good code is. Ever since I did that, I am every day more convinced that the key characteristic of good code is simplicity. Simple code is easy to read and easy to change. And because it is simple, it is less prone to have bugs (the more complex the algorithm you are creating the more changes to make a mistake).

Whenever you are coding, I recommend you to check every time if your code is still simple, and remember: “there are no hard problems, only hard solutions”

The seven main characteristics of simple code are:

1.- Easy to read.

Simple code doesn’t need additional documentation, or it only needs minimal documentation to be understood

2.- Easy to use.

Whoever is using your code will find it intuitive to use your objects.

3. Easy to change.

Simplicity means that there is no duplicated logic. A change should match a single place in your code.

4. Doesn’t use any non-necessary third party, tool or technology.

Through my experience I have observed how some developers just love to use technologies, tools and frameworks just for the sake of making the project more “cool”. Every time you use them, you are adding extra complexity, and extra complexity means that is more difficult to understand, to maintain and is more prone to failures.

5. It looks simple.

If it doesn’t look simple, it is not simple! You know your code is simple if when you finish you are astonished at how simple the final solution is and you ask yourself how it was possible that it took you so long.

6. Lean.

It only does what is necessary, and nothing else. I think most seasoned developers would agree that trying to anticipate the problems your code will have to face in the future is impossible.

7. Is straight forward.

It doesn’t have unnecessary indirections; the most important operations only require a straight method call.

How can we develop simple code?

The key of producing simple code is continuous refactoring, and the only way to do this is continuous testing.

You are going to need refactoring because every time you add a new line to your algorithm, you are probably making it more complex. So every now and then you should refactor to get it back to a simple stage.

You need many tests because you are doing so much refactoring, and if you don’t have a safety net you are going to introduce bugs into your system.

Are we in front of the next era for software development? Are you ready? or are you going to become a dinosaur developer?

I remember that around 10 years ago object oriented started becoming the main stream in software development, even though C++ was already been used, it was not up to the consolidation of Java that companies started moving to OOP massively, that at least, was the experience I had in Spain where I was working back them (I suppose this would happen a few years before in US, but I would appreciate very much the feedback from any reader to confirm it), back in those days, companies expended huge amounts of money training people that was previously working with some other not object oriented languages as C, Pascal, Cobol… there was a lot of excitement, frustration, success stories and also complete catastrophes… it was a revolution as some of you will probably remember, well, in my opinion, something similar is happening again.

199x – The source code revolution (The OOP revolution)

As I said above, UML, Java, OOP paradigm… were the buzz words back in the 90s, people believed again that they had found the silver bullet for development, if you didn’t know how to do a UML class diagram or Java, you were not on the cool side of programming.

Lots of programmers had to learn a new software development paradigm, which was not easy, the most veteran developers were the most troubled, some couldn’t adapt and end up doing “sequential” programming embedded in objects, (which of course defeats the whole purpose of OOP), some of them eventually change their mindset and eventually started thinking in objects, and for some others, quite a big group actually, that mindset change was impossible, and they end up maintaining legacy code, (in the worst case scenario you can still find some dinosaur programmers still doing this “sequential” programming nowadays).

200x – The process revolution (Agile/lean…)

Now we are in the middle of the next revolution, this time not focused on the source code, but in the processes, the buzz words are: agile, lean, TDD… As with the previous revolution there is probably going to be a big readjustment, and the big question is: “Are we prepared for it?” As in the 90’s, there are going to be a lot of professionals which won’t be able to keep up with all the changes, so the big question is: Are we going to be one of those future dinosaurs developers? My own recommendations are:

• Try to be as informed as possible. Read, ask, do some training, but above all make sure you understand, it is almost as dangerous to learn about something without understanding it than no knowing it at all (think on all the programmers that still think that TDD is all about testing).
• Be proactive. Show initiative; show that you actually do understand the new processes.
• Don’t be religious. As with the OOP source code revolution, there are some people that believe that these changes are the silver bullet for software development, be aware that there is no such thing, so be wise and don’t accept something just because is new, first put that in your project context and analyze if it will work for you.

Bonus: 201? – The source code revolution II? (Functional programming?)

Well, if what I called the “Process revolution” is a revolution which in my opinion is happening right now, the “Source code revolution II” is something which I’m guessing will happen just after the “Process revolution” just as soon as people start to understand that is not the silver bullet, so my guess is that the focus will go back to the source code and we will probably see the commercial massive adoption of new programming paradigms as Functional Programming… What do you think? Will this become the next revolution? Is the Process revolution the current revolution?

A few weeks ago I published “10 commandments for creating good code”, the article was focused on the physic code characteristics but it didn’t mention anything about best practices to follow to produce such code, to compliment it in this article I would like to present what in my opinion are the five more important practices to take into consideration to create good code.

Keep it simple

I strongly believe that the number one problem with unreadable code is over engineering, we learn so many techniques, frameworks, tools, patterns… that we actually believe that if we can use them, we should use them, and it actually should be the opposite, the best solution is always the simplest solution, remember: “There are no hard problems, only hard solutions”.

As I wrote in a previous post, there are four main traps that may keep you away from keeping things easy:

1. If all you have is a hammer, everything looks like a nail.
2. Because you can, It doesn’t mean you should.
3. The more things that can fail, the more things that will fail.
4. The more prepared is your code for change, the more difficult is to change your code.

Visit the article: For god’s sake! Keep it simple! to see a detailed explanation on these 4 traps

Make it work, then make it fast.

“The best is the enemy of the good.” Voltaire

“More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason – including blind stupidity.” W.A. Wulf

“We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil.” Donald Knuth

I took all the previous quotes from two excellent articles from Cunningham and Cunningham, “Make it work, make it right, make it fast“, “Rules of optimization”

Automated Test

They are great to:

• Perform regression testing
• Help you design new code
• Validate your new code
• Boost your productivity

Test early, test often, test automatically! If you are interested in further reading, you might want to check out: How to write a good test case and Top 5 considerations to build software without defects.

Code reviews.

My five tips on code reviews are:

1. Make your code reviews often. At least once a day.
2. Make your code review short and Informal. Something like: “Hi Mike! You mind coming over to my for a quick code review”
3. Make your code review with different reviewers.
4. Keep a positive attitude.
5. Learn to enjoy the code reviews. A code review should be seen as a learning opportunity

For more details check the article 5 tips to make good code reviews.

A special mention deserves pair programming, pair programming is the ultimate way of code reviewing, not only makes code reviews unnecessary, but it also increases the code quality and the productivity.

Refactor.

One of the biggest recent mindset changes is from refactor = bad to refactor = good, saying that refactor is good is saying that you know that you are going to make mistakes, but that you are going to be ready to fix them, that’s quite different from the refatoring is bad mindset, where you put a lot of effort in not making mistakes, which is obviously wasted because, let’s face it, that’s impossible.