Making Good Software

The 13th of this month is the 1st anniversary of this blog. A year blogging might sound non impressive, but to me is a big achievement. When I started I only could hope for publishing some articles on dzone and to get maybe a couple of hundred subscribers, I never imagined that I would get to publish 50 articles in dzone, around 10 of them in reddit and ycombinator and 1 in meneame.net (the Spanish equivalent to digg).

This is my way to thank the 150.000 unique visitors, the 520 people that have posted a comment, and the 1700 that are subscribed to the blog, thank you for your support and for your time reading my articles.

In this previous year, 60 articles have been published, from which the 10 most read have been:

1. Programmers top 10 sentences
2. 5 top non-technical mistakes made by programmers
3. The four golden rules to be a better software developer
4. 5 top non-technical mistakes made by programmers
5. How to write a good test case: 5 tips to write better test cases
6. 5 Tips for creating good code every day; or how to become a good software developer
7. The best bug tracking system. Don’t raise bugs, write an automated test!
8. My ten development principles
9. The average Software Developer versus the good Software Developer.
10. The next software development revolution is here… Are you ready?

In this next year I would like to aim for 3-4 posts per month,  improve my writing skills (as you probably have noticed I am not a native English speaker), and to increase my reader base. I can only wish I will be writing a similar post in one year time from now. Once again, thanks to all of you for being at the other side of the screen, to finish off the article I have included some screen shots with statics for the site, bare in mind that they are only from the previous six months, when I started to monitor them.

One of the most recurrent and common mistakes in software development is to focus too much on technologies, frameworks, middleware and other technicalities. It is like forgetting that our mission is to produce killer applications, not to make sure we use the most powerful tools available.

The following are in my opinion the main 7 anti-patterns related with them.

Anti-pattern 1: Hiring people based on the amount of technologies they know.

Hiring people is one of the most clear and common situations where we let the technicalities drive our decisions, and it is also one of the most delicate. It is as if we only care about how many technologies the candidate knows, rather than looking for good coding or design skills, which are much more important.

Anti-pattern 2: Picking up technologies without knowing if they are necessary.

We have all probably been there. The project is about to start, and the first thing we do is list the technologies and frameworks we think may be suitable for the project. It is like creating a shopping list with all the products we fancy from the supermarket. In my opinion it is better to start as simple as possible, and add technologies and frameworks only when they are required or when they will make the developers life easier.

Anti-pattern 3: Overusing a technology.

“When you have a hammer, everything looks like a nail”. Imagine that it’s been decided that Spring is going to be used for dependency injection. Next think you know, Spring is used to create all the objects in the application, and the configuration file has thousands of lines of code.

Anti-pattern 4: Cover design flaws with technologies.

Performance comes to mind immediately when discussing this anti pattern. It is very common when performance is poor to blame and focus on the technologies involved, but in my experience, performance problems usually have their root cause on how the implementation was designed.

Anti-pattern 5: ALWAYS Prefer new technologies and frameworks to a plain approach.

There are some developers that if they find something that can be done with a new technology they automatically discard a plain approach, and that is not always good. It is worthwhile to remember that technologies, frameworks, middleware… come at a price to the project. They require maintenance, learning, support, configuration…

Anti-pattern 6: Prioritize extensibility when it is not necessary.

One of the most common reasons I have heard for using new fancy technologies is that they deliver an extreme degree of flexibility and configuration not achievable otherwise with a plain approach. While this may sound wonderful, it is also true that most of the time they were not necessary for the project, so using them will only increase the risk of over engineering the application.

Anti-pattern 7: MDT – Management driven technology.

This anti-pattern usually goes like this: Your manager reads a magazine/article which talks about a new super-duper technology, he gets all excited, the next day in the office he makes the announcement: “I’ve found the solution to all of our problems, from now on we are going to use ________ !!” (fill in the gap, with the fancy technology of your choice)

After several years developing software, I have acquired a very a strong opinion on how software should be developed, I actually have come to the conclusion that everything boils down to 10 principles, that if well implemented, will make any software development successful.

1.- Customer first.

“If we don’t take care of the customer… somebody else will.”

Customer first means focusing from a customer perspective on real value for the product being developed, other aspects as: contracts, requirements, technologies… should take a secondary role in the project.

Not focusing on the customer in one the top 5 five non technical mistakes made by programmers.

2.- Code quality.

Even though code quality is something very subjective, (all code is crap), it drives many important aspects as: how hard is to maintain the application or how hard is to take on board a new developer.

In my opinion, the main indicators for code quality are: simplicity, readability, robustness and testability. Other qualities as performance or extensibility, if not required, will over engineer your application.

3.- Empowerment.

People are the most valued resource in software development, not the technologies. They are the ones that make the difference between a mediocre product and an excellent product, but they need to be empowered

Empowerment is the process where people is encouraged to be proactive and take decisions. Some of the most effective mechanisms to empower are: coaching, pairing or delegating.

There is an excellent article from Tony Morgan explaining the difference between a classic delegation approach and empowerment.

4.- Continuous integration.

Integration is, from my experience, a major issue in software development. Waiting to integrate at the end of the project or after a big chunk of functionality is completed, is always a traumatic process.

Continuous integration is the process that guarantees that every piece of code committed is automatically integrated in the system. Remember that continuous integration goes beyond continuous compilation.

This article from Martin Fowler, is one of the best references online about continuous integration.

5.- Iterations.

Iterations provide continuous feedback. Continuous feedback is important because it reduces the amount of uncertainty in software development.

While iterations are usually only related to agile methodologies, it is important to remember that there are other methodologies as RUP which also use iterations and they are not part of the agile family.

6.- Automated testing.

Allows refactoring and iterations, provides confidence and if well implemented, enhances the correctness of the final product. To automate tests you may consider some facts about testing and some advices on how to write a good test suite.

7.- Refactoring.

No matter how much care you put on coding, on your first attempt, you are going to be wrong. Refactoring is the practice by which we keep changing the code to meet the changes necessary to introduce in the system.

8.- Informal architecture.

Big design upfront (BDUF), unless you are NASA and can expend 50-60% percent of your project time on it, is a complete waste, but so it is coding without any preparation. Informal architecture is a compromise solution where architecture is discussed in an ongoing basics and is persisted in light documents, boards or similar.

9.- Communication.

Software development is ONLY about communication. The customer explains to the software development team what he wants to accomplish so that the software development team can explain it through code to a computer.

10.- Avoid waste.

Waste accounts is one of the major productivity killers in software development: unnecesary meetings, unnecessary requirements, unnecesary process, unnecesary documentation, being some of the most common and dangerous.

From among all the qualities that make good code, robustness is one of my favourites. Robustness can be explained as a measure of how hard is to break someone’s code. The opposite of robust code is fragile code; an example of fragile code would be having a class with all its member variables marked as public.

My concern with robustness is that is usually overlooked, is not placed as a priority for some developers, which in my opinion is a mistake. The main problem with fragile code, is that for its correct behaviour, it depends on how the code is being used, and that, in my experience, is synonym of bugs.

The benefits of having robust code are many, some of the most important ones are:

1. Easy to change. Robust code is easier to change because external factors don’t affect its behaviour .
2. Less bugs. The more robust the code is, the less combinations that cause a bug in the code.
3. Better integration. Robust code is predictable, so from an integrator point view, is always going to have the same behavior.
4. Consistency. Repeat this with me “Robust code, works fine no matter who, when or how it gets called”.

To help spot fragile code here there is a list of common fragile code patterns.

1. Public getters / setters / fields.
2. Bad equals / hashCode / compareTo methods.
3. Code duplication.
4. Use of plain constants when an enum should be used.
5. Not providing with any factory or build pattern for complex objects.
6. Dependency on global variables.
7. Coding against implementations and not interfaces.

One of the things that upsets me from modern frameworks is that they push you to create non robust code, they sometimes force you to have default constructors, getters and setters for all instance variables… but there’s no much we can do about it, and sometimes the advantage of using them is worth the hacking, but, from my point of view, fragile code shouldn’t be accepted in any other scenario.

Test code deserves a special mention. Test code is usually seen as not as important as production code, and this causes lots of developers to write very fragile tests. I am sure that anyone who has work in a large project using automated tests will absolutely agree with me that there is nothing more tedious than changing 10 lines of test code for each line of production code that changes, and that can be prevented with robust test code. So when writing test codes, please keep in mind that you have to care for it just as much as if was production code!!!

Bug tracking systems are bottlenecks, they perpetuate the continuous handovers between testers and developers and they require extensive documentation to be maintained.

The standard life cycle of a bug is usually as follows: A QA tester finds an error in the application, creates a new bug in their bug tracking system and sends it to the developer, which in turn, as soon as he fixes the bug, notifies back to the QA tester, which then, notifies the product owner which then, closes the bug.

In paper, the previous process sounds wonderful, but there are quite a few drawbacks.

• Communication problems. Sometimes the description of the bug is poor, or the developer misunderstands the problem.
• Regression testing. This process requires the handover of code back and forward between QA and development, this causes versioning problems and duplicated testing effort.
• Low robustness. The process doesn’t guarantee that the error will appear again in the future.
• Bureaucracy.Traditional bug tracking systems switches the team focus to bureaucracy from quality.

The solution for this problems is simple. When anyone finds a bug, he/she immediately writes an automated test that fails based on the bug just discovered. This mechanism have the following advantages.

• Gets rid of versioning problems, everyone works on the head.
• Guarantees that the bug is never gonna reappear undetected.
• Eliminates uncertainty on what the bug exactly is.
• Eliminates bureaucracy
• Consolidates the development activities in one place, the code.

The challenges to switch to this process, where a bug tracking system is not necessary are:

• Requires an automated build. This build have to be in place and running all the automated tests, so when a new automated test is entered because of a bug, the build will break and the team will have to take a decision whether they will fix the fix now, or whether they will ignore it for the time being (That way you will know that all the ignored tests are non-fixed bugs)
• Requires cooperation. Developers and testers must work together so they can create new tests as soon as they find new bugs. You are going to require testers with programming skills.
• Requires discipline. The team must understand that without exception, for every bug found, a test is created, and for every time the build is broken, the priority is to fix the build.
• Requires organization. Tests must be created with criteria and using a common naming convention.

Summarizing, in my opinion, the future of bug management is to get rid of all the traditional bug tracking systems and all the bureaucracy, I also think, as explained just above, that there are quite a few challenges, but if you have the skills and tools in your team to overcome them, I would seriously think about getting rid already of your old bug tracking system.

15 years ago, Microsoft released its first version of Internet Explorer. In 8 years, the mobile devices are going to take over television, computer, phone, gps, internet, console… Everything will be integrated in a single device. Internet connectivity will be universal and inexpensive. The 80% of the applications we will run will be web browser based.

Now, I may be very wrong with my previous prediction, but for those of you who think I may be right, how prepared do you think you are for this next challenge? I’m not saying that there is reason to be afraid now, or in a rush to learn new technologies, but if you think that you are going to be developing software in 10 years, it maybe worth asking yourself if you prefer to be leading what is probably going to be a revolution, or if you prefer to just be one of the developers that will have to go through a technological transition. In case you prefer to be leading, here you can find my personal suggestions to start getting ready.

1.- Get agile!

I already wrote an article where I expose my conviction that one of the current big changes in software development is the process, we are moving from waterfall to agile, in the future, in an on demand environemnt, where packaged applications are going to dissapear, being able to start deploying value to your customer ASAP is critical. Companies with a waterfall mindset will find it difficult to keep up with an environment which is going to demand the development team to be changing continuously.

From a developer point of view, this means that we need to start getting used to some engineering practices as TDD, continuous integration, pair programming, refactoring…

From an educational point of view, there are several great books, from which I would recommend the ones from Mike Cohn, Kent Beck and Martin Fowler, for those who prefer some more formal education, it is also available a certification program for the Scrum process, which is one of many different agile processes.

2.- Web browser development.

In the future, for an average user, the browser is going to be the only application he is going to use, how comfortable are you developing applications for it?

I think the following are the main areas where all developers should have a basic understanding.

• CSS
• Javascript, AJAX techniques, Frameworks as JQuery, Dojo…
• HTML
• HTTP

So why don’t create a web application just for fun?

3.- Mobile development.

We are actually not so far from an unique mobile device that integrates all our needs, currently new products as the iPhone, Android phones or windows phones are on their way to achieve it. Being able to develop applications for them can make a huge difference in you CV.

It is also important to consider that getting used to mobile development will also give you some experience on developing for an tactile and small interfaces.

4.- Security, Integration and Cloud computing

This on demand scenario I am talking about, is going to rely on a heavy server side load which the user is going to access through the Internet.

As developers we have the challenge to adapt our applications to this kind of scenario, the main challenges are;

Security. All the personal data from all the users is going to be stored in central locations which are transparent to the final user, being able to secure them is going to be fundamental, having a basic understanding of authentication, certificates, encryption, digital signatures… is going to play a major role.

Integration. The users will access a number of services and will expect them to interact with each other, to accomplish this, as developers we must be able to integrate our applications with third party applications with very small effort.

Cloud computing. Cloud computing means that the users are gonna access the applications as services where the backend infrastructure is going to be transparent for them, being able to have a basic understanding on cloud computing and its design challenges is going to be a key factor.

Legos

I love Legos. I got my first one when I was a kid. I remember the box had this awesome picture of a robot, so I opened it and started to play around with the pieces. It took me quite a while and I had to follow the instructions step by step, but finally I built it.

After that, I had a few more Lego and I realized that gradually I wouldn’t even need the instructions to assemble the pieces. The building mechanics became something natural: just pick one piece and connect it to the next one. It’s simple, powerful and most importantly consistent.

There are some other great building games and they are all based on the same principle: simplicity and consistency. Once that you master the mechanics of assembling the different pieces, the rest is up to your creativity!

But now, imagine that a new company offers a new building game where its pieces are mixed from other building games. Do you think it will be easy to build something with it? … Well, it will obviously be a disaster, it wouldn’t have any “Design integrity”, some pieces will be Lego, other pieces will be Meccano…you would be probably using glue to keep them together!!!

Design integrity, simplicity and consistency, play the same important role in software development.

Design integrity.

Design integrity is about consistency, is about using always an ORM or always SQL to access the database, is also about using a the same dependency injection approach, or the same mechanism to communicate the different components of the application.

The key to create an integrated design is what Fred Brooks called in his master piece “The mythical man month”, the “Surgical Team”. The surgical Team is a small team, sometimes just one person, who supervises all the design and architectural decisions. In an ideal scenario, the surgical team has both formal and informal authority.

To success creating this surgical team, you need exceptional people, one of those 1 in 100 developers who really make a difference in the project, they also need to be excellent communicators, and they need to understand that they mainly have to coach and not impose. It is also very important that the rest of the team trust them and see them as leaders.

The main risk of failing to have an integrated design is technical debt. Technical debt is one of the biggest risks in software development: take too much debt and your project is gonna be in serious trouble.

My advice is: Get a surgical team ASAP or, if you already have one, make sure that they really are trusted by their colleagues and that they really understand the purpose of what they are doing. Having a good surgical team is one of the most valuable assets in a software development project.

I strongly believe that there are only 4 types of programmers, and that some types of programmers are simply not compatible with others. To prove my theory, I first need to introduce what I believe are the two main metrics that define the way we approach coding, planning and sophistication.

• Planning: The planning metric measures how much time we expend planning before implementing anything.
• Sophistication. The sophistication metric measures how complex our implementations are. The more sophisticated they are, the more extensible and customisable, but also the more complex. In the other hand, low sophistication, (called pragmatic from now on), prioritizes simplicity over extensibility and customisability.

Taking into account this two metrics, it is easy to deduce that there are mainly four roles that a developer can take to in order to do his/her job; these roles can be appreciated in the following matrix.

The previous matrix probably is not going to raise anyone’s eyebrows, is pretty obvious, but what stroke as a revelation to me is when I realized that developers tend to be only comfortable in one of the different 4 axis of this matrix, and that’s what makes the 4 different types of programmer: The planner, the action hero, the pragmatic programmer and the sophisticated programmer.

The planner.

The planner is the type of programmer that prioritizes planning, in the roles matrix is comfortable only in the first row. As a planner, he might have some preferences about being sophisticated or pragmatic in the designs, but will be open for discussion, in the other hand; the planner will never give up planning up front.

From my experience, I would say that planners are mostly old school developers, some recently graduated student, and people with waterfall background.

The action hero.

The action hero is the type of programmer that prioritizes getting things done now, in the roles matrix is comfortable only in the second row. As an action hero he might have some preferences about being sophisticated or pragmatic in the designs, but will be open for discussion, in the other hand, the action hero will never give up the getting things done now.

From my experience, I would say that action heroes are mostly extreme agile developers, technical gurus, and junior developers.

The pragmatic programmer.

The pragmatic programmer is the type of programmer that prioritizes simple implementations, in the roles matrix is comfortable only in the first column. As a pragmatic programmer he might have some preferences about planning up front, but will be open for discussion, in the other hand, the pragmatic programmer will never give up keeping the implementations as simple as possible.

From my experience, I would say that pragmatic programmers are mostly agile developers, and lazy developers.

The sophisticated programmer.

The sophisticated programmers is the type of programmer that prioritizes extensible, customisable and standard implementations, in the roles matrix is comfortable only in the second column. As a sophisticated programmer he might have some preferences about planning up front, but will be open for discussion, in the other hand, the sophisticated programmer will never give up keeping the implementations as sophisticated as possible.

From my experience, I would say that sophisticated programmers are mostly technical gurus.

Why can’t we be friends?

Have you not ever wondered, “how’s possible that always that I talk with this other programmer I get irritated?” To understand why, it may help you to ask yourself what type of programmers you guys are, because, I’ve found that programmers that doesn’t share any role in the role matrix, are simply not compatible with each other. Practically this means that the following two combinations will have a hard time collaborating.

1. Pragmatic programmers and Sophisticated programmers
2. Action heroes and Planners.

Programmers of the same type are likely to collaborate better. Programmers that share only a role will have to discuss more frequently but are likely to get to agreements.

So, what’s the best type of programmer?

The best type of programmer would be the one that can switch role base on a case-by-case basis, but frankly, I haven’t met any yet, as I said previously in the article, all the developers I know are only comfortable in one of the axis of the roles matrix, which is what determines their programmers type.

I also believe that during the development of software, there’s a moment where each of the different roles is necessary, that’s why I believe is good to have a team where the programmer types are balanced.

They are evil!!

It always comes the time in almost every project when some difficult task needs to be performed and so the team will ask themselves: “How are we going to implement it?” is usually at this stage, after several meetings, brainstorming and debates, that an approach is decided, and a proof of concept is arranged… And that’s when the team fails in realizing that they are about to waste their time.

Proofs of concept look ideal on paper; they are supposed to be an excellent mechanism to validate the approach that is about to be taken in the project, but the reality is that they are not, and that’s what I would like to talk about in this article.

Why proofs of concept are bad.

1. They don’t tell you if you are about to do the right thing, they only tell you if that specific approach is feasible. Most of the times, is not about if what we are proving is valid or not, is about how much overhead there is in maintain or integrate what we are proving, but proofs of concept are usually very limited telling us what these boundaries are.
2. People who sponsor a proof of concept get emotionally attached and loose their objectivity. After a proof of concept, a decision needs to be taken, but the people who sponsored the proof of concept, because their ego, will have their objectivity affected and will push to have the proof of concept accepted no matter its output.
3. Proofs of concepts tend to be very specific and focused; they leave outside of the picture many critical areas that need to be proved. For instance, if proving a reporting tool: The proof of concept may be focused in generating some particular type of report, but it may leave outside of the picture some other areas, as the integration of the reporting tool with the application being developed or how customizable the generated reports are.
4. Designing a good proof of concept requires a full understanding of the domain. To design a good proof of concept, is important to identify upfront what are the different risk areas, so they can be proved, but for most situations that’s impossible, so is very likely that after adopting some approach the team is going to hit some unexpected limitations on the approach.
5. Proofs of concept distract the team from the main goal and make them switch to an R&D mentality. Teams doing proofs of concept usually switch to a R&D mentality, with a R&D mentality is very easy to focus in gold platting the application.

All this disadvantages may translate in the following four real major issues in your project.

1. Waste of time
2. Technical debt
3. Fail to deliver all the requirements.
4. Integration issues.

How to avoid Proofs of concept

These are my 5 key practices for avoiding proof of concepts

1. Find the next baby step. Usually when a proof of concept is kicked off is because the team fails to split the biggest problem into smaller issues. Tackling the smaller issues one at a time is easier and is likely to produce a more integrated solution.
2. Keep it simple. Is usual to find proofs of concept to validate at the start of the project how the latest technologies, frameworks… would fit in the application, but I wouldn’t even consider them, I like to start simple, and if necessary add them as we go. I think adding them upfront can become a huge risk factor to the team as they introduce new learning, complexity and integration factors into the project.
3. Just do it. When a team is stuck, I find that the best remedy is just find what the next baby step is and implement it.
4. Work several options in parallel. Sometimes is just impossible to determine in what direction the next baby step should be, for these occasions just work in parallel all the different solutions. The overhead of working in parallel is paid back early in the project because it will be very clear what solution is the most adequate.
5. Delay decision for as late as possible. The general rule of thumb for the whole article can be resumed with the lean principle “Delay your decisions for as late as possible”, find baby steps, work on parallel… but don’t commit to an approach that may cause your project to fail.

Micro optimizations are one of many different bad practices that can be found in a project.

Micro optimizations are bad responses to potential or real project issues, they are created when the team fails to identify the root causes and instead they address the consequences, this usually happens because:

1. Urgency. Not enough time and resources are used to analyze the issue.
2. Experience. The patch strategy is always been followed.
3. Fear. Sometimes people know what the root cause is, but they are afraid to mention that.

Micro optimizations are bad because:

1. They switch the focus of the team from the big picture, they provoke the “Can’t see the forest for the trees” effect.
2. They provide partial feedback.

Micro optimizations are usually implemented as check lists, metrics, reports, documentation or some sort of paper work that needs to be approved in order to move forward.

Some common scenarios of micro optimizations that I have myself observed are:

• Because of the high rate of defects, the team decided they had to freeze the development from time to time and wait for QA to pass complete regression tests. This turned out to be a micro optimization and didn’t help at all. The root cause for the high rate of defects was that the source code quality was very poor, which in turn, was caused by a complete lack of experience and knowledge on the team on the technologies used for that particular project.
• Because of low productivity, management decided to hire more people for the project. Again, this turned out to be useless, the root cause for this issue was that development and product owner were not collaborating as they should, and as consequence, development never understood what was really required by the business.
• Because of poor performance, the team discovered that the bottleneck was on the messaging layer, so they decided to expend lots of money in a commercial solution that delivered better performance, this turned out to be a micro optimization and the performance issues came back. The root cause was that the overall architecture was poorly designed.

To avoid micro optimizations, focus always in the big picture, not in the details; use an out-of-the-box mindset. One helpful, very well known and very simple technique to avoid micro optimization is “the 5 whys technique”, this technique is better demonstrated with a sample.

The team only deliver half of what it was expected for the deadline. (The issue)

1. Why? They didn’t have enough time
2. Why? They had too much work
3. Why? They failed to estimate how much work they could take.
4. Why? They thought that the required activities to complete were simpler.
5. Why? The requirements were not detailed enough.

Avoiding micro optimizations doesn’t mean that the aspects they would usually be focused on, as: performance, punctuality, code coverage… are not important; it means that they serve to a higher purpose.

The biggest effect on moving away from micro optimizations is that you are likely to find that your main issues in your project are not technical, so probably you may want to take a complete new approach on them.