Making Good Software

From among all the qualities that make good code, robustness is one of my favourites. Robustness can be explained as a measure of how hard is to break someone’s code. The opposite of robust code is fragile code; an example of fragile code would be having a class with all its member variables marked as public.

My concern with robustness is that is usually overlooked, is not placed as a priority for some developers, which in my opinion is a mistake. The main problem with fragile code, is that for its correct behaviour, it depends on how the code is being used, and that, in my experience, is synonym of bugs.

The benefits of having robust code are many, some of the most important ones are:

1. Easy to change. Robust code is easier to change because external factors don’t affect its behaviour .
2. Less bugs. The more robust the code is, the less combinations that cause a bug in the code.
3. Better integration. Robust code is predictable, so from an integrator point view, is always going to have the same behavior.
4. Consistency. Repeat this with me “Robust code, works fine no matter who, when or how it gets called”.

To help spot fragile code here there is a list of common fragile code patterns.

1. Public getters / setters / fields.
2. Bad equals / hashCode / compareTo methods.
3. Code duplication.
4. Use of plain constants when an enum should be used.
5. Not providing with any factory or build pattern for complex objects.
6. Dependency on global variables.
7. Coding against implementations and not interfaces.

One of the things that upsets me from modern frameworks is that they push you to create non robust code, they sometimes force you to have default constructors, getters and setters for all instance variables… but there’s no much we can do about it, and sometimes the advantage of using them is worth the hacking, but, from my point of view, fragile code shouldn’t be accepted in any other scenario.

Test code deserves a special mention. Test code is usually seen as not as important as production code, and this causes lots of developers to write very fragile tests. I am sure that anyone who has work in a large project using automated tests will absolutely agree with me that there is nothing more tedious than changing 10 lines of test code for each line of production code that changes, and that can be prevented with robust test code. So when writing test codes, please keep in mind that you have to care for it just as much as if was production code!!!

Bug tracking systems are bottlenecks, they perpetuate the continuous handovers between testers and developers and they require extensive documentation to be maintained.

The standard life cycle of a bug is usually as follows: A QA tester finds an error in the application, creates a new bug in their bug tracking system and sends it to the developer, which in turn, as soon as he fixes the bug, notifies back to the QA tester, which then, notifies the product owner which then, closes the bug.

In paper, the previous process sounds wonderful, but there are quite a few drawbacks.

• Communication problems. Sometimes the description of the bug is poor, or the developer misunderstands the problem.
• Regression testing. This process requires the handover of code back and forward between QA and development, this causes versioning problems and duplicated testing effort.
• Low robustness. The process doesn’t guarantee that the error will appear again in the future.
• Bureaucracy.Traditional bug tracking systems switches the team focus to bureaucracy from quality.

The solution for this problems is simple. When anyone finds a bug, he/she immediately writes an automated test that fails based on the bug just discovered. This mechanism have the following advantages.

• Gets rid of versioning problems, everyone works on the head.
• Guarantees that the bug is never gonna reappear undetected.
• Eliminates uncertainty on what the bug exactly is.
• Eliminates bureaucracy
• Consolidates the development activities in one place, the code.

The challenges to switch to this process, where a bug tracking system is not necessary are:

• Requires an automated build. This build have to be in place and running all the automated tests, so when a new automated test is entered because of a bug, the build will break and the team will have to take a decision whether they will fix the fix now, or whether they will ignore it for the time being (That way you will know that all the ignored tests are non-fixed bugs)
• Requires cooperation. Developers and testers must work together so they can create new tests as soon as they find new bugs. You are going to require testers with programming skills.
• Requires discipline. The team must understand that without exception, for every bug found, a test is created, and for every time the build is broken, the priority is to fix the build.
• Requires organization. Tests must be created with criteria and using a common naming convention.

Summarizing, in my opinion, the future of bug management is to get rid of all the traditional bug tracking systems and all the bureaucracy, I also think, as explained just above, that there are quite a few challenges, but if you have the skills and tools in your team to overcome them, I would seriously think about getting rid already of your old bug tracking system.

15 years ago, Microsoft released its first version of Internet Explorer. In 8 years, the mobile devices are going to take over television, computer, phone, gps, internet, console… Everything will be integrated in a single device. Internet connectivity will be universal and inexpensive. The 80% of the applications we will run will be web browser based.

Now, I may be very wrong with my previous prediction, but for those of you who think I may be right, how prepared do you think you are for this next challenge? I’m not saying that there is reason to be afraid now, or in a rush to learn new technologies, but if you think that you are going to be developing software in 10 years, it maybe worth asking yourself if you prefer to be leading what is probably going to be a revolution, or if you prefer to just be one of the developers that will have to go through a technological transition. In case you prefer to be leading, here you can find my personal suggestions to start getting ready.

1.- Get agile!

I already wrote an article where I expose my conviction that one of the current big changes in software development is the process, we are moving from waterfall to agile, in the future, in an on demand environemnt, where packaged applications are going to dissapear, being able to start deploying value to your customer ASAP is critical. Companies with a waterfall mindset will find it difficult to keep up with an environment which is going to demand the development team to be changing continuously.

From a developer point of view, this means that we need to start getting used to some engineering practices as TDD, continuous integration, pair programming, refactoring…

From an educational point of view, there are several great books, from which I would recommend the ones from Mike Cohn, Kent Beck and Martin Fowler, for those who prefer some more formal education, it is also available a certification program for the Scrum process, which is one of many different agile processes.

2.- Web browser development.

In the future, for an average user, the browser is going to be the only application he is going to use, how comfortable are you developing applications for it?

I think the following are the main areas where all developers should have a basic understanding.

• CSS
• Javascript, AJAX techniques, Frameworks as JQuery, Dojo…
• HTML
• HTTP

So why don’t create a web application just for fun?

3.- Mobile development.

We are actually not so far from an unique mobile device that integrates all our needs, currently new products as the iPhone, Android phones or windows phones are on their way to achieve it. Being able to develop applications for them can make a huge difference in you CV.

It is also important to consider that getting used to mobile development will also give you some experience on developing for an tactile and small interfaces.

4.- Security, Integration and Cloud computing

This on demand scenario I am talking about, is going to rely on a heavy server side load which the user is going to access through the Internet.

As developers we have the challenge to adapt our applications to this kind of scenario, the main challenges are;

Security. All the personal data from all the users is going to be stored in central locations which are transparent to the final user, being able to secure them is going to be fundamental, having a basic understanding of authentication, certificates, encryption, digital signatures… is going to play a major role.

Integration. The users will access a number of services and will expect them to interact with each other, to accomplish this, as developers we must be able to integrate our applications with third party applications with very small effort.

Cloud computing. Cloud computing means that the users are gonna access the applications as services where the backend infrastructure is going to be transparent for them, being able to have a basic understanding on cloud computing and its design challenges is going to be a key factor.

Legos

I love Legos. I got my first one when I was a kid. I remember the box had this awesome picture of a robot, so I opened it and started to play around with the pieces. It took me quite a while and I had to follow the instructions step by step, but finally I built it.

After that, I had a few more Lego and I realized that gradually I wouldn’t even need the instructions to assemble the pieces. The building mechanics became something natural: just pick one piece and connect it to the next one. It’s simple, powerful and most importantly consistent.

There are some other great building games and they are all based on the same principle: simplicity and consistency. Once that you master the mechanics of assembling the different pieces, the rest is up to your creativity!

But now, imagine that a new company offers a new building game where its pieces are mixed from other building games. Do you think it will be easy to build something with it? … Well, it will obviously be a disaster, it wouldn’t have any “Design integrity”, some pieces will be Lego, other pieces will be Meccano…you would be probably using glue to keep them together!!!

Design integrity, simplicity and consistency, play the same important role in software development.

Design integrity.

Design integrity is about consistency, is about using always an ORM or always SQL to access the database, is also about using a the same dependency injection approach, or the same mechanism to communicate the different components of the application.

The key to create an integrated design is what Fred Brooks called in his master piece “The mythical man month”, the “Surgical Team”. The surgical Team is a small team, sometimes just one person, who supervises all the design and architectural decisions. In an ideal scenario, the surgical team has both formal and informal authority.

To success creating this surgical team, you need exceptional people, one of those 1 in 100 developers who really make a difference in the project, they also need to be excellent communicators, and they need to understand that they mainly have to coach and not impose. It is also very important that the rest of the team trust them and see them as leaders.

The main risk of failing to have an integrated design is technical debt. Technical debt is one of the biggest risks in software development: take too much debt and your project is gonna be in serious trouble.

My advice is: Get a surgical team ASAP or, if you already have one, make sure that they really are trusted by their colleagues and that they really understand the purpose of what they are doing. Having a good surgical team is one of the most valuable assets in a software development project.

I strongly believe that there are only 4 types of programmers, and that some types of programmers are simply not compatible with others. To prove my theory, I first need to introduce what I believe are the two main metrics that define the way we approach coding, planning and sophistication.

• Planning: The planning metric measures how much time we expend planning before implementing anything.
• Sophistication. The sophistication metric measures how complex our implementations are. The more sophisticated they are, the more extensible and customisable, but also the more complex. In the other hand, low sophistication, (called pragmatic from now on), prioritizes simplicity over extensibility and customisability.

Taking into account this two metrics, it is easy to deduce that there are mainly four roles that a developer can take to in order to do his/her job; these roles can be appreciated in the following matrix.

The previous matrix probably is not going to raise anyone’s eyebrows, is pretty obvious, but what stroke as a revelation to me is when I realized that developers tend to be only comfortable in one of the different 4 axis of this matrix, and that’s what makes the 4 different types of programmer: The planner, the action hero, the pragmatic programmer and the sophisticated programmer.

The planner.

The planner is the type of programmer that prioritizes planning, in the roles matrix is comfortable only in the first row. As a planner, he might have some preferences about being sophisticated or pragmatic in the designs, but will be open for discussion, in the other hand; the planner will never give up planning up front.

From my experience, I would say that planners are mostly old school developers, some recently graduated student, and people with waterfall background.

The action hero.

The action hero is the type of programmer that prioritizes getting things done now, in the roles matrix is comfortable only in the second row. As an action hero he might have some preferences about being sophisticated or pragmatic in the designs, but will be open for discussion, in the other hand, the action hero will never give up the getting things done now.

From my experience, I would say that action heroes are mostly extreme agile developers, technical gurus, and junior developers.

The pragmatic programmer.

The pragmatic programmer is the type of programmer that prioritizes simple implementations, in the roles matrix is comfortable only in the first column. As a pragmatic programmer he might have some preferences about planning up front, but will be open for discussion, in the other hand, the pragmatic programmer will never give up keeping the implementations as simple as possible.

From my experience, I would say that pragmatic programmers are mostly agile developers, and lazy developers.

The sophisticated programmer.

The sophisticated programmers is the type of programmer that prioritizes extensible, customisable and standard implementations, in the roles matrix is comfortable only in the second column. As a sophisticated programmer he might have some preferences about planning up front, but will be open for discussion, in the other hand, the sophisticated programmer will never give up keeping the implementations as sophisticated as possible.

From my experience, I would say that sophisticated programmers are mostly technical gurus.

Why can’t we be friends?

Have you not ever wondered, “how’s possible that always that I talk with this other programmer I get irritated?” To understand why, it may help you to ask yourself what type of programmers you guys are, because, I’ve found that programmers that doesn’t share any role in the role matrix, are simply not compatible with each other. Practically this means that the following two combinations will have a hard time collaborating.

1. Pragmatic programmers and Sophisticated programmers
2. Action heroes and Planners.

Programmers of the same type are likely to collaborate better. Programmers that share only a role will have to discuss more frequently but are likely to get to agreements.

So, what’s the best type of programmer?

The best type of programmer would be the one that can switch role base on a case-by-case basis, but frankly, I haven’t met any yet, as I said previously in the article, all the developers I know are only comfortable in one of the axis of the roles matrix, which is what determines their programmers type.

I also believe that during the development of software, there’s a moment where each of the different roles is necessary, that’s why I believe is good to have a team where the programmer types are balanced.

They are evil!!

It always comes the time in almost every project when some difficult task needs to be performed and so the team will ask themselves: “How are we going to implement it?” is usually at this stage, after several meetings, brainstorming and debates, that an approach is decided, and a proof of concept is arranged… And that’s when the team fails in realizing that they are about to waste their time.

Proofs of concept look ideal on paper; they are supposed to be an excellent mechanism to validate the approach that is about to be taken in the project, but the reality is that they are not, and that’s what I would like to talk about in this article.

Why proofs of concept are bad.

1. They don’t tell you if you are about to do the right thing, they only tell you if that specific approach is feasible. Most of the times, is not about if what we are proving is valid or not, is about how much overhead there is in maintain or integrate what we are proving, but proofs of concept are usually very limited telling us what these boundaries are.
2. People who sponsor a proof of concept get emotionally attached and loose their objectivity. After a proof of concept, a decision needs to be taken, but the people who sponsored the proof of concept, because their ego, will have their objectivity affected and will push to have the proof of concept accepted no matter its output.
3. Proofs of concepts tend to be very specific and focused; they leave outside of the picture many critical areas that need to be proved. For instance, if proving a reporting tool: The proof of concept may be focused in generating some particular type of report, but it may leave outside of the picture some other areas, as the integration of the reporting tool with the application being developed or how customizable the generated reports are.
4. Designing a good proof of concept requires a full understanding of the domain. To design a good proof of concept, is important to identify upfront what are the different risk areas, so they can be proved, but for most situations that’s impossible, so is very likely that after adopting some approach the team is going to hit some unexpected limitations on the approach.
5. Proofs of concept distract the team from the main goal and make them switch to an R&D mentality. Teams doing proofs of concept usually switch to a R&D mentality, with a R&D mentality is very easy to focus in gold platting the application.

All this disadvantages may translate in the following four real major issues in your project.

1. Waste of time
2. Technical debt
3. Fail to deliver all the requirements.
4. Integration issues.

How to avoid Proofs of concept

These are my 5 key practices for avoiding proof of concepts

1. Find the next baby step. Usually when a proof of concept is kicked off is because the team fails to split the biggest problem into smaller issues. Tackling the smaller issues one at a time is easier and is likely to produce a more integrated solution.
2. Keep it simple. Is usual to find proofs of concept to validate at the start of the project how the latest technologies, frameworks… would fit in the application, but I wouldn’t even consider them, I like to start simple, and if necessary add them as we go. I think adding them upfront can become a huge risk factor to the team as they introduce new learning, complexity and integration factors into the project.
3. Just do it. When a team is stuck, I find that the best remedy is just find what the next baby step is and implement it.
4. Work several options in parallel. Sometimes is just impossible to determine in what direction the next baby step should be, for these occasions just work in parallel all the different solutions. The overhead of working in parallel is paid back early in the project because it will be very clear what solution is the most adequate.
5. Delay decision for as late as possible. The general rule of thumb for the whole article can be resumed with the lean principle “Delay your decisions for as late as possible”, find baby steps, work on parallel… but don’t commit to an approach that may cause your project to fail.

Micro optimizations are one of many different bad practices that can be found in a project.

Micro optimizations are bad responses to potential or real project issues, they are created when the team fails to identify the root causes and instead they address the consequences, this usually happens because:

1. Urgency. Not enough time and resources are used to analyze the issue.
2. Experience. The patch strategy is always been followed.
3. Fear. Sometimes people know what the root cause is, but they are afraid to mention that.

Micro optimizations are bad because:

1. They switch the focus of the team from the big picture, they provoke the “Can’t see the forest for the trees” effect.
2. They provide partial feedback.

Micro optimizations are usually implemented as check lists, metrics, reports, documentation or some sort of paper work that needs to be approved in order to move forward.

Some common scenarios of micro optimizations that I have myself observed are:

• Because of the high rate of defects, the team decided they had to freeze the development from time to time and wait for QA to pass complete regression tests. This turned out to be a micro optimization and didn’t help at all. The root cause for the high rate of defects was that the source code quality was very poor, which in turn, was caused by a complete lack of experience and knowledge on the team on the technologies used for that particular project.
• Because of low productivity, management decided to hire more people for the project. Again, this turned out to be useless, the root cause for this issue was that development and product owner were not collaborating as they should, and as consequence, development never understood what was really required by the business.
• Because of poor performance, the team discovered that the bottleneck was on the messaging layer, so they decided to expend lots of money in a commercial solution that delivered better performance, this turned out to be a micro optimization and the performance issues came back. The root cause was that the overall architecture was poorly designed.

To avoid micro optimizations, focus always in the big picture, not in the details; use an out-of-the-box mindset. One helpful, very well known and very simple technique to avoid micro optimization is “the 5 whys technique”, this technique is better demonstrated with a sample.

The team only deliver half of what it was expected for the deadline. (The issue)

1. Why? They didn’t have enough time
2. Why? They had too much work
3. Why? They failed to estimate how much work they could take.
4. Why? They thought that the required activities to complete were simpler.
5. Why? The requirements were not detailed enough.

Avoiding micro optimizations doesn’t mean that the aspects they would usually be focused on, as: performance, punctuality, code coverage… are not important; it means that they serve to a higher purpose.

The biggest effect on moving away from micro optimizations is that you are likely to find that your main issues in your project are not technical, so probably you may want to take a complete new approach on them.

In my opinion coupling is a very dangerous metric, is something desirable, but if it drives the architecture, it will make your project over complicated, to try to prove my point I am going to start from the basics.

In object oriented programming, coupling refers on how strongly linked are two objects that need to talk to each other.

The degree of coupling is important because, given a change in one of the components, the chances of having to change the other linked object, is proportional to the degree of coupling. When talking about coupling, objects are considered loosely coupled when they are designed so that a change in one object doesn’t require the other object to change.

Loose coupling is one of the most desired qualities in modern software development, but because its very subjective nature, and because the lack of analysis on the different types of coupling, some wrong decisions may be taken in the architecture just for the sake of making the application more loosely coupled.

Considering “the looser the architecture, the better”, is wrong, for each two objects that need to be coupled, their circumstances should be analyzed and based on these circumstances, the most appropriate type of coupling should be used.

Types of coupling.

Direct coupling.

Direct coupling happens when the two objects that need to be linked, talk to each other directly.

From a hardware point of view, this coupling always happens between two objects from the same application which are sitting in the same shared memory area.

Direct coupling is the simplest coupling to implement, but is the one with the highest level of coupling. A change on any of the linked objects is very likely to cause a change in the other object.

There are two styles of direct coupling. Black box coupling and White box coupling.

• Black box coupling is a style of direct coupling where the link between the two objects is created without revealing any implementation detail. The most common form of black box coupling is when object A calls directly a public method in object B.
• White box coupling is a style of direct coupling where the link between the objects carries some implementation information. The most common form of white box coupling is when object A access directly a member variable of the object B. White box coupling is discouraged because it creates even higher coupling than black box coupling and the complexity and effort remains the same.

Indirect coupling.

Indirect coupling uses a third, or more elements, to couple the two objects that need to talk to each other. The purpose of adding additional layers between the two components is so that even if one of the linked objects change, the intermediary doesn’t, or if it does, it changes internally so from the other linked object perspective everything remains the same.

Indirect coupling adds complexity to your application but it offers the loosest coupling.

Indirect coupling can be classified depending on

1. Location of the components.
• In memory. The two objects that need to be linked are part of the same application and are loaded at the same time in memory.
• Remote. The two objects are part of different applications, so they don’t share the memory space. Remote coupling adds complexity but is considered to have a lower coupling than in memory coupling
2. Synchronicity.
   • Synchronous coupling. The standard indirect coupling.
   • Asynchronous coupling. Also known as fire and forget, if asynchronous coupling is required, the complexity of the application would be higher, but it will be likely to have a lower coupling.

Conclusion.

The main conclusion is: The lowest is the coupling, the highest is the complexity. The “Oh yeah! Let’s just take a loose coupling approach”, doesn’t work. What is really important in your design, is how simple it is, so it should be prioritized the simplest approach as possible, which, funny enough, will probably carry a higher coupling.

The degree of coupling should be dictated by the requirements, and the simplest approach should be preferred.

Not dealing with uncertainty efficiently is one of the main causes for software development projects to fail. Traditional approaches assert that uncertainty can be defeated by designing and planning ahead, but that’s wrong. Even in a small development, uncertainty is so high that to discover all of it up front is impossible. That’s why classic approaches, as waterfall, fail in dealing with uncertainty, and that’s why, in my opinion, they also fail in dealing with change.

Change and uncertainty are at the core of any software development, as Heraclitus said: “Change is the only constant”, and software development is not exception. That’s why dealing with uncertainty is so important. Some of the most common consequences of not dealing properly with uncertainty are: false expectations and bad estimations.

False expectations. Uncertainty is going to cause change, and change needs to get fed back to all the stake holders, but is very easy to leave information and people outside the loop, if this happens then the expectations are going to be different across the parties involved in the project causing that some of them will have false expectations.

Bad estimations. As I’ve already said before in this blog, I strongly believe that big planning up front is a waste of time, and that’s mainly because uncertainty. The cone of uncertainty is a very well known diagram which graphically shows this.

Source: http://www.codinghorror.com/blog/archives/000623.html

These are my 3 advices in order to deal with uncertainty.

Small steps

It is better to take many small steps in the right direction than to make a great leap forward only to stumble backward.

Source: http://atriskliving.blogspot.com/2008/09/goodbye-baby-step-1.html

It is impossible to get rid of all the uncertainty, so the best way to deal with it is to take as less uncertainty as possible at a time, your estimation then is going to be more accurate and the expectations across all the stakeholders in the project are going to be aligned. Small steps will also provide with quick feed back, so you can correct the direction in your project as soon as is necessary, and it will help you to reduce the total remainder amount of uncertainty.

As a rule of thumb, I don’t like to have tasks longer than 2-3 days. These tasks should cover a whole end to end scenario in your application and should have a clear acceptance criteria, an example of task for a online book store could be: “Add the option of payment with credit card to the checkout page”.

Iterations

Can’t See the Forest for the Trees

Small steps need to have some sort of higher purpose, if not it would be like trying to climb a mountain by never looking further than 5 meters, just always taking the steepest path, but that is usually not the best path to climb a mountain.

Iterations are short time boxed periods that wraps small steps, their purpose is to serve as control points to demo functionality to the product manager and ensure that the direction of the project is correct.

Communication

Good communication plays a primary role when dealing with uncertainty, it is key, that all the parties involved in software development are aware on how uncertainty develops.

What follows is a summary of my own high level approach for testing software, this strategy is based in facts and principles.

Facts

1. It is impossible to detect all the bugs from an application.

The closer you get to 100% coverage, the harder it is to find the remaining bugs.

2. The more important bugs are in the core layer and in the integration layer (backend).

That’s where the testing needs to be focus on. Core layer bugs and integration layer bugs are the most important because they create a cascade effect causing several parts of the application to fail.

3. Using UI automated tests makes harder the detection of bugs.

Even though they are still very popular, UI automated tests are not very effective finding bugs because they test the core layer and the integration layer indirectly. Testing indirectly the backend makes difficult to exercise it, and makes hard to tell where an error is coming from. It is also important to notice that UI automated tests are also slow and expensive to maintain.

4. Manual testing is still necessary.

There are some important bugs that can only be detected through manual testing, that’s the case of the bugs that can be found doing usability testing and exploratory testing.

5. Testing is worthless if is not executed in a continuous basis.

What’s been proven correct through testing now is going to change very soon so it will have to be proven right again. If this feedback is not fast enough, new changes won’t get proven and eventually new bugs will be entered into the system.

Principles

1. Prioritize what’s going to be tested.

Never have a test strategy that expects to cover 100% of the application.

2. Have as much automated tests as possible.

From your previous prioritization, automate as much as you can.

3. Schedule time for the necessary manual testing to be performed on the project.

Two of the main manual testing activities that are necessary to perform are usability testing and exploratory testing.

4. Use preferably backend automated tests instead of UI automated tests.