Making Good Software

There isn’t any standard java implementation.

In almost every single project I have been involved, at some stage, I have found necessary to use a tree data structure, but I have found myself rewriting the same code to represent it because Java doesn’t provide with standard interfaces and implementations for Trees.

There are some open source tree data structures, but they are not flexible enough.

It is also surprising that there isn’t any major open source project, at least that I’m aware of, that tries to fill this gap. I have found on the Internet a few attempts to implement trees, but even thought they are correct, I don’t think they are as flexible as they should be.

The usual solution, the composite pattern on the objects itself, is far from ideal.

The most common implementation of trees, is using the composite pattern on the objects itself, meaning that no abstractions are created for classes as Tree, or Node. This approach has two main flaws:

• Low cohesion: Now each object which is member of the tree needs to also have methods for navigation, insertion or other operations which are completely out of the scope of their own nature.
• High coupling: Each node knows about their parents or about their children, changing one of them usually requires to changes the others as well.

How it should be (IMHO)

In my opinion, these are at least the main characteristics that a flexible Tree library should have.

1. Ability to branch and sub-branch. Instead hanging nodes directly from other nodes, have the ability to add branches and sub-branches to nodes and hang the child nodes from them.
2. XPath enabled. So you can call something like Node<Something> node = tree.get (“path/to/node”);
3. Navigation specified through strategy. Navigator nav = tree.getNavigator (Strategy);
4. Ability to filter.Navigator nav = tree.getNavigator (Strategy, Filter);
5. Ability to create sub-trees from nodes. Tree subTree = new Tree (tree.get (“path/to/node”));
6. Multi-threaded.

Your opinion and collaboration.

The main reason to write this article, is that I’m actually about to start writing this library myself, and I would like to share it with everyone, but before, I would like to hear from you guys. What’s your opinion: Do you think this sort of library would be useful? Would you use it? Do you think there are classes already that I might not know that have this functionality?

I am really looking forward for your comments!

FYI:

The third and last part of the post Waterfall Vs Agile that I’ve been writing for DZone has just been published, follow the link to read it…

Waterfall vs. Agile: QA and Management

FYI:

The second part of the post Waterfall Vs Agile, has just been published in DZone, follow the link to read it…

Waterfall vs. Agile: Development and Business

Historically QA is been considered the last stage of software development, and QA testers have been seen as not necessarily qualified people, that all they need to do is to run manual or some automated UI tests over applications handed over by development.

This classic approach is somehow still used even in several agile projects, and it has many inconveniences:

1. QA becomes a certification/cop department that adds very little value to Development.
2. It only assures the quality of the final product, but not the quality of the code.
3. It creates the illusion that software can be created without bugs.
4. It’s not synchronized with the business needs, but with the requirements contract.

QA should switch to a new paradigm, QDD – QA Driven Development. In QDD the main role of QA should be to engage with development and the customer.

In order to achieve this new paradigm, QA should:

1. Review not only the final product, but also the code. The code is the big gap in classic QA. It is never itself checked for quality. Low quality code will make the maintenance of the application a nightmare.
2. Be involved in the day to day of development. QA should be present in every single important decision taken during development. They should also be aware of what the code looks like through daily code reviews or pair programming.
3. Have qualified and skilled employees. QA should drive the development, and that requires QA personnel to be very skilled and qualified.
4. Keep continuously updating the customer. QA should make sure that the application as it’s developed is fulfilling the customer’s expectations. It should also be their duty to guarantee that new requirements are identified, and also to identify previous requirements that are not necessary anymore.
5. Decide when a feature is completed. As the top part of the pyramid of software development, QA is the most qualified member to decide when a feature is completed.
6. Create a comprehensive suite of automated tests. QA should aim to cover as much as possible with automated tests, this will guarantee that regression testing should be easy to perform for each new feature.
7. Handover requirements to development. QA should have the best understanding of both, technical and non technical aspects of the application. That is what will make them the perfect candidates to handover new requirements.
8. Have formal authority over development. None of the previous points would be possible is QA is not given formal authority over development.

Based on these characteristics, QA personnel will have to be well trained in computer science, actually, QA should be an specialization of software development, and not a completely different branch. An ideal candidate for QA then should be a seasoned developer, with coach and lead abilities.

Of course to move to QDD there are very big challenges, being the three most importants:

1. Companies that have QA infraestrucutres and personnel not fitted for this new paradigm, and not wishing to lose their investment.
2. Developers not willing to become QA.
3. Companies are afraid of the over cost of switching non-qualified QA people for skilled developers.

While these are important factors to overcome, in my opinion switching to this new QA paradigm will bring benefits in the mid/long run as the quality of the applications delivered will grow exponentially.

To all my readers.

I’ve been honoured to author an article for dZone, “Waterfall vs Agile: Can they be friends?” If you follow the link you should be able to read it.

Thanks!

I still remember how naive I was just after I finished my studies. I was convinced that I was ready to join any software company and start shining as a top class developer. Obviously, no long after I started working, I realized how many things I didn’t know.

As I have been acquiring experience, I have been learning things the hard way, stuff which I was never taught, and which its understanding, is basic to become a good developer. This is my list of the 10 things I wish I had been taught.

1.- We’re always wrong.

Developers, among other non technical defects, have quite big egos, that´s why is very hard for us to recognize that we were wrong about something. I’ve seen many endless design discussions where developers will go on and on with their own ideas… well, guess what.. We are all wrong, the only difference between our positions is the degree of wrongness.

Is very important to understand and embrace this fact, only once that we do so we will be open to listen to others and use their own ideas to build a better solution.

2.- If something can break, it will break.

Aka “hope driven development“, if you are not sure about something, if you find yourself using the word “should”, then you are in trouble.

There is only one solution for this, do whatever is necessary to make sure that it doesn’t break, this could mean that you will have to write a test, debug, clarify the requirements…

3.- All code is crap.

After 10 years complaining about all the code that surrounds me, I have come to a brilliant conclusion, all code (including mine), is crap. Of course there are many degrees of crappy code, but even the best code I’ve ever seen is not easy to read.

This doesn’t mean that is not worthy trying to make your code better, all the opposite, there’s a huge difference between the best kind of code and the worst kind of code.

4.- There is always a bug.

ALWAYS! Is just a matter of how hard you look for it.

5.- The most important thing is the client.

Among many things the client doesn’t care are: the technologies you use in the project, how many more things that required the application does… or in general, if you use good practices.

And since I can imagine how much hate comments I will get if I only leave the previous paragraph, let me clarify what I want to say… We should never forget the client perspective, sometimes developers use technologies or insist in over engineering just for the sake of using best practices, but remember, if it doesn’t add any value for the client, drop it.

6.- Design on paper doesn’t work.

I used to believe that I could put my whole design in a paper upfront, and then just fill in the gaps, but it simply doesn’t work.

Software development is complex, is hard to see all the entities and their relationships until you don’t get the hands dirty. So keep planning and designing upfront, it is very helpful, just don’t try too hard, and don’t take the diagrams as contracts.

7.- Less is more.

Or as you probably know it better: “Keep it simple, stupid!” (KISS). So if is not necessary drop it, because remember: “If something can break, will break”.

8.- Coding is only 20% of what we do.

Be ready to spent 80% of your time thinking, debugging, testing, in meetings, conversations… And all of the other activities are very important, so you have to develop a wide range of skills, not only technical, to become a good software developer.

9.- The customer doesn’t know what he/she wants NEVER!.

Customers have a necessity, or an idea, but they don’t know the details… Software development is all about discovering the details and removing all the uncertainty and converting them into an application.

10.- Someone has done it before.

So don’t reinvent the wheel, use Google, or better yet, ask your colleagues, many times they may have done already the same or something very similar.

Bonus: Hey! Our job is cool!

On the bright side, programming is still cool!

Domain modeling is the most important part of software design. Having a good model allows developers and business to have a common language, which in turn, makes much simpler the communication of requirements and the maintenance of the application.

Having a good model is synonym of having a low representational gap. Having a low representational gap means that the main concepts and their relationships from the real business model are represented almost identically in the software domain model.

Creating a good domain model is one of the most difficult challenges developers have to face. What follows are some advices to create a good domain model

1.- Understand the business model completely.

In order to create a good domain model, developers need to look into the detail of the the business model, not only in the surface. It requires to understand and distill the main entities and then find their closest equivalent software abstractions.

2.- Use good names for all your domain model classes.

Naming is critical in software domain modeling. There are two qualities that you should be looking for in a good name for an entity in your model:

• Consistent. Business and developers should always use the same term to define a single concept.
• Exact. A name should exactly define the responsibilities of an entity in the domain model.

3.- Use metaphors.

Many times is necessary to model abstract operations/entities in software development, (processing orders, simulations…), for these cases, metaphors will help you to simplify the design of the domain model.

4.- Make the domain model self-contained.

The domain model should contain the raw logic of the application. Any additional logic should be contained in some other layers, as for instance the work-flow, dependency injection, persistence…

5.- Make the domain model independent.

The domain model shouldn’t change because the underlying persistence implementation or the UI or any other layer changes.

6.- Make the domain model technically simple.

It should only contain POJOs. The responsibilities assigned to an entity in the domain model should be limited to only those necessary for the software domain

7.- Keep the domain model updated.

Synchronize it always with the business domain model, having two unsynchronized models is one of the biggest sources of productivity killers. Synchronization requires refactoring and enough test coverage to make sure that things don’t break as they are refactored.

8.- Make the software domain model match the business domain model.

A software domain model should be easy to understand for a business domain model expert and it shouldn’t have any discrepancy.

9.- Care for the code quality of your domain model.

There are two reasons why you want to make sure that the code quality of the domain model is top-notch.

• The domain model is the foundation of your application, if it is bad, the whole thing can fall apart
• It changes a lot. There are very few areas in your code that are going to change so many times as the domain model

10.- Test the domain model independently, and involve the customer in the test.

As already mentioned, the domain model is an independent and very important part of the architecture of the application. That’s why is so important to pay attention to test it in isolation. Involving the customers in this test phase can be very productive as they can identify test cases that are likely in production and which probably are the most interesting ones to test.

There is lately a lot of hype around engineering practices as TDD, Refactoring, keeping things simple (KISS)… and while they are great, it is important to remember that they will never replace your intelligence and experience.

Engineering practices are considered by some developers to be the best tools in your toolbox for coding but the best tools are still the very old same ones used to solve any problem: a board, or a piece of paper, and some other good programmers to discuss the issue with.

Some developers, usually action heroes, tend to overuse engineering practices (mainly refactoring and KISS), falling in the “no need to think” mode. This particular kind of Hope Driven Development (HDD) is a constant source of bugs and productivity killers. It is like if we try to go from A to B only looking at our feet. It can only work when there isn’t any big, or unexpected obstacle in the path.

Engineering practices only help you to drive your code, but they can’t take any decisions for you, so before you start coding Think!

• Always ask other colleagues to review your ideas and code, and offer yourself to do the same.
• Use meeting rooms with big boards. It might look ridiculous if your are not used to it, but drawing on a board is a great way to share your ideas.
• Thinking beforehand doesn’t mean that you have to come up with the complete design of the application. That is impossible, but at least try to identify the big  picture of  what you need to design.
• Don’t let the enthusiasm for engineering practices make you forget that your intelligence and analysis are your best tools to create good code.
• Use engineering practices.  They are great and will make you a better developer, but don’t hope that they will come up with the design for you.
• Don’t ignore parts of your design because they are complex. If you have to ignore them, do it because it is not necessary to deal with them right now. If you are going to code something and you know already that you are going to deal with some complexity try to fit how you are going to deal with it in your code.
• When coding a part of the application, think on the big picture and how are you going to integrate that part with the rest of components of your application.
• Remember that apart from your intelligence there are other things that are more important than the engineering practices such as having fun or other non technical skills.

Many project managers consider finding major bugs during QA an unexpected occurrence, which is like going to war hoping not to have any causalities. One consequence of such naive reasoning is that they usually schedule QA as the last stage of software development just to find that:

• It doesn’t allow for any reaction time when major bugs are found.
• It creates a lot of tension between QA and development. It becomes crucial for development to pass QA because they know they won’t have enough time to fix any major bugs.
• It is not suited for exploratory testing. Exploratory testing, allows testers to have an out of the box approach to testing which is excellent to find the most important bugs of the application.
• It requires heavy handovers and setup.

A lesson I learned quite a while ago is that QA as a separate process may work in other industries, like manufacturing, but it doesn’t work in software development. Every single unit of functionality that is coded for an application should be QA’d immediately. In particular referring to QA, these are my principles.

1.- There are no handovers between development and testing.

Coding and testing are not two separate processes, they are performed in parallel, and one can’t be completed without the other one and vice-versa. [extracted from: Continuous Integration, going beyond continuous compilation]

2.- Developers and Testers have a common goal.

They don’t perform different activities. Developers are not just focused on developing and testers in testing. They share a common goal- to produce quality software in coordination with the product owner expectations. [extracted from: Continuous Integration, going beyond continuous compilation]

3.- QA and development must provide their schedules as one unique schedule.

There isn’t one schedule for development and another for QA, and that’s because they depend on each other.

4.- The QA tester role shouldn’t be less valuable, skilled or recognized than the developer role.

I have worked in many projects where QA resources are seen as second class employees that perform easier tasks than developers. In many projects, the kind of role that is pursued is a tester that only follows a test script, taking notes on the expected and actual result for each step.

In my opinion, testers add a lot of value to the project, they should be responsible for writing automated tests, setting up stage environments and coordinate with the business making sure that they know what they expectations are.

5.- QA tester and Developer don’t need to be two different people, it is only two different roles.

Because QA used to happen at the last stage of software development, and because it was seen as a sort of certification process, management decided that QA should be part of a completely independent team, and that introduced the idea that the QA tester and the developer should always be not only two different roles, but two different people.

6.- QA should be scheduled so that I could take up to 50% of time of development.

This is just a rule of thumb, and of course it depends in a case by case basis, but is important to understand that QA is a very important task that when is completed could take up to 50% of the development time.

Many times, software development projects fail because of bad project management. From my experience, these are the 5 most common and dangerous mistakes made by software development project managers.

1.- Believing that productivity is mainly associated with man power and working hours.

Productivity is today’s equivalent to the philosopher’s stone in the middle age, every project manager tries to find it applying basic alchemy: the more developers and working hours, the better.

Unfortunately, this approach is delusional, it is not scientific at all. There are many studies that prove that productivity can be even negatively affected applying this basic principle.

As for references, the most important one is probably “The mythical man month“, one of the most influential books in software development for the last 20 years, (unfortunately I’m afraid is mostly read by developers and not project managers). As for articles in Internet, there is one that I would particularly recommend, ii is from lostgarden.com: “[PDF] Laws of productivity – 8 productivity experiments you don’t need to repeat“. I encourage you to forward it to any project manager you may know that you think may need it.

Obviously then the question is: What improves productivity? Well, while I can’t say I know the exact answer for this, (If I knew it I would probably be rich now). In my opinion it’s mostly affected by two major factors: motivation and leadership. That is, willingness to move forward, and someone to show what the best direction to move forward is.

2.- Not managing customer expectations.

Project managers need to make sure that customers understand what and when is going to be released.

One of the most recurrent situations where expectations need to be managed is when a project is going to be late. What usually happens when the project is late is that the customer is not told about it because the project manager hopes that the deviation can be corrected or because is afraid of disappointing the client. BUT what more often happens is that the customer is told too late of the delay, when there’s no other option, and what it could have been a minor, or not so big issue by managing the expectations and postponing the deadline or the scope of the release, becomes a major issue that sometimes puts at risk the entire project.

3.- Ignoring feedback from developers.

A Project Manager need to have his thoughts in a higher level than developers, but that doesn’t mean that he shouldn’t listen to them.

In software development, programmers are the only source of productivity, and for that reason, there is nothing more important than making sure that they have what they need to do their job.

Unfortunately many project managers prefer to focus on dates, meetings and deadlines, which sometimes is good as it makes the project move forward, but sometimes developer feedback is overlooked causing productivity issues, technical debt, low quality products, etc.

4.- Controlling as opposite of empowering.

Software development requires creativity and pro activity, and that’s two qualities that control freak project managers destroy.

Controlling is especially practiced by project managers that still believe that all the requirements for a product can altogether, with the technical specification and the schedule, be specified in documents at the beginning of the project.

Empowering is all about making developers better than they are, and it is usually the cornerstone for killing applications.

5.- Being afraid of making decisions.

One of the reasons why project managers make more money than developers is because it’s up to them to guarantee that the project is on track. Often that requires them to make tough decisions like fire, hire or reprimand people.

Project managers that struggle to make tough decisions as soon as they are necessary, are failing to their team. Delaying tough but important decisions will cause even more severe consequences in the future.