Making Good Software

A blog by Alberto G (Alberto Gutierrez)

Written by Alberto Gutierrez

January 27th, 2012 at 7:54 am

The evil unit test.

with 14 comments

Unit tests can be evil, I know that sounds harsh, but I think there is a battle to be fight to make people understand that unit tests are not always good, and that they actually can become evil, specially when overused.

Think in a foreach, most languages have an implementation of this flow statement. Is the foreach a useful utility for programmers? Definitely! What about function pointers? or Design patterns? or ORM frameworks?… All of them are useful utilities, but can you imagine a programmer, or a book naming the following principles?

  • You shall use a foreach every time you need to perform a loop.
  • Every third method shall implement a design pattern.
  • You will never access your database if you are not using an ORM.

Stupid, right? So why is it that there are programmers happy with statements like:

  • Every public method shall have a unit test.
  • Unit tests are not to be deleted. There is always a reason for a unit test to exist!
  • Unit tests must be created before the code they test.
  • Unit test coverage should be 100%.

Unit tests, as any other utility, tool, principle, philosophy etc. are only to be used when applicable and convenient, and not to be taken as religious dogmas. I have already made this point in my previous article, but since it generated so much controversy I thought it would be good to expand on it. If you can’t unit test a piece of code from a behaviour/black box perspective, then you probably shouldn’t write a unit test at all. Your unit test should check what your code is doing, not how. If you create a unit test that it only focus on how your code is, instead of what it does, then you are writing an evil test because:

1. Evil tests create a lock on how the code is implemented.

Every time the implementation of the method changes, even if its behaviour is maintained, the test becomes red. There is no reason to lock the implementation, because there is no advantage on doing so. If there is some code that is especially tricky, locking its implementation with tests won’t make it easier to understand or won’t force anyone to think twice before changing it.

2. Cause duplication.

The beauty of code with good unit tests is that they compliment each other; the code to be tested represents the implementation (the how), and the test guarantees behaviour (the what). On the other hand, evil unit tests look again into the how, so you will have two places where you state how do you think a problem should be solved, only that you will be using different expressions to do so. Is like saying: “2*3 = 2 + 2 + 2”, and then writing a test that says “Guarantee that 2*3 is the addition of 2 three times”, as opposite to “Guarantee that 2*3 is 6”. Testing the implementation and not the behaviour is a waste of time and serves no purpose since you can always go to the code to find what the implementation is.

3. Builds uncertainty on the tests (red is meaningless).

One of the main advantages of having an automated test suite is that feedback loops become shorter. It takes less time from creating a bug to detecting it. When the code gets polluted with bad unit tests, the automated tests seem to start getting broken randomly and this sometimes leads to a point where developers don’t care much anymore about having their tests passing as usual as they should.

4. Decrease productivity.

Given all the circumstances mentioned already, developers dealing with evil unit tests are faced with many inconveniences that forces them to expend time in unproductive tasks, like refactoring useless unit tests, writing them…

5. Discourage change.

Code bloated with bad unit tests is a nightmare to maintain, you make an implementation change, not a behaviour change, and you have to expend ages fixing tests, hence you minimize your implementation changes which is the core idea behind refactoring.

Consider the following principles then:

1. Delete evil tests. There seems to be a taboo around deleting unit tests, is like they are untouchables, well that’s no true, if the test is not adding value, then the test is an evil test and MUST be deleted.

2. Minimise the areas of your code that can’t be effectively unit tested. It is also true that sometimes the problem is that the code is written so that areas that should be unit testable are tied to components that make difficult the unit testing, in this case, the programmers needs to expend more time understanding the principles behind loose coupling and separation of concerns.

3. Write integrations tests for areas that are not unit testable. There is always going to be code in your application that is going to be legitimately not unit testable, for this code, simply don’t write a unit test, write an integration test.

 


Written by Alberto Gutierrez

December 15th, 2011 at 4:43 pm

How to write efficient unit tests. 5 principles for unit testing.

with 11 comments

In these last years, since the first unit testing frameworks were made available, and methodologies like TDD have become mainstream, unit testing is turning into a more popular strategy in software development.

The main advantages that unit testing can bring into a software development project can be summarised in mainly two purposes:

  1. Design purpose. Help programmers creating new code.
  2. Correctness purpose. Make sure that the behaviour of the different independent small parts of the newly created code are correct.

But what usually gets overlooked is that unit testing has also risks involved, specially when applied using dogmatic approaches like “every public method should be unit tested” or “everything needs to be design so that is easily unit tested”.

The main risk with unit testing appears when too many unnecessary tests are created. This risk becomes obvious when every time that the code is changed the time spent fixing tests is way too high, hence impacting in the productivity of the developers.

The key to effectively use unit tests is to find a balance between your tests and the amount of time you need to maintain them. When looking for this balance is important to remember a few principles to ensure that you write efficient unit tests.

1. Behaviour is the key element to test.

Focusing in testing behaviour is the key to produce a good unit test. This way if you refactor the logic inside a method without breaking its behaviour you should keep all the related tests passing.

2. Not every method is applicable for unit testing.

There are many dogmas in agile, and to have a unit test for each public method seems to be one. While is true that if possible a method should be unit tested, if you cannot test its behaviour, is better to leave it alone. (See point 1) Unnecessary unit tests are going to lock you down from changing your code for no good reason, (See point 5). These areas of your code that can´t be unit tested should be tested from an Integration or manual perspective. (See point 4)

Some clear examples of these types of methods are those that encapsulate calls to a framework, methods that loop through a list of items and then delegate to a different method, methods that log out…

3. Unit tests which are not useful anymore should be deleted.

Tests are created mainly in two fashions:

  • After the code is completed: These tests are created to built in an automated check for correctness in the associated methods. Creating useless tests (See point 1 and 2) may be made by mistake, if so, you shouldn’t feel any shame of deleting these tests, or if possible to refactor them to focus on the behaviour.
  • Before the code is completed. (Specially known for TDD). In these cases, tests have a second goal, to help programmers to come up with a cleaner code. After the code is complete, is usually a good idea to review the tests created and to delete/refactor them wherever applies. This unfortunately never made it to the list of steps to follow in TDD…

 4. Unit tests will never substitute manual and integration testing.

Unit tests, once that your code is completed, help you diagnose weather the individual parts of your application are working as you expect. This is important, but is very far away of proving that your application is robust and works according to your customer expectations, which is your main goal.

Units tests are only a small part of the complete picture, you are going to need integration tests for areas of your code where unit tests can’t prove their behaviour, and you are going to need manual testing in areas where you can’t create an automated test, or for more abstract areas like, usability and UI testing.

5. Unit tests that lock down your code from changes are evil.

If there is one particular type of test to be avoided at any cost, these are the tests that lock down your from changes without adding any value. Let me illustrate this with some pseudo-code:

MyClass.MyMethod (magicParam1, magicParam2)
   START
      magicReturnValue = someOtherClass.doSomething (magicParam1, magicParam2)
      veryRemoteClass.stuff (magicReturnValue)
      managerOfManager.buzzinga(magicParam2)
   END

MyTestClass.MyMethodTest
   START
      when (someOtherClass.doSomething (magicParam1, magicParam2)).
         thenReturn (magicReturnValue)
      myClassToTest.MyMethod (magicParam1, magicParam2)
      verifyICalledThis (someOtherClass.doSomething (magicParam1, magicParam2)).
         andReturnedValueIs (magicReturnValue)
      verifyICalledThis (veryRemoteClass.stuff (magicReturnValue))
      verifyICalledThis (managerOfManager.buzzinga(magicParam2))
   END

What is the previous test achieving? Well, is actually achieving a lot… of pain… This is the one thing that sets me off when I see others people code, not only this is not proving anything about the expected behaviour of your code, but if someone refactors the main class maintaining the same logic, he will find that this test fails miserably, only because the code changed, not because there is any unexpected change of behavior in the code…

Funny thing about this type of tests, at least for my experience, is that they are usually most defended by the extremist agilist, everything must be unit testedHave they perhaps forgot about their beloved agile process motto?


Written by Alberto Gutierrez

September 12th, 2011 at 3:43 pm

My 7 principles to design the architecture for a software project.

with 5 comments

Software architecture is quite a grey area. It involves taking decisions that you are not likely to revisit in the future. These decisions usually involve frameworks, programming languages, application servers, databases, e.t.c.

Software architecture is about coming up with the foundation from where developers are going to build up the application. This also includes: development environments: (source control, building tools, IDEs…), QA environments, Continuous integration environments, etc.

Bad architectural decisions can make the development of your project a complete failure, and are the most difficult decisions to revert.

Good architecture, in the other hand, for most projects, doesn’t really bring any advantage, it only allows for its normal development. Good architecture usually remains hidden under normal circumstances.

When designing the architecture, is essential to avoid bad decisions that are going to tax your development in the future. What follows is a list of principles to help you come up with safe and scalable architectures.

1.- Start with the minimal necessary architecture.

Avoid unnecessary complexity like unnecessary new technologies or frameworks.

Any architectural element needs to have a very good reason to be used in the project. If the benefits or necessity of an architectural element can’t be proved, then that architectural element should be removed.

2.- Consider the specifics of your project: Constraints and risks.

Defining the architecture is something that has to be adapted to the constraints and risks of your specific project.

You shouldn’t aim for the same kind of architecture if it is a start-up or a project for a multi-national company. You should also consider the expertise of your team, if they are very experienced in a particular programming language/framework… you should probably use it even if it is not the one you prefer.

Risk areas, like a requirement to have some service with a high availability, are going to require additional considerations in the architecture that you need to keep in mind.

Constraints and risks are the element that are going to move you away from your comfort zone, but they need to be looked at, to make sure that if necessary, they get addressed from the architecture.

3.- Grow as you need.

Delay any architectural element as much as you can, especially if they are complex.

Sometimes requirements seems to call for complex integrations/frameworks… like rules frameworks, ESBs… for these cases, I find it better to ignore them at the beginning and try to complete the first end-to-end scenario as soon as possible. When this first scenario is completed, then grow from there, one step at a time, adding new functionality as requested by the user. This way, it will become obvious if that architectural element is really necessary, and if it is, it will also be obvious how you should integrate it with the rest of the architectural elements in your project.

4.- Deliver continuously.

Continuous delivery is the vehicle to allow you to refine your architecture as you go.

Continuous delivery is based in four principles:

Deliver soon: The purpose of delivering soon is to shorten the feedback cycles with the customer, allowing for changes in the requirements if necessary.

Deliver many times: Delivering many times guarantees that the feedback cycle is smooth.

Bug free: The code must be clean of critical bugs.

Production ready: It should be deployable at any time.

 5.- Require a customer.

Having someone acting as a customer, making decisions about priorities and signing off releases is critical for the good development of your project.

Failure to have someone representing correctly the role of the customer causes misdirection in the development and consequently causes that the final product developed won’t meet expectations.

6.- Avoid waste.

Architecture can lead to an excess production of artefacts, inexperienced architects may pretend to foresee the entire architecture of the application and have it specified in paper.

This is especially true in companies following waterfall like processes, and sometimes it cannot be avoid, forcing the architects to produce such and such document/diagram.

Excessive initial architecture/design, aka Big design upfront (BDUF), is not only bad because is wasteful, its main risk is to let the development team believe that they cannot deviate from the original specifications, making it very difficult to be adaptive to future changes.

7.- Maximise feedback and transparency

Feedback and transparency guarantees no surprises for the customer, which in turn helps building a more trustworthy and productive relationship.

Maximising the feedback and transparency are one of the ultimate goals of the development process for any project, this allows for early and informed reaction when change is necessary, whether something is going wrong, or whether the customer wants to introduce a change in the specifications.


Written by Alberto Gutierrez

July 31st, 2011 at 4:07 pm

Agile: The good, the bad and the ugly.

with 5 comments

Most companies are adopting Agile as the “de facto” process for their new projects, while is clear that overall is better than waterfall, it still has its defects. This article does a brief overview of what is good, bad and ugly about agile.

The Good.

Embrace change. Change is the only constant in software development; not embracing this is like swimming against the flow. Embrace change is precisely Agile’s motto. Agile supports change by developing in short iterations, which allow for faster feedback and consequently allows for changes in the features to be developed.

Empower programmers. Good, empowered programmers make good software. But that’s not easy, agile empowers programmers by giving them the most important role in software development moving it away from contracts, schedules and management.

Disregard BDUF. Big design upfront is one of the major signature features of the classic waterfall developments, not a single line of code is produced until a complete design of the application is completed. This is not only bad because it goes completely against embracing change, is also not realistic since any complex design has too much uncertainty making it impossible to produce a complete correct design upfront.

The Bad.

In some cases, an extreme approach to Agile may cause some issues, which from all of them IMHO the next ones are the most relevants:

Too much focus on the code. All code sucks, pushing too hard to produce beautiful code is pointless. This doesn’t mean that there aren’t big differences between different codes, there is code that is terrible and there is code that is alright, but there isn’t beautiful code. Unfortunately extreme agilists tend to push the idea that the beautiful code is achievable causing developers to spend too much time looking after it.

Complete disregard of design. BDUF is bad, but not doing any design at all is also bad. In extreme agile there is a tendency to let the code drive the design, which is the opposite of BDUF. The ideal situation is a healthy balance where some informal design and thinking are performed before coding, then some coding, then some more design and so on.

The Ugly.

Apart from tendencies which are clearly bad, agile also promotes some other practices, which while not being extremely bad feel like a waste or like they add very little value.

Holy events. In agile there are many activities that the team performs religiously, like stand ups, planning meetings and so on. These activities are performed like mass, forcing the same structure and same formulas all over again. These “holy events” usually add very little value and may even become a communication anti pattern.

Alternative estimation methods. Accurate software estimation is impossible, for two main reasons, change and uncertainty. Ironically some agile families still try to provide with some sort of reliable estimation mechanisms.

 

 


Written by Alberto Gutierrez

July 17th, 2011 at 3:47 pm

Managing crises effectively in software development projects.

with one comment

In a green field project, there is always an initial euphoric phase where everything is great.

Unfortunately, sooner or later things turn for the worse and the first crisis hits the project.

  • You cannot avoid having a crisis, but you must manage them. There hasn’t ever been any project that didn’t have a crisis, and yours is not going to be an exception.

Crises are about not being been able to meet expectations and they involve different actors, including “the business”.

  • Manage expectations, make sure that “the business” understands that having crises is normal, and do it before they hit you; the best period is the honeymoon that is the start of the project. Sit with the business and explain what your crisis plan is, availability, backup plans …

Crises can be sometimes prevented beforehand, the worst type of crises are the ones caused by development not being honest with the business, common examples are not telling that the development is late or lying on the real set of features already developed.

  • Be transparent. Managing a project is to take responsibility when things go wrong and making sure that all the stakeholders are well informed of what is the current status of the project. Hiding information is never good in the long run.

Unfortunately, one of the most difficult activities is to learn what the real status of the project is. There are many projects that died because it was discovered too late that the project was heading the wrong direction and it was too late.

  • Be aware of what the real status of the project is. Many project managers are so much obsessed with only reporting green light to upper management that they live in denial. Gather as much information as possible, disregard subjective information as “this should be fine”, or “we are almost done”.

The budget and or the schedule, can’t be both sacred, that’s one of the toughest realizations in software development and one of the most difficult things to explain for project managers, the main reason for this is uncertainty, there is always a lot of uncertainty in software development that causes a lot of change.

  • Manage change and uncertainty. Be aware when the conditions of the project change and make sure that the necessary actions are taken to accommodate the project to the new conditions.

Even when the project is under control, you still are going to be face with crises, but they are going to be crises of a different nature, mainly accidental nature: data lost, hardware failures, bugs…

  • Be always ready for accidental crises. Have an emergency plan, you need to know who, how and when can be contacted in case of emergency and what availability they are supposed to have. You also need a data restoration plan. Most importantly remember to do fire drills: Simulate crises in a production like environment and make sure that everything works as you expect.

As already mentioned, there is nothing that can fully prevent you from having a crisis, so when they do appear, make sure you react accordingly.

  • Be professional. React calmly but with decision. Focus your energies in finding a solution for your crisis.

When there is a crisis, the project mentality switches to emergency mode. In emergency mode everyone should be given the 110% of themselves, this is very productive, but also, very stressful.

  • Try to spend as less time as possible in emergency mode. When you spend too much time under a high quantity of pressure you will start becoming less productive.

Too many crises in a short period of time is usually the sign of a major underlying issue, the worst remedy is to keep the team in perpetual emergency mode trying to extinguish all the fires at the same time.

  • On perpetual crises. Stop, analyze the situation and tackle one crisis at a time. This is sometimes very difficult to explain to the stakeholders, which may expect immediate solutions, but is the only way out in the medium-long term.

Unfortunately crises are also usually recurrent, and they keep reappearing.

  • When tackling a crisis be sure not to focus only in the consequences, but also the causes. A crisis should be fixed with guarantees that it won’t happen again in the future.

One of the biggest differences between software development teams is how they react to crises, being able to manage them makes a huge difference. Keep in mind that many times making mistakes is not as important as how you manage the situation after a mistake is being made.


Written by Alberto Gutierrez

June 15th, 2011 at 4:38 pm

Top 10 basic java advices.

with 6 comments

1.- Make the classes that contain the logic of your application stateless.

A stateless class is a class which doesn’t have state, or which state is formed by injected dependencies used to delegate behaviour. Since these classes don’t have state, the outcome of their methods is only dependant on their input. Stateless classes have the following advantages.

  • Thread safe friendly.
  • Deterministic behaviour
  • Easy to test.

2.- Use unmodifiable beans as much as you can.

Unmodifiable beans are classes which state are set on creation and from them are prevented to change; in java, using the modifier final in your member fields best approaches this.

Unmodifiable beans offer the following advantages.

  • Thread safe friendly. Actually: stateless logic classes + unmodifiable beans = Thread safety!
  • Robustness. Unmodifiable beans eliminate the risk of holding references to objects that keep changing.

3.- Try to have your classes matching one of the following categories.

There are two main concerns a class can deal with: logic and data modelling. Having classes that deal with logic or data that don’t belong to them  make your code more confusing and cluttered. Based on these two concerns, there are mainly 5 types of classes that you should try to adhere to:

  • Service. 100% logic, publishes the main public end points of your application, they usually delegate to managers through dependency injection.
  • Bean. Mainly data model but also logic, used to carry data and perform operations, the operations the bean can perform should be self contained.
  • DTO. 100% data model, is a particular type of bean used to transport data betweeen different mediums.
  • Business logic manager. 100% logic, encapsulates the implementation of the business rules defined for your domain.
  • Integration manager. 100% logic, glues together business logic manager + external frameworks used in your project + beans.

4.- Avoid static methods.

Static methods usually contain miscellaneous logic required in your application, because is difficult to find an already existing class in the code to host them, they sometimes are created as static methods in abstract classes. It is much better to encapsulate the static methods as instance methods in utility classes such as AppUtils… The disadvantages of using static methods are:

  • They are more difficult to mock out in tests
  • Their implementation can’t be hidden behind an interface
  • They can’t be injected.

5.- Use composition over inheritance.

Inheritance heavily violates encapsulation, and easily generates obscure code. One clear example is the java io library which entirely designed with the decorator pattern. Composition allows you to have completely equivalent functionality, but without breaking encapsulation, it is also the core concept of the most important design pattern from all them all, the strategy pattern.

6.- Design in layers.

Almost every single business application has a few broad main areas of concern that are completely unrelated from each other. These areas of concern should be translated into layers in your design to allow for a cleaner code. The most common layers are:

  • Persistency
  • Model
  • Integration
  • UI

7.- Avoid child to parent, or child to sibling relationships.

Design is the art of abstracting a problem into its different entities and their relationships. These relationships most of the time express some sort of parent-child relationship, as entity A contains entity B, or entity A manages entity B. Having these relationships implemented only from the parent to the child allows for clearer an less coupled code. One clear example of this would be the way Lists are implemented in Java.

8.- Keep the invariants to be maintained by the programmer to a minimum.

Invariants are conditions that need to be kept in order for an algorithm to work as expected. For instance, constants are usually defined with Simple data types as integers, strings… and if they are used anywhere in your code, is to be expected that you will receive not any value, but the ones specified by your constants. Other examples of invariants are not so obvious, imagine that there is a service with a method process, that retrieves an order, the only parameter that should be accepted is the order id, if the method requires something else, like the creation date of the order, we are adding a unnecessary invariant to be kept by our consumers.

9.- Avoid side effects in methods.

A method should only do one thing. Methods with side effects are dangerous, as the output of the method is never clear, you should always aim for deterministic methods with no side effects and which only read their input, never change them.

10.- Test.

Test, test, test… In software development is more productive to produce 7/10 of a robust application than a completely flaky application. Based on the previous classification of classes I would recommend the following approach for testing.

  • Don’t test your beans.
  • Unit tests your business logic managers.
  • Integration tests for services and integration managers.
  • And finally… always spend some time doing manual end to end test.

10+1.- Your judgement must be above everything.

Software development is not an enginery; there isn’t a set of well-defined problems that have well-defined solutions. Is not like building a bridge, or a building… every development is unique, what applies to one project doesn’t necessarily apply to other. What this means, is that there isn’t absolutely any rule that has to always be applied, all are best practices, or advises, but is up to you to decide what approach is better suited for the problem you have to solve now.


Written by Alberto Gutierrez

May 23rd, 2011 at 3:36 pm

Top 7 programmers bad habits

with 21 comments

1.- The all code is crap, except mine, attitude.

I have bad news for you buddy, all code is crap. No matter how much effort you put on it, there is always a majority of programmers who are going to think that your code sucks and that they could have done it 10 times better. I have already covered this topic in previous posts, you can find more information of what exactly I mean when I say that all the code is crap here and here.

How to fix it: Don’t criticise others people code, it could be yours the one in the spotlight, try to make objective and professional observations instead, but don’t judge. Be humble and try to learn from everyone around you, hopefully then your code won’t be so bad.

2.- The “I fix that in a second” catastrophe.

homer-simpson-dohTaking shortcuts is very tempting, everyone has done it. There are actually situations where they are necessary, but in overall, they are dangerous, very dangerous and should be avoided. A shortcut which goes wrong may save you a few hours, but may cause months of pain.

How to fix it: Don’t trust yourself when carrying delicate activities. Ask someone else to review what you are doing. Make sure that if you are about to take a shortcut, you make very clear to the stakeholders what the reasons and the risks are. Try to get a manager to sign off every time you are about to take a shortcut, aka “cover your ass”.

3.- The “That will only take a second” misconception.

Being Barcelona my hometown, I am very proud of the Sagrada Familia Cathedral, which is very well know for its beautiness, and also for the time is estimated it will take to complete, (in construction since 1882), but that’s probably because they didn’t ask a programmer to estimate, otherwise the estimate would probably have been somewhere around 2 weeks.

How to fix it: For starters, is important to understand that accurate estimations in software development for non trivial solutions are impossible, we can only guess. Also remember that is very likely that you will find so many things which you didn’t foresee when you start developing that is worth multiplying the estimate to cover for those, I usually go with 1.5 or 2.

4.- The ego spiral.

chicken dance

The last design meeting...

Many programmers discussions look more like rooster fights than human discussions. This usually happens in design and architectural meetings. It is actually quite easy to detect this ego spirals, you just have to substitute most of what the contenders are saying with COC! COC! COCOCOOCCC! COOC!

How to fix it: Leave your ego at home. Big egos are one of the biggest non technical issues for any programmer. Keep in mind some basic considerations when making decisions.

5.- “It wasn’t me!”

In my opinion, other bad habit from most programmers is the lack of accountability. We always have an excuse… It’s like if we were saying that under normal conditions we would never make a mistake, which honestly is quite hard to believe.

How to fix it: No need to cry, or to perform seppuku, (aka harakiri), when we make a mistake. Having a healthy attitude where we can you just say something like: “yeah, sorry, now we need to do this to fix this issue, my fault” is a very professional attitude, and it will help you to build a reputation and to be better considered by your colleagues.

6.- The demotivated genius.

Repetitive and simple tasks are not the best motivators, but they need to be done, programmers tend to get demotivated and very unproductive when they need to complete them.

How to fix it: Discipline. Unfortunately, there isn’t any other remedy I can think of.

7.- The premature programmer.

If programming was sex, there would be a lot of unsatisfied computers. You can just not go in, do things halfway through and then fall sleep. One of the concepts that I find most programmers struggling with is the concept of “Done”. Remember that done means: tested (and not only unit tested), documented, committed, merged…

How to fix it: This one is tricky, the complexity of non apparent necessary tasks to fully complete some functionality is quite high and usually requires discipline and training. Probably the two easiest ways to help a programmer understand if some code is done are peer reviews and demos.


Written by Alberto Gutierrez

May 16th, 2011 at 3:07 pm

Progamming a chess engine with Java (I) – Finding where a piece can move.

without comments

A few weeks ago, I posted an article describing some Java code which would be able to solve any Sudoku using tree data structures. After this article, I thought it would be nice to continue developing around tree data structures, and that’s when I decided to code a chess engine in Java.

This is the first installment of a series of articles that will describe this chess engine as I am developing it. It will also use, the tree library that I created for the Sudoku resolver. You can find the complete source code in the link provided at the bottom of the article, any feedback on the code will be very much appreaciated.

The Chess Board class

public class ChessBoard {
	private final Squares squares;

	public ChessBoard() {
		this (new Squares());
	}

	public ChessBoard(Squares squares) {
		this.squares = squares;
	}

	public ChessBoard performMovement(Piece piece, Location locationFrom, Location locationTo) {
		Squares copy = squares.copy();
		copy.setContent(SquareContent.EMPTY_SQUARE, locationFrom.getCoordinateX(), locationFrom.getCoordinateY());
		copy.setContent(piece, locationTo.getCoordinateX(), locationTo.getCoordinateY());
		return new ChessBoard(copy);
	}

	public ChessBoard performMovement(PieceOnLocation pieceInBoard, Location locationTo) {
		return performMovement(pieceInBoard.getPiece(), pieceInBoard.getLocation(), locationTo);
	}

	public ChessBoard performMovement(Movement movement) {
		return performMovement(movement.getMovingPiece(), movement.getFrom(), movement.getTo());
	}

	public SquareContent getSquareContent(Location location) {
		return squares.getContent(location.getCoordinateX(), location.getCoordinateY());
	}

	public ChessBoard addPiece(Piece piece, Location location) {
		Squares copy = squares.copy();
		copy.setContent (piece, location.getCoordinateX(), location.getCoordinateY());
		return new ChessBoard(copy);
	}

	public List<PieceOnLocation> getPieces(Color movingColor) {
		List<PieceOnLocation> pieces = new ArrayList<PieceOnLocation>();
		for (Location location : Location.values()) {
			SquareContent content = squares.getContent(location.getCoordinateX(), location.getCoordinateY());
			if (! content.isEmpty()){
				Piece piece = (Piece) content;
				if (piece.getColor() == movingColor) pieces.add(piece.on(location));
			}
		}
		return pieces;
	}

	public List<PieceOnLocation> find(Piece pieceToFind) {
		List<PieceOnLocation> pieces = new ArrayList<PieceOnLocation>();
		for (Location location : Location.values()) {
			SquareContent content = squares.getContent(location.getCoordinateX(), location.getCoordinateY());
			if (! content.isEmpty()){
				Piece piece = (Piece) content;
				if (piece == pieceToFind) pieces.add(piece.on(location));
			}
		}
		return pieces;
	}

	public PieceOnLocation getPieceOnLocation(Location location) {
		return new PieceOnLocation(getSquareContent(location).asPiece(), location);
	}

	public ChessBoard empty(Location location) {
		Squares copy = squares.copy();
		copy.setContent(SquareContent.EMPTY_SQUARE, location.getCoordinateX(), location.getCoordinateY());
		return new ChessBoard(copy);
	}
}

About the Chess Board class is important to notice 3 things:

  1. Is thread safe. All is member variables are private and final, and is immutable, all the “changing operations” create a copy of the object with the new state.
  2. It delegates all the functionality to Squares. Squares is the main class, it contains all the logic for holding SquareContent objects (Pieces and empty squares). It is wrapped by the Board to achieve thread safeness.
  3. It is easy to create and customise a board, we only have to create a new board, and call the method addPiece. (See example below).

Creating a new board with a few pieces.

board = new ChessBoard().
			addPiece(Piece.BLACK_PAWN, Location.C5).
			addPiece(Piece.WHITE_PAWN, Location.D5).
			addPiece(Piece.WHITE_KING, Location.D4);

The Chess Analiser

This class contains the main entry points for all the main logic, in particular, we are interested in the method findReachableLocations and in the default implementation ChessAnaliserImpl.

public interface ChessAnaliser {
	[...]
	public List findReachableLocations(PieceOnLocation pieceOnLocation, ChessBoard board, Movement previousMovement);
}
public class ChessAnaliserImpl implements ChessAnaliser {
@Override
	public List<Location> findReachableLocations(PieceOnLocation pieceOnLocation, ChessBoard board, Movement previousMovement) {
		Piece piece = pieceOnLocation.getPiece();
		List<Location> toReturn = new ArrayList<Location>();

		for (MovementLine movementLine : chessReader.findMovementLines(pieceOnLocation)) {
			if (movementLine.isApplicable (board, previousMovement)){
				List<Location> potentiallyReachablePositions = movementLine.filterPotentiallyReachablePositions (pieceOnLocation.getColor(), board);
				for (Location location : potentiallyReachablePositions) {
					ChessBoard nextPossibleBoard = nextBoard(board, movementLine.getType(), pieceOnLocation, location, previousMovement);
					if (!isInCheck(nextPossibleBoard, piece.getColor())){
						toReturn.add(location);
					}
				}
			}
		}

		return toReturn;
	}
}

Take into consideration:

  1. [Line 07]. The base of any movement is the movement line. The MovementLine class defines the type of movement and the different locations for that movement type that a piece can potentially move to given a starting position. Movement lines don’t take into consideration other pieces in the board.
  2. [Line 08]. The method isApplicable, in the movement line, checks if that movement line is applicable for a given board taking into consideration the previous movement. The clearest example here would be the movement line for a pawn to do an “en passant” movement, this method will check if all the conditions for an en passant movement are true.
  3. [Line 09] The method filterPotentiallyReachablePositions checks from all the possible locations of a movement line, how many are reachable, so if there is a piece that is actually standing in the middle of the board for that movement line, it will filter out all the positions after that piece . (See examples below).
  4. [Lines 11 and 12] For all the potential locations that given piece can move to, it finds what the next board would be, and then it checks wether if the movement would cause the pieces for that color to be in check. This would be an illegal movement, so the location would be discarded.

Tying everything together

The following are real examples to find what the possible locations for the given pieces are:

public class ChessAnaliserImplIntegrationTest {
	private ChessAnaliserImpl testObj;
	private ChessReader chessReader = new ChessReaderImpl();
	private ChessBoard board;

	@BeforeMethod (groups="integration")
	public void setup (){
		testObj = new ChessAnaliserImpl (chessReader);
	}

	@Test (groups="integration")
	public void testFindReachableLocations_forRook_withSameColorObstacles (){
		//GIVEN
		board = new ChessBoard().
			addPiece(Piece.WHITE_ROOK, Location.A1).
			addPiece(Piece.WHITE_QUEEN, Location.A5).
			addPiece(Piece.WHITE_KNIGHT, Location.C1).
			addPiece(Piece.WHITE_KING, Location.C8);

		//WHEN
		List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A1), board, null);

		//THEN
		Assert.assertEquals(actualReachablePositions.size(), 4);
		Assert.assertEquals(actualReachablePositions.get(0), Location.B1);
		Assert.assertEquals(actualReachablePositions.get(1), Location.A2);
		Assert.assertEquals(actualReachablePositions.get(2), Location.A3);
		Assert.assertEquals(actualReachablePositions.get(3), Location.A4);
	}

	@Test (groups="integration")
	public void testFindReachableLocations_forRook_withDifferentColorObstacles (){
		//GIVEN
		board = new ChessBoard().
			addPiece(Piece.WHITE_ROOK, Location.A1).
			addPiece(Piece.BLACK_QUEEN, Location.A5).
			addPiece(Piece.WHITE_KNIGHT, Location.C1).
			addPiece(Piece.WHITE_KING, Location.C8);

		//WHEN
		List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A1), board, null);

		//THEN
		Assert.assertEquals(actualReachablePositions.size(), 5);
		Assert.assertEquals(actualReachablePositions.get(0), Location.B1);
		Assert.assertEquals(actualReachablePositions.get(1), Location.A2);
		Assert.assertEquals(actualReachablePositions.get(2), Location.A3);
		Assert.assertEquals(actualReachablePositions.get(3), Location.A4);
		Assert.assertEquals(actualReachablePositions.get(4), Location.A5);
	}

	@Test (groups="integration")
	public void testFindReachableLocations_forRook_whenCantMove (){
		//GIVEN
		board = new ChessBoard().
			addPiece(Piece.WHITE_KING, Location.D1).
			addPiece(Piece.WHITE_ROOK, Location.C2).
			addPiece(Piece.BLACK_QUEEN, Location.A4).
			addPiece(Piece.WHITE_KNIGHT, Location.C1);

		//WHEN
		List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.C2), board, null);

		//THEN
		Assert.assertEquals(actualReachablePositions.size(), 0);
	}

	@Test (groups="integration")
	public void testFindReachableLocations_forPawn_whenInFirstRow (){
		//GIVEN
		board = new ChessBoard().
			addPiece(Piece.BLACK_PAWN, Location.A7).
			addPiece(Piece.BLACK_KING, Location.D4);

		//WHEN
		List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A7), board, null);

		//THEN
		Assert.assertEquals(actualReachablePositions.size(), 2);
		Assert.assertEquals(actualReachablePositions.get(0), Location.A6);
		Assert.assertEquals(actualReachablePositions.get(1), Location.A5);
	}

	@Test (groups="integration")
	public void testFindReachableLocations_forPawn_enPassant (){
		//GIVEN
		board = new ChessBoard().
			addPiece(Piece.BLACK_PAWN, Location.C5).
			addPiece(Piece.WHITE_PAWN, Location.D5).
			addPiece(Piece.WHITE_KING, Location.D4);

		Movement enPassantEnabler = new Movement(Piece.BLACK_PAWN, Location.C7, Location.C5);
		//WHEN
		List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.D5), board, enPassantEnabler);

		//THEN
		Assert.assertEquals(actualReachablePositions.size(), 1);
		Assert.assertEquals(actualReachablePositions.get(0), Location.C6);
	}
}

Source code

All the source code can be found in the following SVN URL http://subversion.assembla.com/svn/making-good-software/tags/mgsChess/blog20110515/


Written by Alberto Gutierrez

April 20th, 2011 at 4:07 pm

10 characteristics of a robust application or service.

with 2 comments

Robustness is one of the most important qualities of any application, is also one of the most difficult to achieve. What follows is a list of what in my opinion are the 10 principal characteristics to be considered when designing any robust application.

1.- Transactions.

Having the right transactionality in your application is critical to guarantee the integrity of the data. No matter which framework you use to manage the transactions, is important that you consider making transactional the system calls that require ACID properties. Some key areas to look for:

  • Transaction modes.
  • Transaction frameworks and APIs.

2.- Database access.

Modern frameworks usually try to abstract the access to the underlying database of the application, while this is certainly good, is also causing an increasing lack of understanding of what is happening behind the scenes. Being able to explain what is the SQL likely to be executed by your framework, and when is going to be executed will help you to create better database access code, and to optimize the database access, specially if you don’t have a DBA in your project. Some key areas to look for:

  • SQL
  • Indexes
  • Lazy loading.
  • ORM frameworks and APIs
  • Caching

3.- Multithreading.

Multithreading is the next big thing in software development, is been there now for ages, but with the new hardware architectures going multi-core, multithreading is emerging as the new must-know skill. It is also critical to acknowledge that your code is probably already running in a multithreading platform so you need to consider it to prevent your code to fail. Some key areas to look for:

  • Stateless services
  • Unmodifiable beans
  • Race conditions
  • Locks

4.- Logging and auditing.

Logging and auditing are one of the most underrated qualities of an application, especially for server applications. Is easy to forget in development how useful a log is when the application has already been rolled into production. Some key areas to look for:

  • Consistent identifiers. Try to use the same identifier in every log message so is easy to track one thread of execution.
  • Consistent format: If you have to grep or quey your logs, you will appreciate using always the same format for your log messages.
  • Completion: It should be possible to track an entire thread of execution from the log being able to see the main events.

5.- Exception management.

An exception that gets thrown out to the user level is usually a sign of a fragile application, a proper exception handling can make you application more robust. I already wrote an article with some basic considerations regarding exceptions.

6.- Invariants.

Invariants are conditions that need to be kept across several data structures to guarantee the integrity of the logic of the application. There are several design techniques or language features, as encapsulation, generics, enums… to help keeping the invariants safe. Something so simple as changing a list of constants to an enum or encapsulating two attributes that need to change in synchronization can make a huge difference building a more robust application.

7.- Disabling/Enabling/Swapping services or modules.

Shit happens, so you better be ready, being able to disable a service, or a module, or being able to write a dummy implementation to overcome a temporary incident, accounting for this in development, may save your day.

8.- Ubiquity and low coupling.

Being able to run chunks of your application independently, and at any time, is a sign of good design, and can also help you recover for unforeseen errors.

9.- Self diagnosis.

One very valuable feature of any service call is to provide with self diagnosis on error, so the cause for the issue becomes obvious and it facilitates the search for a solution.

10.- Documentation.

Any non trivial system requires documentation. Good documentation made available to the right people can save you from many errors.


Written by Alberto Gutierrez

April 14th, 2011 at 4:14 pm

What is the mission of any software developer?

with 5 comments

What is important in an application?

In software development there are only three measurable areas that encapsulate all that is important in an application: Rentability, quality and maintainability.

Rentability
dollar
Ultimate goal of any application. Determines if the amount of time and resources spent in maintaining the application are worth.

Main responsibles: Business, sales, marketing.

Quality
bug
Measures how well an application does what is supposed to do.

Main responsibles: System administrators, DBA, developers, QA.

Maintainability
bug
Measures how long it takes and how much it cost to successfully deploy a change/new development from the change request until is available for use.

Main responsibles: Management, developers, QA.

From these metrics, is important to understand two things:

  1. Rentability is the most important metric. No rentability = failure.
  2. Quality and maintainability are only important because they have an impact in the rentability. They basically represent the cost of maintaining and fixing the application.

Some examples of these metrics would be:

  • Bank legacy applications. Very high rentability, very high quality but very bad maintainability.
  • Successful web start-ups. High rentability, low quality and good maintainability. New web-apps that manage to get a lot of attention, with a lot of bugs, but with a small core that is easy to change.

The mission

I find it quite interesting how many developers seem to ignore these three previous areas, the mission of any software developer should be: Increase the rentability of the application through the quality and maintainabilaty.

Is important to note how code is not mentioned. The code is not an end, is a mean. If what you are doing doesn’t improve the quality or the build process, then, don’t do it. This maybe interpreted as an attack against some programming techniques, in particular refactoring, but is not. Refactoring plays an important role in maintainability, as any other engineering practice, they serve a bigger purpose, and is not the code, is rentability.

Maximising the quality

Some of the main aspects of quality to look for are:

Correctness: Correctness is about expectations. A software feature is correct if it fulfils the expectations of the customer.

Traceability: Traceability is about being able to explain what happened in the application in the past and why it happened.

Supportability: Supportability is about how easy is to fix possible errors/corruptions in the application

Usability: Usability is about how good the overall user experience is on the application, is usually about performance, UI design…

Robustness: Robustness measures how gracefully the application handles edge cases.

Scalability: Scalability is the ability to handle growing amounts of work in a graceful manner

Improving the maintainability of the application

One of the main specific areas where developers can make a bigger difference is the build. Having a bad build process is one of the biggest productivity killers in software development. A build process comprehends all the stages a change goes through since a developer commits it until it reaches production, including all the QA and deploys activities. The output of a successful build is a set of new features that work as expected in production. Any build process has two main characteristics.

Reliability. The reliability of the build process is a measure on how frequently builds that were supposed to be successful turned out to have bugs, and how critical these bugs were.

Performance. Performance is a measure of how long it takes to make a build attempt.

But is worth remembering that a non reliable build is useless, no matter how fast it is.