Making Good Software

A blog by Alberto G (Alberto Gutierrez)

Written by Alberto Gutierrez

January 14th, 2011 at 10:53 am

How to interact with web services, databases and other integration points. 4 main considerations.

with one comment

Interacting with external resources from your application is a common practice in modern applications, especially for “enterprise like” applications.

An external resource is anything we need to interact with, which sits outside of the application runtime environment. An integration point is where our application handles the interaction with the external resource. The most common integration point types you are likely to deal with are: Web services, databases, files, ftp, external devices…

The problem with any integration point is that you don’t have any control over them, which means that they are not reliable; this requires the developer to have in mind additional considerations to successfully develop robust integration points.

What follows are, in my opinion, the 4 main considerations for software developers to have in mind when developing an integration point.

1.- Always assume that your integration point may fail.

Every single integration point may fail; keep always this mind when developing. This would be the only consideration I would highlight if I only could highlight one.

Is important to make it clear that this consideration doesn’t mean that you have to always handle exceptional errors on the integration points. For instance, let’s say you are logging to a file. A file is an integration point, so you know it may fail, in this scenario it could be fair to say: If there’s an exceptional error on logging, let the log process crash, so you don’t handle the exceptional behavior. But at least you are asking yourself the golden question “What if the integration point crashes? ”

The golden rule to deal with unexpected errors in your integration points is to flag and record them so they can be reproduced later.

2.- Cover your back.

One of the main problems when dealing with integration errors is that when they fail, they usually cause data corruption, or other major issues in your application. These issues can many times only be recovered or tracked by extracting information from your logs. That is the reason why you need good logs in production.

Logging requires additional thought in your integration points than other parts of your applications, just because they are more likely to fail. In general any integration point should be able to tell you in the logs 3 things:

• What is the message that initiated the interaction?

• Was there any unexpected error?

• The result of the interaction with the integration point.

Keep in mind that saying that the integration point should be able to log the three previous questions, doesn’t mean that it always have to log them, or that is the only required logging, that’s something that is only going to be clear once that the application is deployed and actually used. It means that you should be able to turn on or off this logging. There is very few worst things than having an integration point failing in production and not being able to turn on any logging to see what is going on.

3.- Isolate the interactions with the integration point.

As already specified, interacting with integration points is dangerous, so is good to strategize its interactions to make sure that their outcome don’t affect the normal behaviour of the application. Some considerations would be:

• Minimize the amount of interactions with integration points.

• Wrap the logic of your interactions in transactions.

• Allow for automatic retries

• Separate the logic that handles your integration point and your business logic.

4.- Do “what if..” integration testing before going to production.

Getting your integration points right is very difficult, in my opinion, this and synchronicity are the two most difficult things to get right in enterprise software development.

For this reason is important that you assume that if you don’t prove yourself wrong, your integration points are going to fail. Make sure that before you deploy into production you test as close as end to end as many as possible “what if…” scenarios as possible.