Sunday, April 27, 2008

To hype or not to hype

I have written a few times before about SOA (service oriented architecture) and recently I get more and more exposed to consultants in suits that have SOA in all their Powerpoint slides and I start to realise that we are at the top of the hype cycle with respect to SOA and data integration. Quite often it becomes something that sounds like this: 'if you have a hammer than everything looks like a nail'.

So my advice to all people that require integration: Avoid these people and concentrate on the question: what kind of integration problem am I trying to solve!

My view is that we should sell real solutions and not label everything with the next buzzword. Note that I do not disagree with the general principles of SOA - I'm just very sceptical about overselling something that is rarely understood and which is still hard work! And the other thing is: Business people do not want to hear about SOA or any other techno jargon. They just want to see solutions that work and do not want to sink millions in mega projects with a high chance of failure.

So - consultants - let's get back to the coalface and show me an incremental integration strategy that delivers something quickly! And forget about SOA.

Labels:

Wednesday, April 16, 2008

Different types of data integration

Integration continues to be a difficult subject and this is quite often because there is so much variety in integration needs and available solutions. Therefore it is important to distinguish what kind of data integration we are trying to solve. Making an inventory of all the integration requirements and a grouping per type


  1. One record (write / update / delete) - usually for data entry of transactions or reference data
  2. Retrieve one record (read - look-up) - usually for master reference data
  3. Retrieve one record (read - transfer) - usually for a transaction, but also to be used for master data if it needs to be replicated
  4. Combining data from different sources (read / combine / potentially transform / present) - usually for reporting
  5. A bulk data transfer (retrieve / transfer - including a potential transformation) - for e.g. data warehousing purposes, i.e. to support a better performance in a different de-coupled data model
  6. A bulk data transfer (retrieve / transfer - including a potential transformation) - for analytical and modelling purposes, where the data in the target system will be subject to change
Enterprise information architecture needs to deal with all these types of integration and usually it will deal with these different types in different ways. By understanding the different types and grouping the integration requirements per type it is possible to rationalise the technology supporting the integration.

Through this it is also possible to see that certain technologies only meet the requirements of a few of the types. For example SOA type architectures are usually good for single record situations (type 1-4).

Labels:

Monday, April 14, 2008

More on integration ...

As I wrote before, one of the main aims of Data Architecture is to support the needs for integration in the organisation. There are different ways to achieve this and therefore Architects are usually in favour of Big Projects. Developments like a Common Database (with a common data model) or a Data Services layer (with a canonical model in the service layer) are the most common projects. These projects are never popular and hardly ever a success.

At the end of the day most organisation live with a lot of point-to-point connections and because they are more practical, cheaper & faster to build they may be a better solution (even though we always way that they are less maintainable). So the challenge of the architect is to balance the cheap, fast ad hoc link with a nicer layered architecture. Usually the cheap thing wins and for a good reason ...

One way of achieving a solution for this is balancing act is to define a standard for the common framework and let projects develop their 'point-to-point' as part of this standard. Through publishing the specific link as meta data (with an XML description) we will see the services layer grow!

Labels: , , , ,