Being an Information Manager...

City Planning

2010-06-22T16:23:00.001-07:00

Actually the job as an information architect in IT is a bit of a funny one, since in information architecture it is very rare you actually design something for construction (I assume technical architects and solutions architects are closer to the construction of stuff). Information architecture is closer to a strategy & planning job. I like making the analogy to the job of a city planner; information architecture is all about knowing where we have the main traffic jams in information flows, where we are building new processing plants and creating the understanding how all these elements relate to each other. To stretch the analogy, information architecture is closest to planning urban renewal!

When we talk urban renewal, then I immediately can tell you that many of the existing methods in architecture (e.g TOGAF, Archimede, ...) are to much geared towards greenfield sites or systems that do not cover large & complex enterprises. They assume we create from scratch complete designs covering all aspects of the architecture stack, whilst in most cases we just adding bits and pieces on top of strata of diverse IT systems, that are like layers of a historic town. Information architecture is therefore more like working in an archaeological excavation, than doing planning from a blank sheet of paper. Unfortunately this is the reality in most businesses today. We have usually quite a few decades of legacy and the job of the information architect is to carefully plan structures that do not damage what we have, whilst providing improved capabilities for the future. More on my City Planning challenge in my later posts ...

New series!

2010-06-22T16:19:00.000-07:00

Has it been two years? I look at the blog and realise that time flies when having fun. In the mean time the world moves on and I pick up new learnings that I hopefully can share.
I have moved to a new job, high up in the ivory tower of a large global enterprise, looking at information architecture, enterprise wide .... and a lot is happening!

Technology again

2008-06-11T10:34:00.001-07:00

MRD management turns quickly into an overengineered cottage industry. You cannot imagine the number of components that make up a good MRD service - here to name a few: a data store supporting flexible data models that can keep track of every change in every attribute (and maintain their history), message-based integration to any other data store subscribing the with MRD content, workflow for the MRD maintenance, roll-back capability if something goes wrong in the process, attribute level security, ... etc etc etc. All great, but do we really need it, or is this becoming architecture for architecture sake? Personally I think this is MRD in the hands of technology-people and not MRD for the sake of better integration & quality. So my advice would be to start from the ground up and see what the requirements are one by one. Addressing them is usually possible in the existing systems. When the requirements for the services grow, then migrating to a generic MRD service may become an opportunity, but not as a start!

Types of Master reference data

2008-06-09T12:06:00.000-07:00

I have been pulled into some detailed discussions on Master reference data management and through this it occurred to me that there are different types of data that need a different treatment. Here are a few examples:

The first type I would like to call 'external standard lists' - this is the reference data for which we already have (international) standards (like ISO). Examples are Countries, Post codes, Units of Measure, Language codes. Characteristics: Standard is defined, source is outside the company, no debate necessary. Approach: Just identify the data type and agree to use the standard from now on. Even better is to make it centrally available as a service.
The second type is similar - but I would call them the 'internal standard lists' - things inside the company for which you like to have an agreed enumerated list - i.e. a fixed domain. Think about standard lists for security levels, status codes, ... Characteristics are: Standard is usually a lot harder to define due to debate from different stakeholders, while this debate is not really adding a lot of value. Approach: Identify the data type, propose a standard after analysis. Challenge the parties to make a case if they don't think they can comply to this standard. Make the standard lists available as a service.
Third is a type you should skip and this is the set of items that people think as common attributes, but which are actually not manageable. Think about document types (everybody has a different opinion), keywords, names of project teams (too many!), etc. It is hard to define this category, but it is usually about lists that change constantly, that do not have a clear source or owner and where there are no clear definitions (or where it is just over the top to manage it as reference data!). I see a folksonomy as a good alternative for this category.
Finally the fourth is the most important category - and that is for objects / things / people shared across the company the 'master reference objects' - this data you probably already maintain in one place or another and the trick here is to get agreement where you have the master etc. (stuff I covered before). Note that these objects can be organised in hierarchies - which in themselves can be seen as reference data (but lets leave that for now!)

Social Networks

2008-05-24T12:30:00.001-07:00

There is a lot of exitement in the industry about Social networks (already for some time). We all know by now the MySpaces & Facebooks of this world and I wonder if this is really worth the billion dollar fuzz (apart from for the advertisers).

In general we see that these networks are 'walled gardens' - a bit like AOL in its old days and although there are some announcements of opening up, I do not see the phenomena as much of a paradigm shift as the last two major changes: the surge of Google and Wikipedia. In my case Linkedin is the only place where I have registered myself, which is great for finding (former) professional contacts, but I hardly can see how they make serious money on this, or how this will drive a general revolution. A new development which is gaining more press is Ning from the former Netscape guru Andriessen (now valued at $ 600 mln). This is a place where you can create your own MySpace, but to be honest it is hardly more than a more configurable version of Yahoo Groups ...

But can we learn from this? Does it mean anything for Enterprise 2.0? You would think from my starting remarks that I would not think highly in terms of impact, but to be honest there is probably more to the Social Networking thing for Enterprises than for the general WWW ...

First of all I think that the concept of a 'walled garden' works well inside a company, since that's a natural fit. Second, there is a lot to gain from better team working. People have a tendency to take care of their own stuff ... when given the freedom. Third - I do not think we need to make a technology shift! We can use we have - it is mainly a culture change.

Probably the best place to start is exploiting Microsoft's Sharepoint, which can be set up to serve team sites, personal sites, blogs etc. Just like an in-company MySpace ... The main change is a shift in mindset from 'control' to Web 2.0 thinking. There is probably no need to regulate this, since the power of search will allow information to be found. If something is highly connected, and used widely it will score higher in the ranking - no need for taxonomies here!

What are the benefits? Probably we will see more content moving online. We may see better team work, since it will be easier to set up collaboration environments for various teams (our web masters today make life too difficult to do this) and finally doing it well will give you an opportunity to get rid of shared drives!

Bottom line - social networking will support moving away from hoarding and show the road to sharing.

Master reference data and Folksonomy

2008-05-22T10:18:00.000-07:00

Building up master reference data in terms of key objects in the company is difficult, but doable. Building up a reference data list in terms of 'information types' (e.g. document types, or other taxonomy) is even more difficult. From experience I can say it is usually a disaster (too much detail, too complex ...). Therefore the concept of a managed Folksonomy is very compelling alternative. It is a Darwinian answer to data management (let the fittest document types survive). The approach could be as follows:

Data managers can start with a first cut list of information types and other attributes available for users to pick and choose - the data managers can even do a bit of a mapping exercise to some of the documents, so at least some information is linked to the information types
Then users are invited to use the document types for their own publication processes (probably with an accellerated speed if a document type is a mandatory attribute). Allow users to choose from a picklist or to create their own if they don't know what to choose
Create statistics and remove the taxonomy entries with a low number of links - invite the users to reclassify if they have used the removed items

Through this gradually the list of types will grow and they will become fit for purpose.

Search vs Structure

2008-05-15T10:56:00.000-07:00

Search is the religion on the Internet today, with Google as its high-priest. And the story has become so convincing that IT executives in companies start to believe that we can do the same inside the walls of a company. Earlier I already wrote about a Google pilot (the Power of Search) and then I already noted that Google has some advantages, but definitely not all the answers. So before we take the plunge towards relying everything on Search we need to step back a little and think. We could make the same mistake as we did when we moved to desktop computing (which was a recipe for anarchy in terms of information management).

Funny enough the other hype - around master reference data management - is at the other extreme end of the scale. This is where we try to set up the taxonomies of the company. So it could be that the answer is in the middle: We should do 'Search' - it helps if all information is indexed and easily available ... and we should see what we can do to define some key master reference data objects, because this will provide a mechanism to provide more context to the information we try to index. Information related to master reference data will rank higher and the reference data itself will provide navigation in the mountains of results available through search. So the seamingly extreme ends of the scale actually provide building blocks for the retrieval architecture of tomorrow.

Governance

2008-05-14T12:19:00.000-07:00

A lot about Data Management or Architecture is related to governance, or in other words - related to the way we decide about our priorities. So the key role in DM and Architecture is to shift the focus to data and providing structure. This is quite a major challenge, since historically the focus in IT has been on

Functionality before data
Project delivery before reuse
Hardware before intangibles

And therefore the shift in governance will be hard since all our managers have grown in a culture where functionality, project delivery and infrastructure are king.

So how to make them change their minds? Here are a few ideas:

Organise an audit with an unsatisfactory result (think Sox, or what ever is relevant for your organisation) - you can even invent your own set of rules and call this a Management System
Show the cost of acquiring data vs the cost of applications - include all cost (man hours for handling, QC, ... etc.). The information asset has no bottom line - but through the transparency of its acquisition and handling cost it becomes clear how much value has sunk into data
Change your architecture pictures with more focus on data & reusable components and less on the core functionality of applications
Show clearly what you're planning to do with the money. Usually the application people have a simple cost picture: license cost + support = total sum and a simple result (now we have it installed with 5000 users ...), while data cost are harder to add up and cannot be linked to specific installations / usage. Tangible actions are easier to sell (then you can always slot in the less tangible later, once you have credibility)
Understand what is seen as critical
Start small and make it scalable (e.g. clean up one type of data for one department, don't plan to improve everything). if you can show in the pilot that 20% of the data was wrong and 30% of the data not retrievable, then the business will wake up (especially if it is seen as critical data)

Just a few ideas - but they are proven to help!

Master reference data - revisited

2008-05-10T12:42:00.000-07:00

I get the feeling data master reference data is another hype in the making. In my previous post I already about two titans gearing up for the big fight in this space, but apart from that I see now also all EAI vendors and of course all consultancies coming up with a Master data story ... If it gets like this in the IT industry, then get yourself ready for a bubble (and a burst after all the promises are not met after huge investments ...).

So let's get back to the original problem. Do we want to manage master reference data? In principle I would say yes. But then the next question pops up: Do we really want to have the data managed separate from its usage? And here I start to feel doubts! I think it is good to understand what is master reference data, where it resides in the different systems and boring stuff like ownership etc. But to have it completely separate ... I don't know if this is a good idea. If we would have an Information Asset Manager in the company with a clear bottom line responsibility for the data asset's health. Then yes, I get more positive, but even then I think we could do the job with just a set of good procedures and some proper interfacing.

So consultants: let's focus on what really counts, and don't try to sell us another expensive solution!

Master reference data management tools

2008-05-08T11:38:00.000-07:00

A few weeks ago I have seen demo's of the new Master data management (MDM) solutions of SAP and Microsoft. And it is interesting to make a quick comparison. Let's start with a quick description:

SAP has been trying for some time now to come up with a good offering. It has been a struggle ... Their current product is based on a catalogue type of tool, where there is a central master table (e.g. Product or Customer) - designed for fast performance, which you can relate to a number of hierarchies of classes. The tools has its origin in vertical markets (e.g. eProcurement and CRM). The tool allows data cleansing and some basic workflow for maintenance (although it does not allow iterations), but it does not really do any integration (e.g back into R/3, surprisingly this is not available out of the box). In general I see the tool as a difficult and not so useful tool, but having said this - the new release is promising a lot more, with support for more complex data models as the biggest improvement.

Microsoft has no history of data management, so therefore I treat any development in this space with interest. All I have seen is a pre-release (based on the product Stratature, which they bought some time ago) - but it seems like they've got a good strategy in mind. 1) the tool is made compatible with the other products (Sharepoint, Biztalk etc) and 2) they treat it as a piece of infrastructure, i.e. they see a set of foundation elements (generic for each type of MDM service) and on top of that they allow vertical solutions.

Could it be that Microsoft is going to outperform the big old ERP vendor when Microsoft MDM finally gets released? Personally I think this will be an interesting game, since Microsoft owns the desktop and can become the key to content management (Microsoft MDM may start to power the taxonomies of Sharepoint). While SAP owns the back-end, but if does not get its own act together on putting SAP MDM center stage with R3, then it will fail in the long run.

To hype or not to hype

2008-04-27T11:06:00.001-07:00

I have written a few times before about SOA (service oriented architecture) and recently I get more and more exposed to consultants in suits that have SOA in all their Powerpoint slides and I start to realise that we are at the top of the hype cycle with respect to SOA and data integration. Quite often it becomes something that sounds like this: 'if you have a hammer than everything looks like a nail'.

So my advice to all people that require integration: Avoid these people and concentrate on the question: what kind of integration problem am I trying to solve!

My view is that we should sell real solutions and not label everything with the next buzzword. Note that I do not disagree with the general principles of SOA - I'm just very sceptical about overselling something that is rarely understood and which is still hard work! And the other thing is: Business people do not want to hear about SOA or any other techno jargon. They just want to see solutions that work and do not want to sink millions in mega projects with a high chance of failure.

So - consultants - let's get back to the coalface and show me an incremental integration strategy that delivers something quickly! And forget about SOA.

Different types of data integration

2008-04-16T11:37:00.000-07:00

Integration continues to be a difficult subject and this is quite often because there is so much variety in integration needs and available solutions. Therefore it is important to distinguish what kind of data integration we are trying to solve. Making an inventory of all the integration requirements and a grouping per type

One record (write / update / delete) - usually for data entry of transactions or reference data
Retrieve one record (read - look-up) - usually for master reference data
Retrieve one record (read - transfer) - usually for a transaction, but also to be used for master data if it needs to be replicated
Combining data from different sources (read / combine / potentially transform / present) - usually for reporting
A bulk data transfer (retrieve / transfer - including a potential transformation) - for e.g. data warehousing purposes, i.e. to support a better performance in a different de-coupled data model
A bulk data transfer (retrieve / transfer - including a potential transformation) - for analytical and modelling purposes, where the data in the target system will be subject to change

Enterprise information architecture needs to deal with all these types of integration and usually it will deal with these different types in different ways. By understanding the different types and grouping the integration requirements per type it is possible to rationalise the technology supporting the integration.

Through this it is also possible to see that certain technologies only meet the requirements of a few of the types. For example SOA type architectures are usually good for single record situations (type 1-4).

More on integration ...

2008-04-14T11:49:00.000-07:00

As I wrote before, one of the main aims of Data Architecture is to support the needs for integration in the organisation. There are different ways to achieve this and therefore Architects are usually in favour of Big Projects. Developments like a Common Database (with a common data model) or a Data Services layer (with a canonical model in the service layer) are the most common projects. These projects are never popular and hardly ever a success.

At the end of the day most organisation live with a lot of point-to-point connections and because they are more practical, cheaper & faster to build they may be a better solution (even though we always way that they are less maintainable). So the challenge of the architect is to balance the cheap, fast ad hoc link with a nicer layered architecture. Usually the cheap thing wins and for a good reason ...

One way of achieving a solution for this is balancing act is to define a standard for the common framework and let projects develop their 'point-to-point' as part of this standard. Through publishing the specific link as meta data (with an XML description) we will see the services layer grow!

Technical trends: Virtualisation

2008-03-31T12:22:00.001-07:00

In an early article I wrote that technology usually is quite irrelevant for achieving the goals of data management, but in some cases there are trends worth noting.

One of the latest trends (or hypes) is Virtualisation - this is a very technical trend, especially in the light of something as non-technical as data management, but it is still interesting. Why? One of the key issues in data management is that users keep their data in files in their local environment. There is all sorts of ways to get a handle on this (e.g. synchronise the local drive with the network, or even better get rid of the local drives all together and introduce Sharepoint team workspaces), but for applications there is still a challenge. People want to run their apps close to their data (think performance).

By moving the applications to a virtual machine (in the data center), than the data in the data center is actually closer to the application, then if it would be on the local drive ... That's worth a thought!

Data Management & Architecture

2008-03-24T03:20:00.000-07:00

It is surprising how close Data (or Information) Management and Architecture are to each other. Where Architecture is more technical and Data Management is more about how we handle data / information in processes, there is also a lot in common. Both are strategic and are about changing behaviour, about implementing standards and rules, and both are usually seen as overhead.

Maybe there is merit in bringing Data Mgt and Architecture closer together ... Shouldn't have any large IM&T department a group for Data, Architecture & Strategy?

Big Choices [2]

2008-03-09T09:42:00.000-07:00

When making big architectural choices it helps to do the following:

define the different areas in which choices can be made (e.g. deployment platform, data & integration, processing, ...)
for each of the areas define at least two scenarios (preferably these scenarios are black & white, like thin client vs thick client).
describe the attributes of each scenario. When doing this you will see that there are usually a lot of dependencies in the scenarios. And some attributes do not really matter (they are a given, so no choice!).
describe the impact (positive, negative) of each scenario, in terms of costs, but also in terms of impact on users, etc.
and then make your choice ...

And finally: Do not make a Big Choice if you cannot follow up the choice with a clear impact on investment.

Big Choices & Investment Proposals

2008-03-03T11:07:00.000-08:00

So architecture can be descriptive (pretty pictures) or prescriptive (rules) and although I think that the pictures are important, the architecture will only start to work when the rules are being applied.

So if architecture is prescriptive (see previous post), then the foundation of architecture is making choices, Big Choices. Of course making the choice is not good enough - it needs to be based on a good rationale. So when the architecture prescribes to use e.g. Linux as the platform for the Oracle database, then the choice needs to be based on a number of good reasons (e.g. the support for grid computing, cost, etc). The good reasons usually can be derived from the principles - as described in an earlier post.

Once a choice is made it should also be supported by sufficient investment. You cannot make an architectural choice for say Linux, without providing the means to support that platform. With other words - architecture cannot live in isolation. It cannot describe a desired future state, without influence on the means to achieve it. So bottom line: Don't start doing architecture if you do not think you can influence the investments direction with your Big Choices!

Unity & integration

2008-02-27T12:39:00.000-08:00

Last week I went to a presentation by Jan Hoogervorst, an ex-VP for IT strategy at a major airline. It was the first time I've seen some real practical usage of enterprise architecture. Jan's definition was quite narrow - to him architecture is mostly presciptive (not descriptive - as in pretty pictures) and aims at unity & integration in the portfolio, with the focus on risk areas. How straight forward! Personally I like the pictures as well (a picture says more than a thousand words ...), but his message is clear. Just show the rules of the game and then you're in business as an architect!

Workflow

2008-02-21T12:03:00.000-08:00

Supporting clever workflows is one of these holy grails in IT. And it seems that we are not able to crack this very easily. We have of course at one side of the spectrum the very retrictive transaction processing (I do the payment, my boss approves ...), and here we have made quite some progress (nothing difficult here). But the other side of the spectrum is where it gets more interesting.

More complex is trying to get a handle on the more diffuse ways of working, medical protocols, research, ... In the 1980's we have seen attempts at expert systems, but they never really became mainstream. And also now, 20 years later, all I have seen to date is very basic ... Of course we do very clever stuff with demonstrator robots from Sony (and others) that recognize patterns and know how to react on this (this is a step forward), but in the office domain I am still looking for a breakthrough.

Why is this? Probably because a lot of what we humans do is not easily replicated. Workflows like the analysis of a data set, or writing a report on a difficult subject, or developing an idea are still very fuzzy and hard to define. The only thing we have today for this part is the more control oriented workflow, where we check that we have taken the right steps (i.e. protocols, like used by doctors), or that we have documented what we have defined is mandatory (i.e. tracking of mandatory deliverables related to decision gates). The first one is not main stream yet (so I still expect more developments in this space) and the second one is so basic, that I hardly can call this workflow. So we have to settle for the simpler higher level type of workflow - and still wait for the breakthrough!

Collaboration tools

2008-02-18T10:54:00.000-08:00

As information architects we have a challenge when dealing with the large number of tools emerging the last decade in the space of knowledge management. The worst area of KM is related to the many types of collaboration tools - where for the sake of simplicity I would like to exclude the ones related to direct communication (like e-mail and instant messaging).

I think in short there are roughly three types of collaboration tools; 1) common interest networks (like the old news groups, and newer hypes like MySpace, etc.), 2) collaborative bookmarking & tagging (think del.cio.us) and 3) collaborative authoring (think Sharepoint in the Office area, but this also includes wiki's and blogs).

So when your KM architecture requires collaboration, then define what types of collaboration are required and how they relate. My view is that especially the collaborative authoring world that emerged holds the biggest potential for improving the way we manage knowledge. It moves the focus from approvals to content creation and that's what it should be all about. So when rethinking your collaboration space - think wiki's first.

Knowledge management

2008-02-17T10:50:00.000-08:00

As soon as you mention the word 'Knowledge management' people will start to become uncomfortable. Maybe they will grin (and stop taking you seriously), maybe they will try to move to a different subject, or even worse, maybe they will have an opionion about document management.

Just as I wrote on 'architecture' I think the world of KM is suffering from a high degree of immaturity and because of this we see a vast number of methods on offer.

What we need is a reference model, a model which describes all aspects of KM. Just try and read the entry in Wikipedia and you will see how much people have struggled with some definitions (it received more than 500 edits last year, so no wonder ...). Shouldn't we just make it more simple? My view is that we have three types of things in KM - that are all very different: 1) Different types of Collaborative Content creation (formal / informal), 2) Systems that help you through workflows (protocols, expert systems) and 3) stores with Knowledge in the form of reference information (like e.g. Wikipedia).

Around these three families you will find lots of tools & flavours. And also a lot of hype (like all web2.0 stuff). So if you need elements of KM in your architecture, you will definitely get lost! But if you just focus on the three elements (collaboration, workflow protocols and knowledge reference) then maybe you have a chance ...

Information Asset Manager

2008-02-16T11:03:00.000-08:00

Nowadays de CIO of a company is usually an IT person taking care of buying desktops, the contract for the network, some application development, etc. In short - the CIO is mainly in charge of the T of technology, and therefore in most IT policy the T is more important than the I of information.

My vision is that in the future this will be moving to become an Information Asset Manager. Gradually we are seeing the commoditisation of desktop support, networks, and even lots of applications (to me SAP R3 is also a commodity ...). Hopefully this will give us more room to focus on what really counts - out assets, our information assets.

Probably this will require a new generation of managers, since the current generation at the top (and below) has all been grown and rewarded in a technology culture. Therefore we should be hiring a different type of people today to become the managers of tomorrow. Else we will nog see a culture change (and therefore no step change in the way we manage information!)

Application Services

2008-02-09T06:11:00.000-08:00

Probably the big paradigm shift currently happening in the industry is the move to 'Services' hosted by 3rd parties. Who wrote some years ago already that the 'network is the computer'? (well, it was one of Microsofts rivals - SUN). This vision is becoming a reality now. Why have Word and Outlook installed, if their online sisters are more reliable, cheaper, ...

Right now Google apps is growing to become a serious competitor of Microsoft, and even Amazon is entering this market with offering storage services ... Soon we have SAP moving to a service model (they are already working on this for some years) and soon after this others will follow.

This trend will give new opportunities and challenges to the industry. The opportunity is that we get the option to focus more on what really matters for the company (because services like storage, word processing, accounting are becoming commidities, just like electric power ...). The other challenge is that we need to rethink the way we manage our networks. At the end of the day the only asset that stays is 'our' information (the data and documents) that need to be managed, protected, etc. Opening up to market services changes this model.

I see it mostly as an opportunity, since it will put information management firm at the center of what we have to manage!

Immature methodologies [2]

2008-02-07T10:25:00.000-08:00

As I wrote earlier - the architecture methodologies are not mature. So what do we need? I think we need a simple protocol (with examples) for making inventories in the different stages of architecture development. For example:

stage 1, list main drivers & objects, then create an architecture vision based on how architecture can contribute to achieving the main objectives
stage 2, create a context diagram and list the core functions and related business services, assesss for each function or interface if it is an issue / improvement area and then prioritise, based on the key objectives from stage 1
stage 3, list key functions & key data types, create a mapping matrix between the two, and check where the identified key issues of stage 2 should be address (e.g. through rationalisation of the architecture) ...
etc.

My view is that a good architecture method should provide a simple cookbook - step by step - with some minimum standards for a few diagrams and decisions trees. Most methods I have seen try to do this, but go overboard with too much detail, too many different types of diagrams and a clear spill over into the system design space.

Immature methodologies

2008-01-25T11:21:00.000-08:00

Togaf, Dynamic Architecture, etc. lots of methods exist to define your ' Enterprise Architecture'. The existance of this plethora of methods is a sign of the immaturity of our discipline. We spend lots of efforts in coming up with even more ideas on how to do things right, but I guess we are missing a point here, since the more methods we invent, the more confusion we create. And the more we realise that we actually do not know ...

The other immaturity is that whatever we invent seems to be too complex for implementation in large organisations. With my current experience I would almost recommend to skip the details of any method and focus on the big picture - and here we see a lot of commonality between the methods.

The other thing is that most methods seem to be focused on a relatively confined scope (a single business proces) and in these cases it is quite often overkill to use it (just keep it simple and use the tried and tested methods for system design and don't sex it up to call this 'architecture') .

Re-coupling

2008-01-21T13:23:00.001-08:00

Architecture is a lot about de-coupling - this is trying to ringfence functionality in order to manage complexity, allow specialisation and allow improvements through 'service management'. This trend has happened for many years in the industry, but now a new trend is emerging: re-coupling. Some examples:

travellers now book their own flights (and not via the travel agent - the old point of de-coupling)

bank customers do their own data entry (and not the back office in the bank)

specialist processing activities can be carried out by non-specialists

This trend is interesting, since in information management we see that the de-coupling of e.g. data related activities has quite often not happened and now we see more information related activities moving back to the user again.

Is this a good trend? I think it is not in the case of running a business. If we can move certain information related activities to information handling professionals (e.g. data receipt, basic QC, cataloguing, help with retrieval services, etc.) then we will make the more expensive engineers more effective. So bottom line - before we start re-coupling (which seems to be more a consumer trend), let's focus on de-coupling of services related to information management.

Fighting perceptions, through certification?

2008-01-20T03:15:00.000-08:00

Recently I read an interesting letter in a dutch newspaper - which told me a lot about perceptions people have about IT. The industry was accused of being 'technology pushers', with lots of promises, no realism and through that responsible for the many well-published failures. The letter ended with the idea to claim billions of dollars from the IT industry for frauds like the Millenium bug.

The letter was a reaction on an article about large government IT projects - why they fail. The article itself was pretty clear; using IT to push change, with ill defined scopes, ... etc were all to blame - and I fully agree with this. And as you may understand from my blog - I am convinced that more 'architecture' and more focus on the softer side in project in these type of project may help (so technology becomes less of an issue as well).

But I think there is more. Why is it that still many people see IT as a big scapegoat for everything that goes wrong? Why do people underestimate the complexity of large change projects (and think it is an IT problem)? Maybe it is true (in a way) that some of our professionals do push technology before looking for real solutions - and that's why I have advocated to but more 'I' in IT. But how do we achieve this?

Maybe as an industry we should have a 'code of practice' that prevents us from doing projects that are un-achievable? Maybe we should only do projects when there is sufficient resourcing & the right approach! I guess that's all wishfull thinking. As an industry we have already tried to set up all sorts of certification scheme's - but the bad news is that certification does not always help! CMMi level 5 means first and foremost a lot of bureaucracy - and not necessarily that you get the right solution for a clever designer for the right price. PMI certification does not mean that the project manager is a really good communicator (which is 50% of project management!). Now I have also seen architecture certificates popping up and even stuff related to data and document management ...

As I wrote earlier - IM is an Art, in some ways, and in many ways it is closer to social science than it is to technical & engineering. In the same way Architecture is an art as well - but an art with a purpose. Just like Norman Foster creates beautiful functional buildings - we should recognise the great IM & IT architects in terms of their artistics skills beyond the certificates. And we should make clear that these enablers for real change cost more than the bog standard VB programmer (so when we source project that we do not always look at price).

I only don't know how we can define this quality label, if certification is not the answer. Can we learn from the academic world? (citation index, peer reviews, ...) I don't know - but it is a thought.

Process Architecture [2]

2008-01-15T10:29:00.000-08:00

Earlier I wrote about Process Architecture and how we usually have a bias towards functionality. Re-reading this I must admit that I stepped over the fact that process analysis is still very useful. Understanding the main functions in the organisation and understanding if they can be run as 'services' is a useful exercise. Services are organisational functions which can be ringfenced and managed through well defined interfaces. The interfaces usually have the nature of a Service Level Agreement. Defining services also allows de-coupling and usually it is data which goes over the interface - and there we see a purpose for data management again.

qaotiq.com

2008-01-09T12:04:00.000-08:00

I have been writing over the last few months a few times about the power of wiki's and the importance of reference data and now I would like to combine & test these ideas. Therefore I have created a new site - qaotiq.com. A soft go live of the site was yesterday (there are still some bugs ...).

The idea is that it is possible to create your own lists of reference data and share these with others. The items in the lists can link to e.g. wikipedia or sites like Flickr or Google maps.

Eventually over time I expect a data model to emerge of all sorts of linked lists ... Let's see if it works - you're invited to join!

Architecture and Information Services

2008-01-02T12:04:00.001-08:00

I strongly believe that most organisations today are mostly dealing in information and therefore - building a layer of 'information services' is something that should be considered when trying to build a sustainable and agile architecture.

What is the story of information services? Information services are a way of decoupling data from functionality. In the past this was already done by creating databases - but usually the databases were application specific. In the mind of most programmers there cannot be data without an application.

When applications started to grow and started overlapping, then the concept of shared data emerged - this happened at the end of the 'eighties' and suddenly people started to think about designing major data models. One of the most glorious attempts was the 'Epicentre' datamodel - designed by the Petrotechnical Open Software Corporation, an object relational database for the oil industry with more than 2000 tables. It was very clever, very flexible and very hard to implement. And therefore it was never a success.

Obviously these heroic attempts were doomed and with the growth of off-the-shelf applications we saw the rise of ETL tools (to get a grip on all the interfaces) And quickly after the rise of ETL, with the rise of the Internet, we saw a complete market growing of middleware solutions.
As information managers found out - it is also possible to create spagghetti with middleware solutions as well and now eight years after the millenium we are back at where we started - we are crying out to get more control on the information.

And this is where the information services story comes back again. If you put information at the center and 'wrap this up' as a service, using a canonical data model (a bit like the Epicentre data model, but then more pragmatic ...), then it is possible to reduce the complexity of your architecture. The data can be in various places (so don't think about large corporate data stores) and the middleware just 'knows' where to get it. The applications just 'shop' for data from the middle layer and the 'bus' is doing the work. Well this is where the complexity is - building the 'bus' is the hard work - since it requires

rationalisation of data sources (agree where is the master)
common definitions (or at least agree how to transform)
routing (agree how the data flows)

And this is where we in IT are not good at - it is great to define a lot of new functionality, but to crawl through the sewers of data management is very hard. But if you want to succeed in getting control on your information centric entreprise, then think 'information services'!

Process Architecture

2007-12-19T10:19:00.000-08:00

One of the things that usually surprise when I have to deal with McKinsey types is that their focus on business is very process oriented. Activities flow from step to step, from unit to unit, but where is the data?? By not addressing the data dimension in organisations I see that they usually leave stop-gap measures to business issues, since only part of the solution is addressed.

This is why Enterprise Architecture (EA) methodologies are relevant for data management as well - EA addresses process, applications, data and infrastructure and treat them all as equal ..., but because most consultants have a enormous focus on functionality or technology it is important to give data an even higher profile during an EA exercise. So Data Managers - use your architects to get data on the map!

Keep it simple

2007-12-17T11:05:00.000-08:00

I noticed that I wandered off quite a bit over the last few months - from the original trail about data management, towards more architecture oriented notes. Probably because I started noticing that architecture can actually add a lot of value to the cause of information management. An architect can do a lot of things - add principles, look at reusable services, create an overview - look at more than just the requirement of that moment by that small group - etc. But I think the key role of the architect is to make things simple.

The synthesis of the analysis over various systems should be something that is simple to sell, because it is simple to understand. I have now seen people trying to sell a concept like an Enterprise Service Bus (ESB) - and since they could not explain what it was they failed in their message. If their message would have been about integration and agility, then it would have made a bit more sense. If the message would also have shown a stepwise approach towards a full blown solution, then maybe people would have bought it ...

So as an architect - the key message is the good old KISS principle - Keep it simple, stupid!

Architecture maturity

2007-12-10T10:28:00.000-08:00

Recently I came across a book from consultants from Sogeti. The book describes how to set up an architecture practice.

The most valuable bit in the book is a matrix about architecture maturity. The nice thing about this matrix is that it helps positioning where your organisation is in terms of maturity and what the next things are you need to do to get higher up the scale. It also gives the message that architecture development needs to be a balanced program with activities in many areas - so investing in e.g. just skills, or just organisation is not enough.

An eye opener for instance is that tools for describing meta data only work in mature architectural set-ups, so therefore investing in it at the start is basically wasted money. So bottom line - this is recommended reading if you start thinking about setting up an architecture organisation or program.

More on principles ...

2007-12-07T13:02:00.001-08:00

Earlier I wrote about data management principles and data architecture principles, and I stayed with these principles in the comfortable realm of IM&T - i.e. the area where we have more or less control over what we do. The thing I learned recently is that there are also 'higher' level principles that may be of value to drive the 'lower' level principles on data & architecture that I mentioned earlier.

Let me call these higher level principles the Business Principles. These are things based on where our business context drives us and what we want as a company. They are basically the things that link the data architecture to the business.

Usually we get a bit vague when we talk business drivers and objectives. But when doing architecture, they are vital for defining direction. For example - say your business is about differentiating technically. In that case your IT architecture should support this and a complete focus on e.g. operational efficiency and cost is probably the wrong focus. Or say your business is about reacting fast to market changes - in that case you cannot afford projects running for 4 years to deliver large solutions, but you should in fast in a platform that allows for agility and change.

Defining these business principles is a key step towards defining a good architecture, and therefore a necessary step, which is quite often forgotten (especially by IT managers with a focus on costs)

Six Sigma

2007-11-28T10:46:00.000-08:00

If you talk to Japanese manufacturing companies (say Toyota or Toshiba) and you talk data quality, then soon you'll talk 'Six Sigma'. Six sigma is a term derived from the statistical term describing the standard deviation (sigma) and therefore describes the likelyhood to make an error. If you reach Six Sigma, you reach a quality where you hardly make any mistakes. This is particularly important in manufacturing.

When talking Six Sigma I think it is mostly striking to see that the quality process is everywhere; it is engrained in the organisation -from workers up to sr managers - quality is everywhere. And that is - I think - the main message of Six Sigma: You can only reach high levels of quality (and therefore also information quality) if this is engrained in everything we do.

Some vendors of data quality tools start to use the term Six Sigma in their toolkit and that makes me sceptical, since it is not the tool that does the job, but the complete culture change. So if you see managers buying a Six Sigma tool and not do anything else - than don't trust them!

Making Controls Work

2007-11-25T05:43:00.000-08:00

Data architects need to balance constantly between what we want in terms of easy to use functionality and building in some level of control. Users do not want to be bothered with storing data in the right place, or adding all sorts of meta data, but data managers know that without this it is not easy to re-use the data elsewhere, or later during the life cycle of the data.

So one of the holy grails for data architects is to find that right balance between easy functionality and the controls. My view is that we need to focus with controls on the 'must-have' and further see how much we can already populate through some smart algorithms. At the end of the day we all know that if things are automated they will happen (like the a search index), and when you rely on the user it is more likely that meta data stays incomplete and inconsistent, unless there is a lot of control. The right balance is even easier to find if some of the user interaction can be supported by work done by automated processes - e.g. if the meta data can be pre-populated, and users only need to click OK (and take out the glaring mistakes), then this process may work.

Another option is to make the data quality more visible; more transparent retrieval mechanisms (like e.g. Spotfire or BusinessObjects) are great enablers for this. Suddenly users see that data is missing and may even grasp the effect of this.

Integration is the Key

2007-11-22T10:38:00.000-08:00

Things like data management and architecture are quite often still elusive subjects for senior managers. We produce pretty pictures, we make comments that are hard to disagree with, but when people ask us the 'so what' question we usually have a complex story about roles and responsibilities, compliance, implementation of difficult 4-dimensional data models and other intangible things. What we usually miss is the elevator speech that sells our story.

I have been reading through previous posts and I see a lot of things the fall in the first category I mentioned - the difficult & intangible overhead stuff. Therefore it is time now to focus on the one-liners ... I even think we can reduce it to a single word: The word Integration. Probably the main reason why we do data management is because we want to integrate something:

Integrate business process flows
Integrate information from various processes into a single management view
Integrate information over time - so we can compare today with yesterday
Integrate the inside with the outside
Integrate data with documents, maps, ...

OK - there's a few other things - risk management, compliance, etc. (pretty important stuff), but they key thing that people understand is the word integration. So probably in the future I am going to drop the word data management in discussions. When people ask why when I propose a certain measure, than I will say: Because you need to integrate ...

Location, location, location

2007-11-19T12:12:00.000-08:00

Another aspect of master reference data is its spatial representation. Spatial information becomes more and more important (think Google Earth) and therefore the proper management of location, shape, orientation, etc. becomes very important. Using a spatial location is a very intuitive way of finding information, and after all the freebies from Google Earth and the upsurge of TomTom type of devices the spatial information is now firmly on our 'map' of data management.
A starting point is to have a standard coordinate reference system (CRS) agreed in the company. Each country (or in some cases even regions) may have a different CRS and therefore it is important to choose a common standard. The most common are WGS84 or UTM. Different coordinate systems for the same point can lead to a mismatch of 100's of meters.
The second thing is related to the 'spatialisation' process. Usually the spatial representation of an object is stored in spatially enable Oracle database, but quite often this is a different data base than where the original source of the data is. In a way you can compare the spatial information with an index - it helps you to find the object in a defined space. But when the spatialisation is not standardised, or not owned by anybody, than there will be quality issues with the data very soon.
Having a spatial index in place centrally is useful, but if you have a large environment this may not be practical. Too much centralisation can lead to security complexities, synchronisation issues and preformance problems. The best thing is to have a strategy to store the spatial information together with the master reference data, whilst using a standardised spatialisation service. Than you can re-use the security of the master data source and render the information anyway you like - in a table, on a graph and also on a map!

Master reference data management

2007-11-11T12:37:00.000-08:00

Implementing a service oriented architecture (SOA) also leads to the move to set up master reference data management services.

I am sure that there are no real best practices around at the moment, so therefore this is a bit like exploring undiscovered territory. The big step is that you start to de-couple data from functionality. With other words - the management of data becomes a business process in itself.

So how do we get this done? Some people start straight away with developing the best possible data model, but I think that this is not a good idea. Don't develop a data model that would meet everybody's requirements from the start, but follow the steps, as laid out earlier in ... http://infomgr.blogspot.com/2007/06/data-management-fundamentals.html and the best is to start small. So in short do the following:

Pin down the scope & keep this as small as possible to start
Define the ownership, including the operational responsibilitie
Define the process / data flow, including the operational roles
Set up the data model & keep this as flexible
Finally - set up the 'service'

The service itself should contain the following:

The canonical data model + optionally the container for the data (and keep the scope limited to start!)
A messaging service; to transport the ins/upd/del transactions
A routing service; to move the data to the right place
A key mapping service; to know how to translate keys, if required
And optionally a user interface for power users

This all looks a bit daunting, but by starting small it is a feasible feat. Let's see if I can convince my colleagues of this wisdom as well!

Canonical Data Model

2007-11-09T10:44:00.000-08:00

The main challenge continues to be the data model. In SOA speak: the canonical data model. Through some sort of difficult reason it is very hard for most IT people to deal with data models. Obviously they are not so sexy as the functionality in the application. Unfortunately it still requires quite a bit of skill to develop one ... and expecially an intermediate data model needs to fit a lot of requirements. It needs to...

Support the superset of requirements of the related systems - and therefore it is important to model cardinalities, identifiers, attributes, etc. in the most flexible way. Cardinalities need to be flexible (by default many to many). Identifiers need to be unique (by default meaningless numbers). And with every attribute it needs to be clear that this is an attribute and not potentially an entity ...
Support the reference data over time (4D), i.e. everything needs to be time stamped and no reference data gets deleted ...

If developers want to meet all these requirements, than they need to dust off some skills we dropped during the nineties.

Enterprise Service Bus

2007-11-04T10:27:00.000-08:00

The flavour of today in integration architecture is the Enterprise Service Bus (ESB), which is in a way nothing new. The idea of integration via an intermediate model has been around for some time and has led to many disappointments. But maybe this time there is a chance to make it work, since technology has moved on.

So what's new? First of all XML has matured to some sort of lingua franca; gradually all players in the market start to support the paradigms of SOAP, XML schema's, message based services, etc. And that means that momentum is building up. Second - the main difference is that everything is loosely coupled and therefore the responsibility for integration is now firmly put at where it belongs - with the applications. With other words - it is not the middle layer that needs to take care of everything, and this may be a factor determining its success.

Incremental Architecture

2007-11-02T12:34:00.000-07:00

Implementing Enterprise Architecture is not an insignificant task and usually when carried out as a separate activity it has a high likelihood of failure. This due to its size and its lack of clear owners. Owners are usually more interested in more tangible pieces of functionality and less in the plumbing.

Luckily there is also another option and that's called incremental architecture. It works as follows:

Develop a high level of understanding of the direction for the portfolio / enterprise (so you cannot work without an overarching blueprint, but the ambition-level can vary

Map the existing program to this high level picture - and you will see that projects overlap with elements that need to be developed to make the blueprint or vision a reality

Expand the scope of the existing planned projects and include the common elements needed in the future architecture as deliverables to ...

This gives the opportunity to develop the integration gradually and has a higher likelihood for success, since it is part of the existing ongoing program. The only downside is that it is very hard to achieve a real step change ...

Architecture reviews

2007-10-22T11:54:00.000-07:00

If your company has a mature project management process, than this process includes a regular review (decision gates). It is very useful to ride on the back of this process when establishing architecture. If an architecture review, based on some clear criteria is part of these decision points, then it is likely there is more attention for architecture.

The other upside is that the architects will have a higher chance of getting involved in the projects - know what is going on - know what works and is required - and do some marketing for the architecture related ideas.

Gap analysis

2007-10-21T12:43:00.000-07:00

Data management can be a bit daunting sometimes (there is so much to do, so little time, and nobody really supports it). So using an architectural approach can be counter productive (too many issues, too many fluffy slides) and turn people off in their drive for improvement.

In this case it can be more useful to have an approach that has the focus on addressing the major gaps and clear deliverables. This approach can make use of the architecture patterns, which I posted earlier. The patterns can identify very quickly components that are missing. Say you have a transaction environment and no business intelligence solution - than this is quite likely a gap (etc), so something that may be a pain point that needs addressing.

If this is still too complex (the patterns can be complex to understand, because people mix up physical data stores with 'roles'), then the last resort is to use a simple set of interviews to create a heat map of where people experience pain with their IT systems. This pain is usually caused by the usual data management problems (no ownership, no clear rules & enforcement for quality, spaghetti integration, no standards, etc.). So the trick is to link the pain from the interviews to some root causes and then create a simple staircase diagram addressing the key gaps in a logical sequence. If you cannot define these clear deliverables that link to existing projects or programs, then it is very likely you will fail with any architectural attempt.

Data Management in de Project Lifecycle

2007-10-11T13:19:00.000-07:00

Project managers give me a surprise on a regular basis. I know that the main aim of their work is to deliver what was agreed, but that does not dismiss them from responsibility for the quality of the deliverable.
It may be a bit of a rough remark, but most project managers have a 'tunnel vision'. The project needs to be delivered, on time, on budget. But this can be bad for data management

So what do I recommend to project managers before they begin?

Do a lot of 'front-loading' - i.e. before the project begins it is important that the DM deliverables are defined and that the planning takes care of delivering them during the solution architecture phase (the high level design) and beyond. If you miss them at an early stage - it is hard to recover ...
Ensure that you as a project manager also understand the project in its 'data context'. This includes understanding the related other systems, but also responsibilities like data ownership
Agree resources for taking care of the data management element. Data architects, or data analysts are useful resources

And then during the project it is a matter of ensure that the scope - agreed during the front-loading - is delivered!

Being everything to everybody

2007-10-08T10:35:00.000-07:00

One of the regular mistakes I see on an almost day-to-day basis is the aim of many systems to be everything to everybody. A user always wants more and that's a nice trigger for vendors to let their systems grow.

Usually packages originate from a single functional requirements and over time more and more gets added to the scope. In the background there is also this drive related to 'if you have a hammer, than everything looks like a nail' (that's also why SAP systems grow beyond what they were intended for ...).

One of the key roles of an architect is to keep the scope of systems close to what they were intended to and avoid the scope-creep causing overlapping functionalities. It is not always a popular message, but it avoids a lot of spagghetti and wildgrowth. At the same time it is important to be not too harsh - sometimes new paradigms can flower under the wings of something else and can suddenly create a lot of value. Just like the Internet was never designed for what it's used for today ...

So how to keep this balance? I guess it is at the end of the day all a matter of judgement - some flexibility at one side (let the flowers bloom) and some rules at the other (start pruning when needed). There are no simple answers in architecture!

Data management and Web2.0

2007-10-03T11:38:00.000-07:00

After a few months of data architecture I notice that my original thoughts of using the paradigms of Web 2.0 have moved slowly a bit more to the background. Still I have to remind myself how we can move away from 'old' centralistic thinking to enabling the 'new' collaborative content creation model.

I think there are several roles for data management in enabling this transaction, but the more I think about it, the more I know that this is not a simple journey.

First of all I think that data management can play a major role in establishing a services oriented architecture that will move the complexity of data storage and data integration towards the background. If the provision of quality data becomes like infrastructure, than the usage will become more natural and will enable easier collaboration. But this requires a lot of work related to moving the complexity to the background and establishing good old data mgt processes for creating proper quality data in the right master source (so not really webby2.0)

Second - I need to fight for opening up the management of Meta data to the 'masses'. Why not publish this as a wiki? I have already started piloting this with our main data management principles (and soon the architecture principles should follow as well) and for me it already starts to work. A wiki is relatively easy to maintain, it can link to anything and it is very accessible. On the first point (maintenance) I see one downside - it is not easy to automate. Some structured content is just easier maintained in a database. Maybe this is an idea for a further development of the wiki (a wikidb based on MySQL, where structure content like standard picklists can be managed). On the second point (linking) I only see upsides - a lot of content I do not have to invent myself - it is already there! and this also helps the accessibility. Many anti-wiki people fear for the lack of 'control' of a wiki, but I see that the audit trail feature and the 'official' nature of the publishing mechanism helps to keep the content professional.

Another thing worth pushing is the use of Blogging and Indexing in environments where data is interpreted. Should we really keep an audit trail of everything if users can keep a Blog of what they've done and Indexing provides the basis of search and ranking mechanisms towards the right content? Should we introduce Search on top of structure data?

Finally I think that the idea of collaboration environments are a good principle for many data management paradigms. Maybe not for transaction processing (that's all about control), but for interpretation environments, research & development contexts and engineering it makes sense to organise information more related to workgroups and less to 'corporate control'.

Architecture in different dimensions

2007-09-30T03:56:00.000-07:00

One of the mistakes I see on a regular basis is that designers try to fit too many different dimensions in one architectural design. Therefore in this post I would like to plead for a few standard views that describe the architecture in different dimensions.

In my view I would plead for a high level data flow as the basis of any data architecture. In this flow you can see the main data stores and interfaces. The data stores should fit in the data architecture patterns described a few weeks ago. The data flow can be divided in the main layers for e.g. corporate / master data, project data / data marts, archives, etc.

On top of this view you can have multiple solution architecture views for the technology stack related to a solution. This should include the main data stores at the bottom (in line with the main high level data flow), middleware in the middle and applications + viewers at the top.

Providing these different views will help the discussions on data flows (e.g. where is the master?) and the actual technical implementation of a particular solution.

De-coupling data from its usage

2007-09-19T10:59:00.000-07:00

One of the trends in integration architecture at the moment is the drive for a Service Oriented Architecture (SOA). For years we have been trying to rationalise applications and databases and time and time again we realised that with every migration we had to migrate a lot other stuff as well (like the increasing number of interfaces). Moving one application to something else would also lead to moving a lot of interfaces, reports, dashboards, you name it.

So now the idea is to hide all the complexity behind an abstraction layer. Why not publish all information via Web services, so everything below this becomes easy to replace with something else? 'All' we need to do is to describe the way want to see the information. The best way to do this is via defining a flexible data model (through an XML schema) and this is where the new trouble starts, because haven't we tried to do this before at the database level with enterprise data models? The good news is that this time we have more chance to succeed, since the middleware layer is much more flexible, since the data is not stored in the format of the schema, it is just described that way. And when needed you may even have more than one schema decribing the same information, without the need to store the data in a different way.

It looks promising, but we have a long way to go, since we only have started ...

Architecture Patterns

2007-09-17T13:12:00.000-07:00

Data architecture is not easy if the nature of the data stores in the architecture is not understood. Usually it is good to split stores with different functionality into different architectural elements; and quite often this choice is driven by the nature of the data (transactions vs reference data). A few examples of rules are below ...

Spit reference data management from managing transactions

Split transaction data stores from business intelligence solutions like warehouses and data marts

Split project data stores (with data that can be changed and reinterpreted) from corporate data stores where final data resides

Split current data from archived data

Looking at these elements it is possible to start seeing patterns; e.g. the pattern around a transaction data store where the master reference data is managed separately, the BI is arranged through data marts and retention is arranged in an archive store. Creating these patterns for specific business situations helps with understanding if proposed solutions are complete, or if designers plan to cover requirements with the wrong type of store (if you have a hammer, everything looks like a nail).

Transaction data versus reference data

2007-09-06T13:05:00.001-07:00

One of the key steps in data architecture is the understanding of the difference between Transaction data and Reference data.

Transaction data is about events, actions, state changes and are usually described with verbs. Think work management, thinking paying the bills.

Reference data is about things, physical or imaginary and are usually described with nouns. Think assets, cost centers, people.

When 'architecting' solutions we see that transaction data is treated differently than reference data.

Transactions usually form a flow of events and are therefore characterising the change through the lifecycle of a business process. Further transactions can be summarised, aggregated and become management information, or archived when not longer needed. A typical transaction environment is SAP

Reference data usually relate to one or more transactions and can be same all over the business process (although transactions may change the attributes of a reference object). Reference data can be organised in hierarchies and then the higher levels of the hierarchies are the aggregation levels for the transaction data. Reference data shared across a number of functions can be called master reference data. In order to keep this data consistent it is important to share it from one place (enter only once, use many times)

This all seems pretty obvious, but the experience I have is that people mix up the two types and do not understand that there are different rules governing the management of systems with this data.

Data Management as a Process

2007-09-05T11:00:00.000-07:00

It is good to re-iterate that DM is a process. Quite often business users think that a one off exercise in cleaning up the data helps them to get things right ...

I would like to make the comparison with brushing your teeth - if you leave it for a day it does not become an issue, but if you leave it for a while it will start to hurt. Managing data is like any maintenance activity; it is recurring and people do not like it.
So plan the right resources, schedule the activities in conjunction with the business processes which are supported by the data, execute it with sufficient resources, and eventually evaluate the results and learn.

This can be done two ways: 1) as an ongoing activity within the business process - which requires quite often a behaviour change or 2) by a dedicated group DM professionals which helps building DM skills and professionalising the activity, but which is only possible with sufficient volume of work (and suffient standardisation of the activities)

Bottom line is that it should be understood that this is a job to be done - and it should be rewarded!

Why to document high level information architecture

2007-08-30T11:46:00.000-07:00

Why we are documenting higher level architecture? Isn't the high level too meaningless for doing anything with it? Are these high level exercises ever going to be successful?

Well - I disagree - it is not a waste of time - and this is why:
- it is a method to communicate (mainly with users) - so it should be simple and related to business language. If you can pin-point easily the weaknesses, bottlenecks in your architecture, than it is easier to get approval for improvement.
- it is a way of documenting the high level context, which helps positioning the system - so it should not contain detailed content. High level position will make it easier for systems to find relations to others and helps avoiding overlaps or duplication
- it is a way of helping later stages of development or later projects understanding the system & its context quickly -
- it is a way of helping portfolio management - usually business users are very functionality oriented and therefore there is insufficient appetite for more fundamental integration issues. High level pictures help making the case for this.

These benefits also point at the main requirements for architecture: so it should be short - no details - and it should be easy and quickly accessible

So the end result may look easy, but to actually develop it is quite hard. It is important to focus at the key messages ... And that's still an art (which is rarely covered by case tools).

Documenting information architecture ...

2007-08-24T10:56:00.001-07:00

I am struggling to find a good way of documenting information architecture. At the lowest level of technical meta data there are different methods available (physical data models, entity relationship diagrams, CRUD's ...), but if you go up in abstraction level we enter the world of 'architects' where Powerpoint and Visio are still king & queen. I have not seen a common format or common tool for documenting the high level. The only common symbol is the one for databases (looks like a flat drum), but for the rest it is arrows and boxes that can mean anything.

So my question is if we can agree on symbols for some of the key elements?
- types of data stores (e.g. a corporate store should be different than a project / application store)
- interfaces (point to point) or middleware (hub and spoke)
- applications & utilities - with differences for data retrieval and data capture applications

The second thing is that I have not seen a definite list of what to document in terms of higher level data architecture. I can think of the following:
- the high level overview of the main data stores and the key data transported via interfaces (this can be done e.g. as a context diagram per business area or key business process) - this overview should reflect the different lifecycle phases of the information.
- the main components of the application / integration / data stack (in a different view) per system (or set of systems)

Data Architecture principles

2007-08-17T07:38:00.000-07:00

One of the Data Management principles is about applying proper architectural principles - it is a bit of a meta-principle - and this covers actually a whole category of things. So what is it? It is actually a lot ... Let me try to list a few ideas (it is again a list of 10!):

Data should be created in one place (a defined master source)
Data should be distributed via a hub and spoke architecture (so avoid spagghetti)
When using middleware adopt a single integration architecture (don't stack middleware on top of middleware ...)
When communicating with external parties adopt XML as a standard - adopt as much as possible one XML dialect only
Split data capture from data retrieval
Only store data together when it needs to be managed together (i.e. when you need to see it together you can also do that in the application)Store as much meta data as reasonably possible with the data
Ensure audit trails are implemented on data
Split project information from corporate versions of information
Don't use transaction systems to store your history (archived information). Store this in some sort of data warehouse
Only retain data when there is a value to retain it - but check this across the data lifecycle!

Data Management Principles

2007-08-16T12:00:00.000-07:00

Usually IT departments are very infrastructure or application focused. Information or data is seldom the first priority. In order to establish data management as one of the cores in the organisation it is required to publish some key principles; it is a bit like a consitution for a country.

Establishing these principles and then repeating them ad nauseum is the only way to get people to know the importance. The principles should be easy and should be part of the overall IT architectural principles (as mandatory).

Here are (what I think) the main points:

1. Data is an Asset (so it should be managed like an asset)

2. Data should have an Owner

3. Data should have known Quality rules

4. Data should have a guaranteed integrity across the Lifecycle

5. Adopt international standards where possible

6. Classify each element in the right security class (is there any confidential data?)

7. Ensure data is accessible to whom needs it (open up by default!)

8. Ensure meta data is in place

9. Adopt principles for data architecture (more on this next time!)

10. Ensure internal & external information is treated with the same diligence

I may have missed one or two (but 10 was such a nice number), but if you can get your projects to adhere to this, than you're pretty mature in terms of data management!

More on Meta data management

2007-08-12T03:42:00.000-07:00

Meta data continues to fascinate me, and especially the way we continuously fail to get it to work. Intuitively I would say that it is easy to convince people about the value of meta data. Would you buy a jar or tin in the supermarket without a label? Would you take a medicine without a prescription?

That makes sense and of course we have meta data all over the place in the realm of IT. The only issue is that we do not manage it. So why is this? I may have a few explanations, and hopefully some further solution directions as well.

Explanations for failure:

1) when we try to set an overall meta data management strategy we try to analyse the full meta data problem and then we create usually very complex models. These complex models lead to complex system that nobody wants to maintain

2) when we deliver applications we focus on the delivery of the applications and not for the contribution to the greater good

3) there are usually no people with an overview beyond single projects

4) there are many competing standards, classification structures, etc. Who decides what's leading?

5) Meta data has no visible pay-back. So why invest?

So how to make it work?

1) Meta data only works when used in a business process where people have the interest in maintaining it. If meta data is collected and does not add any value after collecting it, than this meta data is useless. So only collect just enough, just in time. So only collect data definitions, if there is a guarantee that it will be used. That guarantee only exists when the usage is part of a common business process.

2) Distinguish between technical meta data and business meta data. Quite often people get confused about meta data, because they mix the two types. Technical meta data relates to things within systems (tables definitions, CRUD matrix, data mapping in an interface, etc), while business meta data is something understood by 'simple' business people. Things like Data Ownership, quality rules, positioning of the data in a business process. The two types of meta data require a complete different way of managing. Technical meta data is maintained within the application delivery process and portfolio management, while business meta data is usually a higher level issue (e.g. business process redesign). Business meta data is usually collected earlier in the process of bringing in new solutions than technical meta data.

3) Publish business meta data as part of something that is used by everybody; e.g. an Online help or a glossary. Earlier I advocated the use of wiki's for this, and I still think this is an excellent option. Don't publish technical meta data, but ensure it is part of the technical documentation (build in an architectural check in the project for assurance). It is best when technical meta data is part of the solution (e.g. part of the XML file, part of the ETL definitions, etc.)

4) Keep it simple

Unique identifiers

2007-08-03T11:11:00.000-07:00

Holy wars and worse have been fought about unique identifiers for data items and the last word on it has not been written either. The most purist solution is the meaningless numeric sequence and this is usually the recommendation. So why is it that other solutions are adopted more regularly than this simple recommendation?

Well - people get confused in the discussion on unique identification of things, because if you uniquely identify something, wouldn't it be nice to also recognise it. Why have a meaningless number if you can also call me Evert? And this is where people make the mistake. There is a difference between human recognition of a thing or person and what systems need to do with its related records. Every system has different ways of dealing with the names of things and therefore every recognisable identification will have to go through multiple conversions if the item is used in many systems. A neutral number does not have this problem.

The other issue is that things that you can recognise can change. Take country codes as an example. You would think that a country is a pretty stable object, but Upper Volta became Burkina Faso and Birma became Myanmar and then I even did not start mentioning Serbia ... Every time a country changes you have to change the identification of the object in all systems. This can lead to lots of (technical) issues.

My view is that we should allow for both - a unique (meaningless) number and a unique meaningful alias (or set of aliases). The number is the actual primary key, but the alias is what you use in practice on your screen. You can use the alias as much as you like, but when you need to convert, you don't run into technical problems! The more meaningless the identifier, the less discussion when situations change.

The unique identifier needs to be assigned when the object is created and should never change. The best way is to do this via a 'service', an independent component in your system architecture, but this only makes sense in complex large enterprise wide architectures.

Data capture vs data retrieval

2007-08-01T13:02:00.000-07:00

One old concept of data management which is still valid today (and at the same time insufficiently adopted) is the architectural split between data capture vs data retrieval. It seems a bit artificial to split datastores in this way, but in larger architectures it makes absolute sense. This is not only because of the traditional reasons of tuning the retrieval database for performance (the old data warehouse concept), but mainly because of lots of practical reasons. Here are some:

Data capture is usually a complex process with steps for QC and validation, while retrieval is read-only. This leads to different data models, security models, etc.
Data capture is usually for just a limited number of users with various access rights, while data retrieval requires a focus on sharing
Data retrieval environments are focused on information retention over time
Data capture environments should only exist once for a data type while data retrieval environments can exist in multiple ways. Once data is created it can be shared or replicated instantly via messaging services

Having this concept in the back of the mind whilst architecting a data environment for an enterprise (so not for small systems!) is very useful and can simplify enterprise wide solutions.

Unbelievable

2007-07-21T06:43:00.000-07:00

Last month I wrote about a cyclone in Oman and the one thing that struck me after the event was the lack of preparedness everywhere. I think the worst hit was the Royal Oman Police. I guess they has their data center in the middle of the floodplain where some of the worst damage happened and because of this they lost all their main servers. They clearly had no business continuity plan (BCP) and no disaster recovery (DRP) in place and did not think of a scenario of 3 feet water in the main server room.

Until the cyclone, Oman looked like leaping into the twentyfirst century with their advanced goverment IT integrating the civil administrations of people, immigration, drivers licenses, work permits, car registration, etc. etc. I was amazed by the advanced way in which the data was integrated (e.g. my drivers license number was the same as my number of my id card which allowed me to work in the country, and my car was linked to my drivers license - and therefore also the traffic offences ...). This clearly added a lot to the efficiency of the country, but the bottleneck was the data center in the middle of a wadi.

After the cyclone I could not export my car (since it was not known if there were traffic offences still outstanding), immigration moved back to a paper based system and all kinds of other services grinded to a halt. Luckily Oman is part of the Arab world and therefore there is always a way to get things done (the secretary of the main officer in charge was a cousin of the guy helping me, etc.) and therefore I could leave the country in time without major issues! No data management can help with that

So bottom line is - I think what Oman achieved before the cyclone in terms of government data management can be seen as a 'best practice' (just try to compare this to the UK where hardly anything is linked!), but when leaping forward we cannot forget the basics - arrange sufficient protection for the vital records and have a plan in place for the worst!

Being an Information Architect

2007-07-19T23:14:00.000-07:00

I've got a new job, Global Data Architect and the nice thing is that I can define the job from scratch. So that means I will spend a bit less time on management of people & money (the joys of HR and Finance always overtake the content of somebody's job in the middle ranks) and a bit more data management content. It is another way of putting in practice what I have been preaching.

So what do I plan as my new activities?

Educating others on the main principles of Data Management - I already wrote about this http://infomgr.blogspot.com/2007/06/data-management-fundamentals.html

Defining and promoting Standards - http://infomgr.blogspot.com/2007/06/limits-of-freedom-manage-your-data.html

Defining how much Middleware we need as glue in between our emerging standard data stores - http://infomgr.blogspot.com/2007/06/middleware.html

But most of all I will focus on Change Management, aligning people toward our common goal of a seamless digital company - http://infomgr.blogspot.com/2007/05/art-of-im.html

What I won't do is focusing on building the perfect data model, like the Oil Industry tried with Epicenter (see http://posc.org/ - the good thing is that also these guys have rebranded themselves into Energistics and now focus on standards - so I am sure there will be synergies!).

I also will not rename this blog ;-)

Unlocking value

2007-06-24T02:01:00.000-07:00

Data Management is a bit of a boring subject to many, But when done well it can lead to massive benefits that you cannot even imagine at the start. An example is the success of the Internet. At the end of the day it is actually a data management success. The W3C has been very successful in setting and enforcing a few standards on how to address machines in a network (IP adresses), how to exchange information (the IP Protocol), how to name domains (DNS) - and replicate this across the network, and how to publish and link information within these domains (HTML). Everything else after that is now history. Nobody could have imagined the growth of the Internet in the last decade and half.

What it proofs is that we need to agree on certain basics and at the same time need to give the freedom to everything else. Within data management we see people spending enormous amounts of time in trying to come up with the perfect data model, or have endless discussions about how to define a unique identifier, but to me these discussions are usually a bit academic. What really counts is that a decision is taken; a standard has been agreed. It can be a standard which is not perfect (like the Qwerty keyboard); but if it works it will have benefits beyond what it was designed for.

Think about what would be possible with the wide adoption of XML to the same level as the adoption of HTML? Unfortunately agreement on formatting is much easier than agreement on content. Still I can see enormous steps being taken in the Web2.0 world where information from different sources is being mashed together from lots of different sources (e.g. Google News). Imagine what you would be able to do if information within a company could be mashed together automatically, because the information is easily recognised? So it is still worth the effort to spend some time on meta data, because it may lead to some unplanned by-effects beyond imagination!

Data Management Fundamentals

2007-06-23T20:21:00.000-07:00

As said before Data Management is about control, control on quality, control on replication of information. Data Management is very much part of the plumbing of an Information Architecture and therefore people tend to give a little attention. Remember - At the end of the day users want to use data in applications, but it is just the applications that they see. And therefore the easier it is for them to get the right data, the better it is. They don't care how we get it to them, as long as it is easy to get and easy to understand. If it is good data than that's a bonus, but if bad data is easier to get than good data, than it is more likely that users will choose the former. If you use Google would you browse to page 25 to get the right result out of your search? No.

So what does this tell us? What it tells me is that Data Managers have to provide a platform which is like using electricity. We use a switch and we have light. So users should get their data served to them automatically (i.e. the data is already replicated to the application and there is nothing to worry about), or should be able to get it very easily; in one place, in the right format and with the right quality - without ambiguity.

In many administrative environments this is already a reality - integrated back office systems have been rolled out over the last decade and people cannot even remember anymore that they had to go through different systems to get something ordered and then an invoice paid. Note that some companies still struggle with their master data, but that's more a flaw of SAP, than a lack of intergration and standardisation.

If we talk about the more complex industries like manufacturing, construction or oil & gas, than we see that we are clearly not there yet. And this is where Data Management really becomes interesting. Quite often data is held in many systems and replication is ad hoc, manual, and clearly not perfect and this is where a lot of progress can be made by applying a ten simple architectural measures:

1) Agree the scope = the master reference data; i.e. data shared across systems

2) Agree who decides about the data in scope

3) Agree the standard for the master reference data

4) Agree the master source for this reference data

5) Agree basic quality rules

6) Agree how the information is replicated (format, frequency)

7) If more than one version of the same data should exist, agree how to deal with versioning

8) If users have to shop for the data themselves, than ensure it retrieval is in one place

9) Automate as much as possible, and keep it simple

10) Ensure there is a data helpdesk in place

This is all no rocket science - but in most large organisations this is a struggle!

The limits of Freedom: Manage your Data

2007-06-22T00:03:00.000-07:00

The points made about supporting maximum freedom for the users are mostly true for Document Management, but when we talk data than we should be careful about allowing too much freedom. Data is by definition a more structured type of information than a document and structure means the requirement for control.

In a well functioning data management environment we have data sets that are used in more than one environment and therefore it is of paramount importance that this master reference data is of the best possible quality and adheres to clear and common definitions. Even better - it would be great to have a number of tags with every data item, telling what it is and telling you what has happened to it.

This is actually in line with some of the Web2.0 thinking. For document management the web2.0 looks like reducing the controls, but for data management this is not true - since this is about adding more context. And this is where the ideas of the semantic web come into play. This idea of the semantic web is about putting information into context so the meaning of information (the semantics) can be understood by machines (search engines, integration engines, etc.). Integration should be a problem that you solve once and by establishing open standards this should be achievable.

Please note - I am not advocating to add a lot of overhead to managing data, but merely a number of measures that should be taken into consideration for every data item that is shared across systems. The basics are as follows:

First of all: Know what is should be shared - don't put measures in place on items that are unique to one system (unless that data is critical)
Ensure integration of meta data with data - using standard XML schemas is very good approach
Put in place some quality indicators - automated measures checking completeness and consistency of the data - this can be part of the meta data
Also add other tags like time stamps and userids of people who have done something with the data (audit trail)

A controlled data management environment is a must for making business processes work across the company and across systems. The more you ensure the data adheres to transportable standards and is available with contextual meta data, the more you can integrate. At the end of the day data is only part of the information infrastructure. The real value is in what you do with it.

It also can help reducing the controls on document management. If documents can be automatically tagged via extracting words listed in master reference data, than that reduces the overhead in document management.

Freedom vs Control

2007-06-18T21:47:00.000-07:00

Information Managers (and also IT people) have the tendency of trying to get things under control. That's why we put a password on everything and that's why many systems are cumbersome to use - would you really want to add these 20 mandatory attributes with the document? Do you really need to go through these 6 steps of approval?

What we sometimes forget is that we manage information on behalf of the users. We are there for them, and not the other way around. So to really be successful as an information manager we need to sometimes challenge our own natural inclinations of building in more security than needed, or more meta data than feasible. Some of the measures I described in the item about Hoarding are therefore more than a user can accept.

One of the measures I mentioned was getting rid of shared drives. But maybe this is a typical dictatorial action, with the information management goals made more important than the user. But do we really want to do this? Recently I noticed a colleague taking this brave measure in his organisation. I fear that he will fail, since the technology offered as an alternative is not as simple as a file system ... and therefore users will look for shortcuts that will defy the purpose of this measure. Dictatorship leads to terrorism!

I have already written a lot about the Web2.0 - how we can add a lot of value by increasing the freedom of users in terms of how they classify information and how actually opening up our security model will be beneficial as well. Still we like to have control - that's just natural behaviour, but my view is that we should not get this control by confining the users too much. We can also get this control in an easier way via automated means:

We should not spend too much time setting up complex taxonomies, but should be investing in automatic tagging mechanisms based on master reference data.

We should not spend too much time in asking people to clean up their old information, but have automatic archiving procedures based on where people store it. This gives people the freedom to collect what ever they like

We should not have complex security models, but have 'buckets' that are open and areas that are closed.

This is all technically very easy and is more about a changed mind set (behaviour) than anything else. At the same time it will help the users, since it will not put too many restrictions on them.

Hoarding and ILM

2007-06-14T21:17:00.000-07:00

There is another approach to hoarding - just let it be! It is less of a problem if a few conditions are met, as I described before in some other posts. The conditions are as follows:

If all information is kept in an open digital format

If there is an open security model (i.e. only really confidential information is stored under lock and key)

If there is plenty of disk space (which is cheap today)

And if there is a powerful search engine

Than you can make it work. But this is easier said than done, since managing large volumes of information needs some sophisticated technologies supporting optimal storage, supporting fast backup and effective restore, and it needs to be able to recognise versions of the same information.

Some hardware vendors like HP and EMC are now exploring this space and are coming up with so call ILM solutions (information lifecycle management). These are not just powerful storage servers, but 'intelligent' solutions, since they come with software that enable the optimal storage. The HW vendors see it is a way to get higher up the value chain and companies can benefit from this aim, since the hardware in the company becomes more valuable, since it is managed in a more intelligent way - win-win.

It sounds all promising, but at the same time there are no silver bullets, so I expect that ILM solutions only have the potential to work if they are accompanied by a lot of consultancy and a lot of drive within the company to make the implementations work.

Hoarding

2007-06-12T22:36:00.000-07:00

I am in the process of moving and then you realise how much people are hoarding stuff. Every job move a take about 1 or 2 boxes with me, full of old courses, interesting reference documents, etc. and usually I never see them again until the next job move. It is hard to break this habit and it is clear that I am not the only one. All around the company I see cupboards full of papers and I guess there is a lot of junk in it that we throw away, plus a few valuable gems that we may have lost ...

So is this a problem? Probably ... because paper takes up a lot of space, people loose a lot of time searching in their papers and valuable information may get lost, because the information is not accessible anymore (since it is store in some sort of personal collection).

So there is some merit in address the 'hoarding' problem. And actually there are ways to address this.

One - drastic way - is the idea of Open Plan offices. You may love or hate them, but for hoarding paper it is a good remedy. Just allocate little space for storage and then apply a clean desk policy and soon people are only retaining the most valuable information. Obviously this only works if there is also a good library type of function, else people start to throw things away that should have been retained.

This problem also extends to digital data and therefore a similar measure like clean desks can be applied to shared drives. Offer people a controlled environment for key documents and clear out the team space (say every week).

Another measure is the 'big brother' type measure of having a information delivery compliance monitor for what kind of documents need to be stored at the start and end of each business process. This works for repetitive processes (e.g. every project should have a signed off project plan, stored in the EDMS), but this does not work for more creative, or iterative environments. In that case it is hard to measure the value of what has been posted as key information, but than a manager could sign off on a general statement that key information was managed. All other information should be thrown away, or at least stored outside the prime information traffic areas. Note that this only works if it is on the manager's scorecard!

Getting people to rotate in the organisation is also a good way of cleaning the house once in a while. Usually people tend to clean up a lot of information at the ends of their jobs and the new person coming in can do another sweep.

Vital records protection

2007-06-11T20:48:00.000-07:00

Due to the cyclone of last week I have had quite some thoughts on DRP and BCP, and concluded that the scope of this should be pretty limited. Risk management is a good way of scoping (only records unavailability that causes a high risk needs to be considered as part of the scope), but this can be a slightly too narrow approach and may lead to short term gains and longer term problems. Therefore it is important to define for a company the vital records. A simple definition is: Without these records you can close the shop. So what are the main parameters in defining vital records?

Legal: By law you need to have these records - usually these records are related to agreements, contracts, financial transactions and people. Quite often these records have a legal retention date (think Sarbanes-Oxley act). Obviously these records need to be properly protected. The most important (and therefore vital documents) are the ones related to major agreements. The bulk type (invoices) need less protection.
Asset Integrity: Records on the structure and state of maintenance (integrity) of facilities are vital for any operation. Think drawings, designs, inspections. Quite often these records need to be retained indefinitely and they require a high level of protection in terms of protection against damage or loss. Quite often they're not confidential.
Confidentiality: Some records in the company are company secrets that can be the differentiator for doing business (think intellectual property, major strategic documents). These records need protection both from a confidentiality and a physical protection perspective.

So what to do? I think first of all it is important to have records in an open digital format, i.e. stored on a computer and easy to open with normal desktop tools (think PDF). Paper is great to work with, but digital is the only proper backup mechanism. So if you have paper than it is important to have at least good quality scans of all vital records. The scans need to be indexed properly and need to be made available online (with sufficient security of course). Obviously some records (like facility drawings) may have a more specific format (e.g. Autocad).

Further it is important to have the digital records protected physically against any disaster (flooding, storm, fire, ...) and the easiest way to establish this is having them stored in more than one place (preferably more than two). Note that these places need to be geographically apart. In some countries companies have decided to have their information backup abroad (especially recommended for more volatile places). So if you have your records in Amsterdam (or New Orleans), than it is wise to have a backup in place in a higher place (so not next door, but say in the Alps or the Rocky Mountains).

The backups have to be made on a regular basis (daily) and more importantly: you have to test if the backup can be restored! Note that the same security measures (on accessibility) need to be in place on the backup (so confidential data is also protected in the backup site).

So bottom line: it is important to know what is vital for your business and it is important to have good protection in place for them. Not just in terms of security, but also in terms of physical protection against disasters and the best protection for the latter is replication.

Managing Risks and the world of Web 2.0

2007-06-09T09:08:00.000-07:00

Quite some time ago I wrote about Information Management that it is actually all about managing risks. And our little cyclone has reminded me of this fact. We suddenly realised that most information in the company is actually not important (or let me rephrase, not important enough for business continuity)

So a lot of what we do in Information Management is more at the 'nice to have' end of the scale of things. But having written this statement does not mean that there are some nice things about IM / Web 2.0 that are now emerging as elements that can add value in the continuity of a business or be of help during recovery from a disaster.

The Web2.0 has brought us wiki's, blogs, ... and this is how they could help:

Wiki's can lead to more up to date procedural information, but can also help with collecting the learnings from a disaster event. Everybody can contribute! Many people know more than a few isolated auditors

Blogs can help during the disaster to share news and bring people up to date on the status of services and other things. I noticed for instance that AP, the news agency has started to use pictures from Bloggers in their news coverage. Mobile blogging is also possible, so why not use your phone for things like this?

The free online storage spaces for content (pictures, movies, but also documents etc) can act as a temporary business continuity site when your own services are down

But having these things playing a more vital role means that these Internet based services become more vital and therefore become part of the DRP/BCP (disaster recovery planning and business continuity planning). That's a way to become important!

So the infrastructure side of things needs a possible rethink as well. Usually data centers are setup as a single point of failure and having the Internet as a vital piece of communication technology does not allow for single points of failure. So setting up the companies infrastructure as a set of networked nodes is a model worth considering as well.

Some notes on Disasters

2007-06-08T22:10:00.000-07:00

A few days ago we were hit by a cyclone, and this brought up the subject of disaster recovery and business continuity again. We have spent many hours discussing vital services in the past and how to ensure continuity for them (or how to ensure that we could recover from a disaster) and thanks to the cyclone we could test our assumptions.

On disaster recovery (DR) I noticed that we were lucky. We had an orderly shutdown before the storm hit us and no real damage was done to the data center. So recovery was easy - it was just a matter of getting the computer floor dry, getting power up and running the start-up sequence.
Things would have been worse if the storm had wiped out our facility, since we had only limited backup equipment - and most data was kept in the same site (just 500 m down the road). But arranging a full mirror with sufficient distance is expensive ...

On business continuity (BC) I noticed that really most services are not vital. The main office was completely deserted, but production continued. The next morning when I arrived in the data center I saw nobody around, but all services were humming happily. Is it really that most what we do is not really necessary? It was good to see that most production operations just need basic IT functions and therefore they were not hit. This also gives me some thoughts about our plans for IP telephony (IPT), since this is a more vulnrenable service that the good old fashioned system that we have today, so it is clear that we still need to have it as a back-up.

So what were the things I missed during the storm? I actually missed mostly my access to the Internet! Being cut-off from the rest of the world with just rumours - no real information - can be dangerous. People were making assumptions about the wind, the waves, etc. and through this had the chance of making the wrong decisions. So having a good news service up and running was vital. If the company also has IPT in place, than Internet + IPT become the most essential services to keep up. For the rest it is mainly about keeping power & water up running, so all the basic IT needed for this is vital. The rest can just run on a laptop (e.g. information on emergency and recovery procedures, or even on paper).

Bottom line is that for BC it is only the bare bones of the IT services that are important and this service obviously needs electricity, so any back-up system in place should be able to run for some time in a limited mode on maybe a small generator or batteries, so we keep the basic utilities up and running + the communication to the outside world. Anything else is luxury. And for DR I think it helps to have some conscious decisions on what information (+ apps + hardware) should be available in a recovery site at some distance, because you may need it one day ...

Middleware

2007-06-02T04:19:00.000-07:00

Since the day we started moving from two-tier to multi-tier IT architectures we have seen a growing complexity around how we integrate applications, data, etc. Through this we see a growing number of 'moving parts' emerging in every implementation stack. It increases flexibility, once up and running it will make deployment of changes a lot easier, it even helps with scalability, cost control and ..., but it becomes pretty complex.

The so-called middleware market (covering hubs, connectors, real-time services, etc.) has been growing a lot and quite often I see that we need integrators to integrate the integrators (just to add to the complexity) and that's why it is important to really work on you architecture.

The rules are actually quite simple (and very old fashioned)

Keep it simple (and this is rule nr 2 and 3 as well)
Standardise - try to have one technology for the messaging and data transformation
Do portfolio management (limit versions, vendors, etc.)
And continue with keeping the overview of how data flows from one place to the other. This will help with challenging if another point to point interface is needed (why not channel it via a hub?)
Always challenge the next middleware technology, because quite often there are simpler solutions possible (e.g. if you can solve it with data integration in the database, why have a messaging hub?)
And if you have to choose a technology, than choose an open standard using XML, because that will keep your options open. Especially for integration with the outside world there is a lot of merit of using XML, since it is self-describing.

If you don't do these things you will see spiralling costs, the need for consultants with exotic skills and at the end of the day an unmanageable architecture ...

Information Architecture

2007-06-01T02:53:00.000-07:00

In my previous post I mentioned the word 'architecture' and this is a bit of an elusive concept in Information Management. What does it mean?

One of the most common references in this field is the Zachman framework - a very complete framework that covers data, applications, infrastructure, people, processes and even motivation in all its aspects (from high level to detailed implementation). I think it is a great concept for understanding all aspects of IM&T and it covers a lot I have been writing about.

If we just focus on the data architecture, than the framework is a bit large. It is very easy to lose track in all the things that need to be analysed and documented (before you know it you spend more time on the framework than on improving data management - paralysis by analysis), so therefore I would like to focus on the four main elements of information architecture that are important to me:

- Master reference data: large organisations need to establish as much as possible the master sources for their key objects. So one place to manage people information, product information, customer information, etc. From these master sources this information can be shared. Master data requires common definitions and clear ownership of information.

- Middleware and data integration: large organisations need to define clearly how information flows from system to system. This is to avoid spaghetti integration. Different integration concepts are possible (via a central data store, via a middleware layer, etc.)

- A supporting Data Management organisation: An architecture needs to be owned & maintained. Just like a garden needs a gardener. The Data managers take care of establishing the blueprint, the standards, ensure quality is measured & improved and obviously they take care of the day to day operation of the data stores and interfaces.

- A high level story: To establish all these things requires sustained data management investements and this can only be achieved with sufficient sr management commitment. So the data architect also needs to have a high level story on what architecture can achieve and what improvements (successes) have been made.

The Art of IM

2007-05-27T00:45:00.000-07:00

Some time earlier I already wrote about skills and I am constantly reminded that this is still one of the main issues in Information Management. People development of IM&T staff is still focusing a lot on technology (infrastructure and applications), while the real challenge is about understanding how the new IT elements fit in the bigger business picture in terms of process, people and information.
Unfortunately we prefer to talk a lot about new technology (ie new applications) and get excited and so the next application is implemented without proper data integration, without proper business workflow realignment or even worse people just focus on new graphics and say 'Wow!' and still the quality of the visualised information is bad.

So why are we so technology focused? I think the simple answer is that everything else is a lot less tangible and much harder to solve. Getting people to work with information in a different way is an art and of course if the technology is guiding people in the right direction (ie becomes an enabler) than that's great, but often more than just technology is required. Improving the management of information is about improving the way we do business, changing the way people work and keeping the focus on information as part of the assets of the company.

So how do we change processes?
- we can only change a process if the business wants it and commits to changing itself (see the notes on having a champion, etc.). If IM&T comes in as an outsider than change will not happen. It helps if the IM people can take away chores (data loading, simple QC, data disposal), but that needs to align with the way of working in the business

So how do we address the people aspect?
- people are resistant to change and there will always be people that cannot be convinced that something new is something good. So focus on the ones that want to change (or at least don't resist), address them with their language and at the level they want to be addressed (so don't bore them to death with technical details ...) and then take them by the hand. Again - just like with the process bits - it helps to have dedicated people taking away the chores.
If people see value (better quality, easier access, ...) than change will be easy. Also don't give people too many options, or too many ways to non-comply, because than it won't happen either.

And finally - how do we keep an information focus?
- bringing an information focus into the company is very hard. People see processes and therefore functionality and although intuitively people understand that information is required to do their business, the concept of managing it in a certain way is still hard for most people to grasp. On top of that - we are all information managers, since we all manage information! And therefore every Tom, Dick or Harry with a PC thinks that they are an expert in it.
So what do you need to do as information manager to get this focus? Well, it helps to have some standards / guidelines documented, so the business people know that you have an established business and a set of rules to adhere to. It also helps if you have an Information Architecture - ie you know how information flows through the company in a structured way and where you want to be in the future.
Finally - it helps if you need to comply to some external rules (laws like Sarbanes-Oxley, SEC requirements, etc.). In that case you can threaten your bosses that they go to jail if they don't do proper IM!

So bottom line is that I still see a lot of the same problems in companies with respect to managing information - the problems related to establishing good quality information, which is easily available and and which is protected by security only where it is needed. These problems can only be addressed if the IM&T people have the skills to change processes in the business, help people to change their way of working and create a mindset with value of information as focus. This is an art!

Sharepoint

2007-05-16T05:26:00.000-07:00

I was disappointed by Vista, but Sharepoint is another story. I think Microsoft is definitely up to something that can help with continuing the domination of the office world.

The Sharepoint idea is that everything can be workspace on the web, where you (as a user) do not have to worry about formatting, or managing the links - all you need to know is how to manage your content. You can be in MS Office, publish a document to your workspace (or your team's space) and invite others for review. Creating content & publishing becomes one and websites are not longer separate elements, but integrated as part of the environment where all users can share information.

This works for managing the intranet, because web sites become part of everybody's information sharing, while at the same time reducing the clutter in terms of look & feel. This also works for finding information (as opposed to Windows Explorer ...), since suddenly everything become a web site and therefore searchable. Like with Vista, Microsoft has invested a lot in this search element, since they feel the heat from Google and in this case they may be able to win, since they still dominate the desktop.

So instead of saving on a shared drive you can publish your content on the web - straight from your office tools and that can be very powerful. The courage you need to have as an Information Manager is to get rid of the shared drives at the same time, since this can easily replace it.

There are some flaws as well ...

I was already disappointed about the hierarchical nature of the folder structures as we know it - and which continues to exist in Vista. Unfortunately Sharepoint is also based on this ... It would have been better to make the structure flat and use unique identifiers. So it is hard to change the structure without breaking links
It is too easy to create a lots of navigation & empty pages without any content - so users may loose the way very easily. It is hard to get an easy site map created for managing the content.
The templates can be too much a straight-jacket as well, creating lots of useless links (to various sub-sites you don't need most of the time)
there is no hit-counter - you would like to see if content is read, but right now you can't

But it is a good step foreward, especially because they seem to integrate it with other products are well (e.g. MS Project). We'll watch the developments!

Vista

2007-05-15T04:47:00.000-07:00

Last year I was looking forward to the introduction of the new version of the Microsoft operating system ... Vista, but unfortunately I was a bit disappointed when it was finally released. They definitely worked hard on the graphics (think Apple ...), security and the search capabilities (they feel the heat from Google ...), but from an information management perspective they have missed the opportunity to make a major step forwards.

What could they have done?

Get rid of the hierarchical folders - information does not relate to a single subject! How do I store pictures that relate to a place, a person, a date, ...?? The concept of virtual folders is still too cumbersome
Introduce automatic tagging - why not propose tags when you store a document, or have features like in this Blog (when I add tags it can show my list of tags I've used before). This is not rocket science. There is meta data in all Microsoft files, but this is still cumbersome and not related to anything.
Integrate the meta data from many products. Why not have your contacts in Outlook available as tags, and as virtual folders, ... etc.?

Just some ideas for Mr Gates - if he likes to have a chat about this he can always send a comment to this Blog!

The Value of Meta Data

2007-05-13T09:25:00.000-07:00

In my previous post I already mentioned that I created a wiki dealing with meta data (check out: http://scratchpad.wikia.com/wiki/MetaPedia) and the question is of course if this will ever evolve into anything. My experience with meta data (or information about information) is that it is a bit like flogging a dead horse. Still intuitively I think that if can add a lot of value.

So why is Meta Data management valuable?

Meta data can help a lot when searching for documents. Adding meta data in web pages, documents, etc. increases the accuracy of the search result. The more meta data tags are standardised the more likely the users use them to their benefit
Good meta data adds meaning to data, through context and description it is more likely data is understood and becomes information. An idea we tried out is to use something like a wiki as an online help - moving meta data into the knowledge management space
Meta data can help managing information in a large enterprise context. When companies have to deal with many systems and complex data integration issues, than it is good to have an overview of data definitions, rules for data quality, rules of data integration (e.g. master source), etc.
Meta data help automated data integration. When information is tagged using standard XML schema's than it becomes possible to integrate in all sorts of ways. Standardising on XML requires good meta data management and clarity on definition, else it won't work. The creator of the world wide web - Tim Berners-Lee - is now working on the Semantic Web and this all links back to having a standardised way of defining content (maybe more on this in a later Blog entry ...)

So if there is so much value why don't we do it?

Like said earlier - meta data management is a bit of a dead subject for most. People get excited by new gadgets, functionality, look & feel, but not about something administrative like meta data. It is like plumbing - nobody is interested where shit goes, but it is nice the sewer exists! Good meta data management is hard work and is not for people that are not very organised ... and without getting clear & quick reward nobody will do it.

So how to get it done?

Make it exciting - this happened a bit when XML became a hype at the turn of the century. And the good news is - XML is now everywhere. So with other words - if you use the right 'hype' word, then it may get exciting. XML sounds like it is new, high tech, the next big thing and then some people actually start doing it
Do it integrated - meta data management should not be standing on its own, like an ugly girl at a party waiting to be asked for a dance. When meta data is integrated with the data (like in XML), or with the application (like an online help), or is part of the companies knowledge management (like in Wiki's), then there is a change it will work out.
Make it easy - if I need to tag this blog, than I get automatically a list of Tags I have used before. So why not have something like that built in part and parcel of tools like Word and Powerpoint? Maybe a future 'Vista' will do this automatically for us?
Make it rewarding - as a manager you can reward people for boring things (like organising the junk-free day), so why not have reward for collecting & managing meta data?

Anyway - let's make meta data work - and if you happen to have more information on meta data, than add this to MetaPedia on Scratchpad Wiki.

Wiki's

2007-05-12T10:17:00.000-07:00

Knowledge Management is one of these subjects that raises many eyebrowse when people start discussing it. Some start to giggle and before you know it there is no serious discussion possible. Luckily Web2.0 has come up with something that stops these giggles quite quickly: Wikipedia. Nobody ever thought that it would be possible to harness knowledge from many many people via a process where there is no real line responsibility and control is only possible afterwards. But the great thing is: It Works!

I have tried out if it works on creating a structure on meta data (just check out: http://scratchpad.wikia.com/wiki/MetaPedia) and with a bit of structured thinking I could create lots of pages in a few hours.

Some research has shown that Wiki's are quite often more accurate and more diverse than the Encyclopedia Brittanica and the track record of abuse is minimal. So it seems like we have a great opportunity to learn from this.

So my observation is that Wiki's can also be used within a company. Obviously it is not wise to replicate the effort on the WWW, since the result will never be as good, but for properiatory information, i.e. things that are specific to a company, it can be a great help. Think about:
- process manuals and guideline - they never get out of date again
- online help pages (e.g. for systems in use)
- basically any type of knowledge sharing where content needs to be structured

The great thing is that it is easy, anybody can add or edit it, and due to the audit trail feature people will refrain from adding bullshit ...

So time to move from a structured one writes & many read to a situation where many write, interact & read PLUS a situation where the actual content is more valuable, since it is more up to date and more people have contributed to it.

Information Security considerations

2007-05-08T04:13:00.000-07:00

Protecting information assets, i.e. information security is one of the key roles of an Information Manager. The drive to open up advocated in the last few items in this Blog does not undermine this role. Actually the strategy to open up helps protecting what needs protection, since it provides more focus.

Note that most information does not need protection. So what is really confidential? A short list is below:

Anything related to people privacy (think medical, salaries, performance appraisals, ...)
Anything related to money (think contracts, prices, budgets, tendering strategies, ...)
Anything related to risks (think security arrangements or sr. executive travel plans, ...)
And anything else that may harm the company (or people) when published and available on the street (think elements related to reputation, or think confidential strategies, etc.)

All the rest is NOT confidential! So if by default everything should be open (inside the network, which is obviously protected by a firewall, etc.) and only some areas should contain protected information.

What are the advantages:

Implementing Search in a company is so much easier when the crawler only has to crawl the non-confidential records ...
Collaborative working becomes easier, since by default you have access to the information stored elsewhere in the company.
Don't forget about the reduced complexities of managing security in Windows and the overhead that comes with it. If things are easy, than it takes less time, but also there is more focus on managing what really needs protection.

But opening up requires a culture change. Most infrastructure people & IT managers have been raised in an environment of control, so therefore convincing them is a hard job.

There are also some technical considerations to take into account. E.g. when companies have an incremental back-up, than the back-up may fail if people are allowed to drag whole trees from one place to another, leading to GigaBytes of data to be added to the back-up ... Network security should prevent this (so some control is still needed!).

Taxonomy & Tagging

2007-05-07T05:31:00.000-07:00

People managing information have been trying to find the ultimate taxonomy for many many years. Various mostly hierarchical structures exist that help us finding our way through libraries, collections of equipment, pharmaceuticals, etc. This is very helpful indeed when managing large collections of information, but usually the creative users considers these taxonomies as a nightmare. And if they can avoid it, they will not do it.

I believe (as stated in the previous post) that Search helps us mastering a lot of the information management problems we have today (around finding the information), but I also believe that we need to give Search a helping hand. So how do we do this?

The first thing a corporate tiger can think of is adding more control - i.e. forcing people to use taxonomies, but this will not help a lot. You can of course fix top levels of directory structures and through this enforce some sort of taxonomy, but too much control will frustrate people and have them looking for loop holes.

Therefore we have to trust the user. We have to start relying on their own sense of responsibility. They will tag their documents in their own way. It is not only about trusting the users that they will do it (and in a 'good' way). It is also about trusting that this relative anarchy will give some order at the end of the day.

- About trusting the users: Usually people are not brainless. With other words they will try to do something sensible. Especially if they see that it is easy and that it helps others.

- About trusting it will work: Just look at the whole Web2.0 phenomena, as mentioned a few days ago. We have lots of self-organising sites already today and they work. Why not do this inside companies as well?

You also need to help the users a little, just like in this Blog. If I want to use tags I can see what I have used before. At a company scale it would be even possible to suggest common tags to users (almost like taxonomy), but these tags do not exist because they were invented in an invory tower, but because they were used on a regular basis by others. This is called Folksonomy.

The Power of Search

2007-05-05T23:03:00.000-07:00

Having powerful Search tools in place is one of the critical things that will change the way Information Managers have to work. Our work used to be about organising information, ensuring its availability, security and quality, now it is about opening up our internal infrastructure so people can look for themselves. Search is the key enabler for this.

But we are not there yet!
First of all there is still no 'out-of-the-box' solution that can easily cover everything. Although I must say Google has come a long way. The Google Search Appliance is about the closest thing you can get to 'out-of-the-box'.

I have evaluated the solution and below is my verdict:
+ Ups: it just works like Google on the WWW and therefore people love it & trust it. This is great for internal marketing and yes it actually unlocks a lot of hidden information!

- Downs: 1) it still struggles with internal Security ... Having powerful search in place requires a culture change in the organisation. It requires breaking down the walls between departments. 95% of all information inside a company is open, so why not share it?
2) The other thing is that a lot of information is very homogeneous in a corporate organisation and Google is not great at that (especially if there are no cross links). So therefore there is still work in educating the content creators to be specific about their content when storing the information on the network

IM and the world of Web 2.0

2007-05-04T02:53:00.000-07:00

I just watched a video on Web 2.0 (just search for Web 2.0 on YouTube or download via http://www.mediafire.com/?6duzg3zioyd) and I realised that most companies still live in the world of yesterday. Most information is hidden away in personal drives and protection is the keyword. Only a few people realize that most of the potential of all this hidden information is lost by the fact that it is overprotected and hidden.

I think companies can learn from the Web 2.0 phenomena in rethinking the way they manage their information. This whole wave of new(ish) thinking can also be applied within the firewall of a company. Think about:

Why do we have everything protected? My view is that everything should open up. Only a few parts of the infrastructure are only for named individuals (think contracts, think HR), but the rest should be open for everybody. This will unlock information in an easier way and will reduce the burden of managing security
Why do we try to classify information with complex taxonomies or other classification schemes? People normally just fail to do this, so allow people to add their own tags and ensure these tags are easily visible. This will create a folksonomy for the company over time. Folksonomies are not perfect, but have much more potential than a top-down classification structure (just check out del.icio.us or flickr)
Why don't we have quick company-wide search? Search is obviously one of the answers as well.
Why don't we collaborate on content in a transparent way? Think about Blogging! (even the CIA is doing it these days!), but also think about creating a corporate memory via wiki's (http://www.wikipedia.org/).

Key answer for all questions is: Because most people managing information still live in the world of yesterday.

Crisis, we need a Crisis

2007-05-04T02:44:00.000-07:00

Most people in the US only started thinking about Global Warming after Katrina hit New Orleans. The same thing is to say about Information Management within companies. Sometimes you need a good crisis to put things on the agenda. Think about what 9/11 did for Disaster Recovery Planning!

So the trick is: Look for a crisis, and if it is not seen as a crisis, then at least try to advertise it as such. Think about the following possibilities:

Do we soon expect an audit?
What about a pending office move or reorganisation?
Is the company merging / divesting?
Are people leaving / retiring with a lot of knowledge in their heads?
Are there any plans for outsourcing?
etc. etc.

Anything can look BIG (if you do your best) - and therefore it can be seen as a crisis; a good business case for change.

When reality kicks in ...

2006-11-02T08:53:00.000-08:00

So you know what you want to do; you know your priorities and how to achieve them ... or not? Well, at the end of the day most of the time you will be spending on other things than the improvement of Information management. This is due to ...

- Lack of Competence
- Too much overhead
- Indifference or ignorance

So how to deal with this?

Competence: Don't waste time on hopeless cases. If you can pick one genius - than that's better than 10 mediocre types. I think skills is still the major challenge of IM. People don't understand the concepts, or are simply too technology focused. Try to pick the few bright people that do understand and make them the change agents.

Overheads: Skip what you can get away with. Large organisations usually invent more bureaucracy than you have time. It is unbelievable how much is not really needed. So don't try to be too diligent with the things that don't count for the real content of your job.

Indifference: Well, that's a hard one. Again - if you are on a mission you can ignite yourself. Try to look for natural allies and ignore the real morons. And else ... create a crisis.

And finally - pick your battles!

Technology, great technology

2006-10-29T07:15:00.000-08:00

Of course technology is important too. But the most important things about technology are clearly not the things that most IT people like. Lots of IT people like - the latest of the latest (why role out version 1 if version 1.1 has just been released?), high levels of complexity (why have a simple executable if we can have a multi-tier solution), and most of all they like things that are different from what we already have today ... (why keep going with Windows if we can have Linux?)

But clearly this is not what we need today. We need something that works now (reliability), something that's simple to use (no I don't want a 5 day training course, no I don't want another password) and something that saves me time (no I don't want to browse through a ten-level folder hierarchy before I can find where I can store my document and then be forced to add 25 mandatory attributes). On these three tests most system implementations fail.

And therefore I would like to publish here a few simple rules for implementing IT things:
1. start small and let it grow - this allows the system to be implemented quickly and adapt itselfs to the business while it grows. I am a great fan of iterative development.
2. think developing the applications from building blocks - and then use these building blocks by replicating the same gadget in different business process throughout the organisation. Building blocks can be functionality but also reusable information. Example: A document loader is a document loader, so why develop complete different things for different users. OK the user interface may differ, but the data model (supporting a common taxonomy) and the core functions should not.
3. think information before functionality - always remember that the application will be replaced in a few years time while the information stays behind, so don't make the information application dependent
4. keep it simple (but everybody always breaks this one ...)
5. only protect what needs to be protected (so keep security models to the simplest possible level if you can)
6. only share what needs to be shared (so don't overload the information warehouses with stuff that nobody needs)
7. Get as little as possible other people involved, but get the key people - this counts for both the userside as for the developers (better have one great developer, than 10 mediocre ..., by the way great developer also means somebody who can live up to all the other rules)
8. Avoid the latest technology, unless it has a > 100% advantage
9. Always stop people's private hobbies
10. Be prepared to have a fight about all of these rules.

How to start dealing with the key problems

2006-10-09T08:48:00.000-07:00

Practicing Information Management is a bit like trying to mop up water from the floor while the tap is running. More information will be created in the next ten years than has been created in human history before today and therefore pro-actively solving problems today is key to avoid even bigger ones tomorrow. But the issue is that you can start everywhere and where ever you start the information keeps on flowing in.

The risk assessment is already a way of finding the priorities (only if you think there will be fire you will buy a fire extinguisher), but apart from knowing the risks you need to have a clear Champion in place first before you can even try making a start. The Champion needs to be a pretty senior figure who is there to make things happen for you. As said before - a large part of IM is about changing behaviours - and this cannot be done without high level support. If the person signing your staff report at the end of the year is also pretty keen on getting basic issues with information sorted, then it is more likely IM will get attention (and money!)

This also brings me to the second point: IM elements need to be clearly stated in tasks and targets of all staff - especially if it is measurable then this can make an impact. It is important to describe the responsibilities w.r.t. data and documents and then make it possible to check if people actually met their targets (e.g. a contract is not closed out if the related documents have not been uploaded in the document management system, etc.)

In some cases it also helps to centralise roles into full time jobs. If handling data or documents is 5% of everybody's time, then it is very likely things go wrong. If 19 people have one IM person do the 'chores' for them, then they become more efficient, the IM person is more focused on getting the information sorted and at the end of the day he/she can be an ambassador for IM in the business process.

These things are all low tech, but vital to get to a higher maturity in IM processes in the organisation. And don't forget that only when these things are in place it can be useful to look into technology solutions, else don't even try.

Managing Risk

2006-09-28T03:39:00.000-07:00

Information Management (IM) is in the first place risk management. If there was no risk related to the information, then there is no reason to manage it. Making clear that IM is related to risks already will make it more likely that people change their behaviours with respect to information.

So one of the first measures to take in IM is to understand 'where do we feel the heat'. Only where there is heat, we should take measures, because putting a 50,000 dollar fence around a 5,000 dollar horse does not make sense.

So step 1. is: Understand what is valuable information

Step 2. is to understand where there are risks - think risks related confidentiality, integrity (quality) of the information, accessibility or legal issues

These risks can relate to all steps in the whole lifecycle of the information (during the steps when information is Created or acquired, QC-ed and stored, retrieved and used, reviewed and
disposed.

Then step 3. is to think about the impacts of these risks (financially, HSE, reputation, ... ) and the likelihood of occurence.

And finally you can check what controls are already in place to deal with these risks and see if they are sufficient.

This whole process looks like a pretty elaborate piece of work and the truth is - it is! Please note that when you get a consultant in, then you will a lot more elaborate version of this based on the internation standard from COBIT. I would say - don't even try to implement this, since your business will be broke before you have read all the recommendations.

But if you focus on the top valuable information and the top risks (highest impact), then you can create a pretty good story on what needs to be done to manage this information - with a clear link of running your business.

IM is all about behaviours ...

2006-09-06T22:48:00.000-07:00

OK, so what is it that I do? Well, I talk to people (a lot) and basically half of the work is about issueing motherhood statements like - 'if you have not loaded the final records related your work, then your work is not finished!'. There is a lot about awareness (posters and training) and only a little about developing things.

Therefore I see Information Management mainly as a social, behaviour related science and it has not a lot to do with technology (OK, we must get all these systems to work, but that should be only 20% of the story). We have to try out if all these new things work with people, and we can only do that via improvising, and doing change management as we go along. Brilliant in this field was the late Claudio Ciborra - who laid a good foundation for how we could think about the way we deal with the world of IT. Probably IM is closer to HR than anything else.

IM not IT

2006-09-05T02:45:00.000-07:00

Millions of people in the world of computers (or should I say IT?) love tinkering with technology. That's great, since it delivers a lot of progress to us all, but a lot of them forget why we are doing this in the first place ... We have IT mainly to manage information. We have computers, networks, software, the whole lot, to create communications, documents, data, share knowledge, etc. etc. (even to create blogs like this!). OK, I know IT is doing more than this, but I would guess 95% of computer usage should be about IM (Information Management).

So when people ask me - what is your job? I still say that I'm doing 'something with computers', but I actually want to say - I manage information. Why I am not saying this, mainly because nobody understands IM (not even me). Therefore this is a good reason to write this blog. It is a blog about this job, about what it is (not something with computers), what misconceptions people have, what hurdles we have to take. It is also an attempt to put it on the map as a profession, share thoughts with peers and maybe learn ...

So happy reading ...

Evert