Data, Information and Knowledge.

Many years ago I was working at one unit of CSIRO (Australia's major research organisation) and there were regular brown-bag sessions covering aspects of related areas.
There were a number of good talks, sometimes internal, sometime from visitors from other units, other countries etc. There is one that I particularly remember which was about the difference between 'Data', 'Information' and 'Knowledge'.

As I remember the definitions used were something like:
Data = a record, or a set of related values which tell you something about the world.
Information = a document. I remember the presenter spending sometime justifying this point but don't remember the details. It came down to the fact that information was a concrete entity which could be written down and persisted. Thinking about this now I don't think that a physical document should be required. Just that the information should be abstractly external and therefore documentable.
Knowledge = what is in someone's head. He was clear on the fact that knowledge always required someone to know it. Knowledge is not passed on to someone else, but rather created within their own minds as a partial copy of the original. The point being, of course, that no-one can be sure that what they know about a topic is the same as some-one else's view. If nothing else the mental models and contextual links will be quite different.

Several years later I tried to see if any of this had ever been published or was available somewhere within the organistion. I was unable to define exactly when I had heard it, who had presented or even what unit they were from. Naturally I was unsuccessful.
I have found other definitions of Data, Information & Knowledge* but none seem to match the approach I remember.
In other words - I am not sure if what I have written above matches in any way the actual presentation. Still, it provides a basis for further thought.

The approach has significant consequences for things such as knowledge bases or knowledge transfer sessions etc. An organisation, if treated as an organism, has knowledge of its environment and its internal processes. This knowledge is a conglomeration of the knowledge of the constituents, the individual members, of the organisation and is subject to contexual links between them. (I suspect the same applies to knowledge within individuals). Communication patterns - how information is processed and propagated - within any group of people will have a significant impact on the operational knowledge the group uses to perform its function - and on the sensory knowledge it derives to examine its environment.
To maintain knowledge within an organisation requires that it be sufficiently dispersed amongst the members of the group so that the removal of any specific element has no significant impact. Dispersal of the knowledge requires that sufficient communication paths exist to distribute information amongst the people most involved in it. Since knowledge tends to be stratified within an organisation (i.e. each level in the hierarchy has its own priorities; despite "Undercover Boss" you would not expect the CEO to know how to work on the shop floor - their priorities are different) the most important communication paths are between peers. A good manager should understand this and encourage it; while still understanding enough of his people to be able to cover the inevitable gaps.
There is a whole discipline there about the creation of knowledge in new members of a group. Depending on the maturity and size of an organisation and the clarity of the knowledge, there are very many different mechanisms for achieving this. Teaching, Training, Mentoring, Coaching, Leading, or just chatting between the old hands and the newbies. Each has its own advantages and drawbacks and areas of effectiveness. All are related to re-creating knowledge held by one person in the mind of another.
Converting knowledge to documents allows it to be propagated and stored. But storage of information in a repository can only be useful as a back-up mechanism and with full understanding of the limitations. 'Knowledge bases' are only useful if used as temporary storage and constantly updated. Nailing down knowledge as information makes it static and isolated. It removes context and the documentation is locked in place where it quickly goes stale. If picked up in time, by someone who has some mental framework in which to place it, the information can be brought back to life as knowledge. But over time, the signal to noise ratio decreases and it becomes more and more difficult to identify what is relevant.
In other words, unless carefully organised and maintained - and with regular turn-over - any 'knowledge store' collects so much outdated information that it becomes a major effort to find the uesful tidbits. This is the whole point of librarians. They specialise in the organisation, maintenance and search for meaningful information that may be used to re-create knowledge.

but one point that I think I may return to is that one view of the Internet is as a vast information repository. Most of it is turned over very regularly and hence is relatively useful - depending on the relevance it has to the knowledge you already have in your head**. While not particularly well organised (and, I think, all the more powerful because of it), the information can be reached easily which mitigates somewhat. However current search mechanisms could easily become a problem. 4M responses to a simple query is not targetted and it is very difficult to find anything when the relevant keywords are too generic (try finding about a problem where IE8 freezes occassionally - what other search terms can I think of?). Extracting the signal from the noise becomes more and more difficult as the total volume of information increases.

I am not sure where I am going with this and in re-reading the post seems to ramble about a bit. But it touches on a number of points that I think I will explore in more detail later and I don't want to leave them out. Besides - since no-one is reading this anyway, who cares :-)

* For instance: Information and Knowledge as the first and second derivatives respectively of Data with respect to Intelligence. A cute definition but not particularly useful - even mathematically :-)
** Useful is a very relative term. It has a critical dependence on the inclinations of the person acquiring the knowledge and how closely it can be linked to existing knowledge.

Building a kitchen

I work for an IT company designing and implementing software systems for large companies. The following was an analogy that I put together for one engagement to explain why the request we were asked for was not as simple as the customer expected. The work itself was to introduce a new point of sale application to their retail stores. The old system was a "green-screen" interface to an ancient main frame which was also used to manage their repairs centre.

"Our clients intend to renovate their kitchen. They have bought a set of modular units from Ikea and have hired us to put it in place in their house. Their existing kitchen has only the one row of cupboards with the bench-top and the fridge stuck in the corner. The new set of units contains a proper pantry, work-table and a set of drawers.

The project is currently at the stage of working out how the modules should be arranged in the space available so that it can be used the way that they intend. The trouble that we are running into is that they believe everything they need is “out of the box”. It is - mostly - but the box is a flat pack and the pieces need to be screwed together and fitted together in the most efficient layout. We have our screw-drivers and hex keys in hand but need to know which units go next to each other and what the most common activities in the kitchen will be. And plumbing and wiring were never going to be provided with the kitchen units.

For instance – we have been asked to make sure that there is a power outlet near the cutlery drawer. Fair enough, but why? So that it is not necessary to move around much while making a cup of coffee. It turns out that in the current kitchen the coffee is kept on the bench next to the kettle and the cups are in the cupboard underneath, while the teaspoons are across the room. The client believes that having power near the drawer will allow them to have all these things in one place. BUT – in the new kitchen, the cups will be in a cupboard above the bench and the coffee will be kept in the pantry.

If we follow the request to the letter, it will not solve the underlying problem. So we need to understand the *actual* requirement is not to have the power near the spoons but to make it easy to put together a cup of coffee without travelling all over the room. Extracting this requirement from what the customer *thinks* they want is the skill of the designer.

At the same time – we also need to take into account things that the customer doesn’t even think of. The pantry should be near the main entry point so that groceries don’t have to travel far, and the fridge should ideally be against an outside wall for energy efficiency reasons.

One of complications to all this is that our client has had everything laid out in the old kitchen in logical relation to how it is used. With only a single set of cupboards there is no logical grouping according to purpose, but grouping according to task. Hence food is kept next to the utensils used to prepare it and implements next to where they are used.

The off-the-shelf replacement has a completely different, though equally valid, justification for where everything is placed. And that layout is implicit in the structure of the cupboards such that altering will mean cutting and patching the packaged components.

There is also another issue is the customer has also been doing repair work on the kitchen table and keeping the tools and parts in the cupboards and drawers along with the cutlery and plates. Part of our job is therefore to put together a workshop - which should be, of course, in a separate room. I am expecting this separation to also cause some angst, and I am going to have to have some discussion with our team since the same modules (cupboards) will be used for the workshop as for the kitchen. Most of the tradesmen won’t understand why they need to kept as separate as I intend them to be."

Corporate memory

Every organisation, from its inception and throughout its existence, gathers knowledge about its business environment and about its internal operations. In most cases, this knowledge resides in the minds of the people which make up the organisation: the company direction may be held in the head of the CEO, while the correct way to process a claim form is maintained by finance clerks.

All of this information constitutes the organisation’s knowledge base. It may be considered a sort of ‘corporate memory’. Every structured group of people – from the local football club through to nations and global communities – has a corporate memory. In the former case it may be statistics and a set of stories about past greats. In the latter it is comprised of the history and cultural behaviours. In all cases this collective knowledge guides the behaviour of members.

However, the structure of most corporate memories is extremely messy to say the least. In many (most?) cases it is not formally structured or even recognised as a valuable resource. Pieces of information are scattered in many different brains and a thousand documents. Cross-connections rely on random coagulations of data by the members of the group and random associations based on past knowledge.

Entry into any group will always involve some induction or initiation procedure whereby one is introduced to the corporate memory. These processes go by such names as ‘training’, ‘induction’, ‘gaining experience’ and ‘mentoring’ [This is the subject of an entirely other field of study known as ‘teaching’]. There will also be some method for propagating knowledge amongst existing members, although this is commonly less controlled and often completely unmanaged. Compare how much organisational knowledge is gained through rumour and gossip as opposed to reading formal documents.

Mechanisms for passing on knowledge amongst existing vary greatly. There is some that is passed along in sound-bites; a mode designed to ensure that everyone in the organisation has at least a passing understanding. The corporate vision or mission statement is usually of this kind. [The fallacy in this is that just because everyone has heard the mission statement, company values or corporate vision doesn’t mean that they have understood or believe it. The normal response is to simply repeat the same statement it over and over again. Advertisers also use this approach with marginally more success.] At the other extreme are regulations and policies which may run to hundreds of pages – and which are rarely read.

As an organisation becomes larger and more established, knowledge is nailed down in a set of pre-defined processes and procedures such as templates, check-lists, instruction manuals or forms. All these may be considered as conditioned responses to specific stimulus by the corporate organism. Hence small or young organisations have less standard processes and procedures to define their actions. They have had less time or opportunity to learn and react to troublesome situations. On the other hand large organisations can go too far in this direction and institute so many hard and fast rules that they restrict any variation or flexibility away from THE fixed process. Deciding where and when tight rules are required and when loose guidelines should be implemented is not easy.

A formal knowledge base, attempts to resolve some of the issues surrounding the normal diffuse nature of the corporate memory. It does not (and cannot) expect to contain all the disparate knowledge which may be contained within the organisation. However, it should provide a first point of call for any member of a group to find out about anything relevant to the group.

mental notes made physical

I've taken to carrying around a little note-book.
I found that I think best when moving, such as walking, when my mind is not occupied with all the distractions found at a desk.
I also found that I commonly would explore a concept in my mind and make mental notes about various aspects of it. The thoughts could have been about what was going on around me, drifting or chewing over some topic that had come up earlier, or even ideas about work that should be shared or considered in more detail.
The trouble was remembering the points that I had considered later when I got back to somewhere I could write things down. All the marvellous phraseology and concise explanations disappeared and the fine detail of the thought process was gone.
Hence the idea of a physical manifestation of a mental note. With a small notebook I can jot down some mnemonic or phrase to remind my self later where my head was at.
Part of the point of this blog was to get stuff down on paper so that I did not continue to re-visit it so much later. Re-circulating ideas was becoming a problem. I have even lain awake at night because of worries or ideas or fuming or something buzzing through my head. I found that writing it down trapped the thought and I could let it go. Since I was trapping it anyway, putting it out in public (such as it is) made sense. Killing two birds with one stone as it were.
However, it is generally not possible to write things down immediately on a screen - nor desirable I suppose. The notebook so far is bridging the gap neatly - and also allows me to keep track of those rare ideas about work which would otherwise be lost.

Shrinking Office Space

It recently occurred to me that every time I have changed the work that I do, the available space has reduced. If the trend continues I will be doing all my work on a laptop sitting in a chair in the corner.

This started as an undergraduate where I had an entire room. Granted it was also a bedroom as well and not very large, but it was wholly mine (shared accommodation at universitys are not common in Australia, although I gather it is standard in the US) and was my major workspace at the time. I did a lot of good focussed work in that room - as well as a bit of socialisation.

As a post-graduate, I did nearly all my work in the lab; a room about 8m x 8m with 4m ceilings. I shared with my supervisor, his research assistant and 3-4 other students. Despite not being particularly interested in the topic I was able to get a fair bit done and had close, but not intrusive, contact with everyone I need to collaborate with.

As a post-doc fellow, I had a small glass-walled office in a corner of much smaller lab. The rest of the team (more senior people) had their own offices elsewhere, but that room was the meeting place when we were working together.

My first 'real job' was in a open-plan office. Partitions were shoulder-high on a standing person and each section was about 4m square with 4 seats in it. Since teams were almost always co-located (we moved desks a lot) there was always a lot to talk about with the people in the 'pod' with me and there was not a lot of interruption from outside. Again a productive enviroment - if a little de-personalising.

That was at a client site. When we moved back to our own company offices the sections were smaller - maybe 3m on a side - with walls at waist height. It was possible to stand at one corner of the floor and look over to the other to see if someone was at their desk or not. Noisy conversations (which were not uncommon) in the next partition led to frequent interruptions and it was at this time that I really appreciated the fact that I tend to get in a couple of hours before the majority of my fellows. It was really quite difficult to do any individual work during the main part of the day.

Several years later and our company has moved to new offices and our workspaces have shrunk dramatically. No more sectioning of the space - we now have rows of desks stretching across an open space 100m across. The 'wall' between me and person in the desk facing is roughly eye-level while seated. By roughly, I mean slightly less so that I can look them in the eye if I peer around the screen.

Company management congratulate themselves on the design and how modern and efficient it is to cram as many seats as possible into the space available. I just feel like one of the cattle in a stalls - although with less room or privacy. Or more relevant, like one of a thousand monkeys banging away at a keyboard trying to produce something - even a limerick (forget about aiming for Shakespeare). I get more creative work done on the train on the way home.

I see a scene reminiscent of 1950s movies with wage-slaves, heads bent over hand-cranked calculators, under the watchful eye of the floor supervisor.
I wonder when the whole idea of a productive workplace was lost.

Letting go

I have just come off a project where a great number of things went wrong. Without going into details (I may cover later - or not, given the topic of this post) I found myself fuming most evenings.
It was apparent to me very early on that we were rushing head-long towards the edge of a cliff, but could not get any of the managers to agree. By the end of it, I had been completely marginalised and could say nothing without it being ignored or verified with someone else.
This is despite the fact that most of the concerns I raised blew up at some point and led to major issues. Indeed, throughout the project, we spent more time painting over the cracks without ever addressing the cause. The result is now in production and as beset with more problems than I expected - and still no-one is addressing the underlying issues, just patching the immediate failures.
But that is not the point, and yet again I find myself getting off topic.
The question is why I spend so much time angry and stewing about it. There are probably a number of things I could have done - although what is still not clear. But going over and over what should have been was never very helpful. Even more so now that I am not even involved any more! There is really no point in getting upset over an injustice done over a year ago when no-one else involved even remembers the situation.
[It is quite apparent that our managers never recall the past and only ever look a short distance into the future. This is not right but neither is it something that can be fixed by complaining.]
So, chalk it up as a learning experience (and I DID learn a lot) with one of the major learnings to be when to let go and when to push harder.
Now if I can only put that into practice...

The Master's Cat

There was an old master at a mountain monastery came to own a small kitten which he loved dearly.

Unfortunately the kitten would try to climb into the abbot’s lap every morning while he was meditating. To remove the distraction the abbot asked the youngest of the novices to tie the animal to a nearby tree during the morning observances.

This continued for a number of years. As the novice was promoted another took his place and made sure that the cat was tied up each morning.

Eventually the old master died and was buried with great ceremony. The next morning, the youngest novice again tied the cat to the tree. This was his task and he had not even been at the monastery when the master had first found his mediations being disturbed.

As time passed, the task of tying up the cat continued to be assigned to the newest novice until, as happens to all things, the cat itself grew old and died. By this time, no-one remembered why it needed to be tied up, only that it must be done every morning during observances. So, of course, another cat was obtained to ensure that there would always be one tied to the tree.

In such a way are traditions born.