Pages

    Wednesday, December 30, 2009

    Social Media: Capturing Knowledge in a Self-Organized Way

    It's the early 80s and you sit at your terminal with a stack of papers, a document holder and a keyboard. Your mission: Enter as many of the paper forms into the terminal as possible. Exciting work, isn't it?

    The problem is that this is not an inaccurate way to view data entry today. Granted, a lot of the brute force work has been done and legacy systems exist from which to pull data. Further, the forms are now entered directly into the system as opposed to copied from paper, but as regards entering novel data the situation has changed little.

    The other major problem is that this type of raw entry, which is generally entering data into a form, only captures defined phenomena. The data that is being entered, especially into a form, is often classified and defined in advance. There is no elasticity to what can be captured.

    This is problematic in that you must have a clear picture of what you are capturing in advance. For hard problems and complex situations you very rarely know much, if anything, in advance. If your only valid form of capturing data is via traditional predefined methods, such as forms, then your ability to capture data, and eventually knowledge, is vastly compromised.

    This revelation is nothing new, of course. People have been trying to innovate data entry and knowledge capture for several decades. But, what other types of data can be captured and how?

    The Army is asking this exact question, if indirectly. In reading several SBIRs the concept of capturing the knowledge inherent in soldiers heads is coming to the forefront. It is being recognized that not only do experts have valid perspectives and answers, the boots on the ground do, as well (keep in mind, this is probably not a new perspective in the military, but is one that I have seen in several SBIRs recently). Beyond that, though, they are starting to explore how to bring that knowledge into existing systems.

    The how of this is a serious question. Computer systems today are clearly defined and generally purposeful to a single end. Human thought, on the other hand, is often multi-purposed and the field of understanding human thinking (philosophy) has been around for as long as humans and has yet to reach one shared conclusion on how we think. Even if we could get some Matrix-like data jack implanted into soldiers heads could we really transfer the knowledge as it is represented inside their brains into a computer system?

    Perhaps we need another way of gathering data since it would seem that direct access to the human mind would avail us little. There is a new(-ish) movement which is providing an answer: social media.

    Probably one of the most useful aspects of social media is how important and interesting knowledge percolates to the top. This is done in various ways. For instance, if I see an interesting tweet on Twitter, I will retweet it. If I read something worthwhile on Facebook, I may comment on it or repost it. It's this interaction with the content that causes the interesting bits to rise to the top.

    The exciting thing, at least from a systems perspective, is that this is self-organizing behavior. It is through the interaction of the components of the system (here, the components are the people) that the interesting bits are being obtained. While it may be difficult to capture human thought and knowledge in its native form, it's not as difficult to capture the important pieces as they are being defined by the social system already.

    Further, the nature of social media, in that it tends to interact in bite-sized, discrete pieces, means that the computer system needs not have much understanding of what it is capturing at all. The knowledge is already distilled into its core component, often with attribution, and the computer system merely need remember it. It can be stored without pre-defined labels and fields.

    The thing which the computer system must crucially provide is a robust search capability. Whether this search capability is enacted after the fact, or whether there is a component of the system which searches as knowledge comes in is immaterial. As long as the system can search through the knowledge is what's important.

    Eventually, this captured knowledge can be used and reused as more people interact with it. Each interaction would in essence refine the knowledge, making it more useful to the computer system and the people in general.