Well, I have to admit it. I am really, really, really impressed with Livescribe. This is the single most important, must-have tool for me in my professional life. I have been an Apple guy since my first Powerbook 520c (in the mid 1990’s) but put off the purchase of a MacBook Pro because of the lack of availability of Livescribe Desktop.

If you haven’t used Livescribe before, let me explain. Imagine a cross between a Moleskine notebook, a digital voice recorder, and Tivo. Based on a specialized pen and paper, it allows you to automatically transfer any notes you take onto your computer, where you can search through them or share them as PDF’s. In addition, the pen can record sound as you are writing, which gives a much more human element to your notes. Best of all, these notes can be shared remotely with other people through the power of the web. You really have to see it to believe it.

Let me go on a bit about why I value Livescribe so much …

Flexibility

Livescribe has the basic flexibility of pen and paper, and that can’t be overrated. Back in the day I was that person who brought his laptop to meetings to type up notes and send them out to everyone. It seemed cool, and nice, and it was good to be able to refer to things later. The problem was, it was deceptively ineffective. I was confusing motion for action. If someone is presenting, I may need to take notes. If they are drawing, I may need to do a quick sketch. If they are talking (especially emotionally) I need to understand.

This just can’t be done typing into any text based tools, and I have tried a lot of them (I used the heck out of VoodooPad and heartily recommend it, but not more than paper). I can’t stop in the middle of a meeting to open a drawing program to try and copy what is on the whiteboard. Even if I could, I couldn’t go back and examine the flow of the meeting, determine what the key points were and what I might have missed or could have done better.

Accessibility

Livescribe has the accessibility of both paper and computer. I use a Moleskine-sized notebook as my “daybook”, to record who I met, what I did, and things I need to do in the future. I also take notes on meetings and do a lot of brainstorming before I start coding. This allows me to really focus on the important things (people, concepts, ideas) and not get wrapped up in time-wasters. This isn’t something unique to Livescribe, I got the same thing with the moleskine or a plain pad of paper.

The dramatic advantage that Livescribe has over paper is that I can go back and search my notes for previous entries or words or phrases that might pertain to whatever I am thinking about today. I do that a lot less with Livescribe than I thought I would, simply because I seem to remember things better if I write them down. However, the times that I have had to do a search it has been absolutely invaluable.

Recording and Sharing

The most impressive feature of the system is the ability to share your notes with others. While some people would use this for class notes and the like, I am long out of college and working in the world. As a Sales Engineer I have to travel a lot and visit clients and prospects in pretty remote locations. I don’t always have my Account Executive with me, but he definitely needs to know what is going on whenever possible.

This is where Livescribe really proves its worth. The ability to share a recording of a meeting (along with detailed notes) is awesome. When I get on a plane I can review a meeting myself, and later send it to my partner who can know just about everything that went on in the room, with enough detail to understand what was verbalized and how.

In Conclusion

Livescribe really gives you the best of both worlds, paper and electronic. It fully lives in both media, but combines the two in a way that is greater than the sum of the parts. Sure, I have some grumbles (the pen is large, I wish there were more paper options, the pen doesn’t have a top and there isn’t a good, small pen case available, it is harder than it should be to search across multiple notebooks, you can’t group pages or use metadata with the electronic text) but overall it is invaluable to me.

You will have to pull Livescribe from my cold, dead fingers!

Frank Dravis has a new post on metadata and its growing importance in the marketplace. Dravis wonders how metadata became a subject of interest for non-technical folk …

In years past, metadata was the domain of data architects. It helped them understand what data they had and how it related to the sources and operations from which it came and to which it went. At the first mention of metadata business users would roll their eyes and head for the conference room door. Surely metadata was the stuff of arcane IT discussions best had out of earshot of people driving and running the business.

Then metadata management progressed and someone had the silly idea of articulating the business value, the value to the business side of the house, for metadata. The value came from the resolution of an age old problem. A corporate manager is sitting in a conference room looking at their regular monthly sales report and it is different from what they expected based on anecdotal evidence from the field: the numbers are too low.

Personally, I think that this recent interest is driven by a few things:

1. Regulation and the threat of real penalties for inaccuracies in reporting. People got interested enough to protect their own hides.
2. The rise of ERP and BPM in the marketplace. If everything is in one place then metadata suddenly becomes a lot easier to manage.

Truthfully, I wonder how all of this is going to turn out. I know there are lots of people who want to sell metadata software, but in my experience it takes a lot of resource (time, effort, and expertise) to maintain a comprehensive metadata environment. The threat of jail time helps to keep people motivated enough to save their necks, but not enough to make something useful. Being locked into an ERP package can mean the same thing, only it is your data that is locked.

Share and earn some karma ...These icons link to social bookmarking sites where readers can share and discover new web pages.

• 
• 
• 
• 
• 
• 

Introduction

When an organization begins a concerted effort to improve its information quality, often it gets stuck in trying to figure out exactly where to start. This case study takes this to heart and gives a specific example of an approach to improving information quality.

Previously, we had discussed the semantic and statistical approaches to information quality and linked them to black box and white box testing. In addition, there is a case study on semantic information quality which is used to contrast this case study (you may want to take a look at these if you aren’t familiar with the subjects).

The example that we have been using is …

dealing with call data for a customer contact center. For simplicity, we can assume that all the call data we need is delivered nightly and is loaded into a single table that looks exactly like the files as they have arrived. This table has the following attributes:

• employee_login_number
• site_name
• department_name
• call_local_start_time
• call_local_end_time

… from this data the business analysts are going to figure out how much to pay and to whom. Also, we need to figure out who is handling the highest call volume (vendors, locations, and employees) on a daily basis so that we can resolve issues and negotiate contracts. Our job is to make sure that the data is accurate enough to do this with confidence.

Also, before we get started, realize that with the semantic and statistical approaches we are trying to do the same thing in different ways. So, while we are doing things differently, there is bound to be some overlap.

The Statistical Approach

With a statistical approach, there are several things to consider:

1. From a statistical point of view, there is nothing special about this dataset. It has very similar characteristics to all the ones that came before it and will come after it. We should try to create an architecture that can be re-used where appropriate.
2. There is a lot that we can infer from the dataset itself. We can learn a great deal of information about the dataset very cheaply through black box testing. Focusing on these areas will maximize re-use as well.
3. We can probably assume that any data that we recieve is of reasonably good quality when the process was first designed. Therefore, we can focus on events where the nature of the data changes substantially.

With these in mind, we can start to design a solution.

The place to start is to ask, “what can go wrong in our data?”. I can think of several situations that might impact the quality of this data:

• The employee_login_number is invalid or NULL.
• The site_name is invalid or NULL.
• The department is invalid or NULL.
• The call_local_start_time is invalid or NULL.
• The call_local_end_time is invalid, NULL, or starts before the call_local_start_time.
• Due to errors outside of our control, the process that created the data malfunctioned. Often, this will show up as duplicate values, irregular frequency or distribution of values

Off the top of my head, I have a number of questions about the data that we will see day to day:

• For each column, is there a distinct list of values (call this the domain) that are valid?
• For each column, is there a distinct pattern of values that are valid?
• For each column, can the values be NULL?
• Is there a distinct key? If so, is it unique?
• For column values and keys, should the frequency for particular values be fairly normal?
• Is there a certain number of rows that should be expected (by key or for the entire dataset)?
• Is there a certain number of keys that should be expected?
• For numeric values, can we do descriptive statistics to tell us if things are off-kilter?

Based on these, I think that we can establish a data model that would allow this metadata to be recorded for multiple processes, which would allow it to be used for reporting and decision-making.

For example, consider a table having the following attributes:

• process_id
• process_run_dt
• distinct_value
• distinct_value_type
• distinct_value_count

This would allow the user to keep track of how many distinct values there were generated by a given process. Over time, this could be very useful in tracking down some sticky problems, and perhaps prevent bad data from ever getting into a data store in the first place.

For each of the meausurement processes we mentioned, they can probably be integrated into the overall data model in a process agnostic way. I apologize for not having more details at this point, I plan to move this to the wiki (at some point) and put in a reference model for doing some of these operations.

Comparisons With Data Profiling

For people with some experience with data management this may sound a lot like data profiling. In fact, a lot of the operations inherent in the statistical approach would probably be considered a part of data profiling as well.

However, there are some key differences between Statistical IQ and Data Profiling that need mentioning:

1. Statistical IQ has an operational focus and needs to be as lightweight as possible. We want to use this to make day-to-day operational decisions about our data without slowing anything down.
2. Statistical IQ does not include data discovery, while data profiling often does.
3. One of the core functions of data profiling is establishing relationships between datasets. Statistical IQ has a very limited view of relationships in order to maximize functionality and reusability.

Similar base concepts, focusing on different areas.

Statistical IQ and Mad Libs

One thing that often gets lost in the re-use discussion is the price of user configuration. All too often, programmers push too much decision making out of their code and on to the operator, making it difficult to use.

The trick with Statistical IQ is that you have to be able to tie a generic statement (”there are 15 distinct values in this dataset”) back to something useful (”there is probably missing data, don’t continue the process”). While this might seem like a challenge, it can be done without a lot of heartburn.

In a recent engagement, I designed a solution where we tied every possible error back to an english description of the problem that was stored in an SQL database. This was done in a very generic way, so that new errors could be added or removed without any configuration required by the developer or operator.

Conclusion

There are different approaches to information quality, each with their own strengths, weaknesses, and costs. The statistical approach is cheaper (especially when you factor in Moore’s Law), but gives a less detailed picture of overall quality. The semantic approach is more expensive, but can be as comprehensive as the situation requires. A balanced approach will use both approaches to deliver the solution that is needed.

In Information Science today two competing methods for indexing information: semantics and statistics. While this may not seem to have a lot to do with information quality, bear with me and I promise I will link them up (eventually). Both methods approximately the same job, that is to allow information to be read and manipulated by machines on a grand scale. The difference is in how this is done.

• A semantic approach would have the author define concepts and relationships ahead of time. You can see some examples in this tutorial, as they are long and would be difficult to reproduce here. The Semantic Web would be a good example of this methodology.

• A statistcal approach would simply look at the text that was available and try to determine what is there and how it relates to other things through textual analysis and aggregation. Google is a good example of the use of this approach.

The semantic way of looking at things is very abstract and much more rigorous. It says that there is a truth to be represented, it designs a way of doing it, and expects everyone to follow along. The statistical way of looking at things is much more flexible. It says that there are things to be gleaned regardless of form, and that we should accept this fact and try to make the best of things. Not surprisingly, the semantic approach is the favorite of academia and has been under development for many years, while the statistical approach is already in real-world use.

What got me thinking about this in the first place was the latest issue of Baseline. Specifically, it was an article from Paul A. Strassman titled, “How Clean Data Can Transform Your Business”. Normally Strassman’s stuff is pretty good, but it is helpful to note that Strassman is a senior consultant to the Department of Defense and has been in the business for a long, long, long time.

The crux of his argument was that:

The first step in business transformation: enterprisewide standardization of data. That calls for the declaration of a metadata directory as the template for defining data that can circulate within a firm’s information systems. The policy and implementation of an enforceable metadata directory likely will be resisted by bureaucrats, who see this as a threat to their indispensability. It will not be welcomed by systems developers, contractors and vendors, who prefer to concentrate on upgrading software as a technologically more interesting—and profitable—task.

A classic argument for a semantic model of truth. We just need to get everything defined and then it will be smooth sailing from there. For most vendors and consultants, the semantic view is the accepted one, probably because it is so structured and logical, although at least partially because it all those hours spent defining concepts are billable. Even Strassman acknowledges this reality …

To reach agreement on the representation, semantics and taxonomy of data, you will likely go through a painful political process that must be adjudicated by line management. This can get messy because it will reveal that a large percentage of installed software perpetuates incompatible, unreliable, insufficiently secure and delayed information.

With this in mind, is semantic definition the most efficient way to improve information quality? Is a statistical definition the most descriptive way to understand information quality? We will explore the basis for both of these methods in the next part of this series.

I have been working on some more detailed articles for the wiki to help illustrate some ideas about information quality. While I don’t want to just duplicate that article here, I thought I would post some things on the blog and get some feedback.

I am currently working on the Information Quality Pyramid, which discusses the various components that go into improving information quality across an organization:

The pyramid is made up of several parts, each of which are important in their own right. However, the base components (in blue in green) have the interesting combination of being very important, terribly inexpensive, and totally unglamorous.

Understanding Your Organization – The single most important thing that you can do to ensure success in any information quality effort. Without a solid understanding of how your organization works it is virtually guaranteed that you will not be able to deliver the solution your customers need. This (coupled with the need for extreme customization) is one of the reasons that it is very difficult to outsource this type of work.

Architecture and Design Practices – To put it bluntly, if you build your information architecture in an inconsistent manner then you have to expect inconsistencies in its output. These inconsistencies become quality-related issues very quickly. If you can proactively address (or at least mitigate issues around) consistency through your architecture then you can dramatically improve the quality of information that you produce.

Automation – The key to high-value, high-quality information architecture is automating everything possible. This is because:

1. Moore’s Law will double the speed of computerized processes every two years. It is pretty tough for humans to keep up.
2. Humans make mistakes.
3. In order to automate a process, it has to be understood by more than just the designer or the developer.

Sanity Checks – The easiest and most cost effective ways to catch issues before they become problems.

Data Profiling – The only way to understand your information is to know your data. Intimately. Regularly. Historically. Profiling takes a generic look at an arbitrary dataset and discovers important statistical information about it. Profiling is by far the cheapest and most reliable way of examining data (think of it as an expanded sanity check).

Process Testing – Instead of looking at a dataset in a generic way, process testing looks at things in a very specific way. These should be customized tests that will tell information that are automated and deliver results that are unique to the process. Because of the level of customization and effort, this is significantly more expensive than profiling or sanity checks.

Human Intervention – Anything that involves humans, from adjusting processes already in production to performing manual analysis to resolve concerns to creating new code. Think of it as if all of information quality was outsourced to a 3rd party company and all personnel costs came directly out of your budget. This is the true cost of IQ, it is just that people see it in a more abstract sense.

The one category that I can see people might think is missing here is metadata. I think metadata is an incredibly important part of information quality, but I tend to value it in its most concrete form instead of in the abstract. I will get into this more in the wiki article.
Any feedback would be most appreciated!

technorati tags:information architecture, information quality, data quality, automation, metadata