Information Overload

Information Overload:

Managing the Influx of Data in the New Economy

(This article was written to supplement Easton’s 2001 Maze Calendar.)

Data Overload

Prior to Gutenberg’s printing press, the subject of May’s maze, documents had to be copied painstakingly by hand, ensuring that only the most important manuscripts were duplicated.  But the introduction of Gutenberg’s press changed this by making it possible to reproduce even the largest document at a fraction of the cost.  This was a historically unparalleled shift in information dissemination. 

Until today.

In the past decade, we have seen the cost of disseminating information decrease again by at least an order of magnitude—instead of printing and mailing documents, we can reproduce and distribute them electronically at almost no cost. 

The result is historic, but it has not been completely positive.  Today, large quantities of data are sent without regard to cost, often overwhelming the recipient.  Consider the example of daily e-mail.   Sending e-mail is almost costless for both individual and group correspondence.  The cost of mass electronic mailings is only hundreds of dollars to e-mail millions of addresses.  But the effort involved in sorting and reading e-mail is extraordinary…and expensive.  As a result, most e-mail users wind up spending almost an hour each day wading through mountains of irrelevant, tangential, and unsolicited e-mail.  The result is similar to having an entire phone book to read every morning.  What do we do with all that data?  Usually, nothing.

Employees and customers alike face a phone book each day.  The ease of document reproduction and dissemination has affected their habits, and not always for the better.  Managing information in the new economy means more than having a larger database.  It means responding to the effects of data overload on the critical people both inside and outside the organization—employees and customers.

Diminishing Returns

The May calendar recognizes Gutenberg’s development of the printing press as a historic economic event. The results on society were indisputably positive.  So what is different today?

In Gutenberg’s time, the illiteracy rate provided a cushion to absorb the shock of the information explosion.  As document reproduction increased, the percentage of the population that could read increased accordingly.  Moreover, the cost of reproducing documents was still significant enough that the supply of information did not overwhelm the demand.  Books took hours or days to produce, rather than months or years, but because the number of readers increased, each document produced remained of high value.  Each book coming off Gutenberg’s printing press was passed from reader to reader for years, just as the hand-copied books had been.

Today we have fewer such shock absorbers, as our ability to reproduce and disseminate information has increased much more quickly than our ability to triage and analyze it.  With no accurate accounting for the cost of the time necessary to sort and absorb the information available, the amount of information we are bombarded with has become overwhelming.  Now that documents can be copied in a matter of seconds or minutes, and the number of readers is not increasing at as fast a rate, the value of each document has declined. 

Even when analytic technology catches up and allows us to spend more time reading, another problem with the current influx of information will remain:  the quality of data being transmitted is declining.  As economic disincentives to data transmission are removed, more bad data gets transmitted, reducing the average quality of each piece of data.  This was a problem in Gutenberg’s time, but is greatly magnified today.

The increasing volume of data and its decrease in quality affect both employee productivity and customer relationships.  Employees are spending more and more time sifting through e-mails and other documents, and less time working with the right information.  Customers, also barraged with information of all kinds, have less attention to give marketing material.  In short, despite the ease of transmitting data, the right data just isn’t getting through.

Managing Information

Analytic technology will be advanced enough to handle our massive amounts of data.  In the meantime, there are alternatives.  For instance, we could institute toll charges, or small economic disincentives to data reproduction and transmission.  An analogy can be drawn to the highway system, another network that has experienced tremendous congestion.  As the cost of driving declined and highway usage rose, toll charges helped ease congestion on heavily trafficked roads.  Likewise, small disincentives can help to keep data dissemination at a manageable level.

Creating toll booths is a difficult proposition for the Internet.  Is it really reasonable to expect every ISP to charge users for e-mail, Internet postage, etc.? Probably not.  But information could be managed more carefully on an organizational level, perhaps by limiting the number of messages that each employee is allowed to send each month. This would force the sender to forego less important messages, simplifying the triaging process for other employees.

Unfortunately, it is not possible for one company to control the volume of information its customers face, making customer acquisition and relationships all the more important.  With the amount of information presented increasing, mass marketing campaigns become less effective.  Knowing and understanding customer behavior—who your customers are and how to get to the top of their pile of information—becomes more important than ever before.  One-to-one marketing will not just be a possibility, it will be a necessity.

Clearly, the efficiency of faster reproduction and dissemination of documents benefits everyone.  But other effects of the technology, including lower data quality and information overload, actually decrease overall efficiency.  Eventually analytic technology will catch up with data reproduction and transmission.  In the interim, however, successful companies will use better information flow to their advantage while planning for its adverse effects.