Vannevar Bush was serving as head of the U.S. Office for Scientific Research and Development (OSRD) during WWII when he was confronted by a huge problem — a problem that might sound familiar today.
The problem was information overload. Scientists working on national security projects couldn't keep up with the high number of articles and textbooks being published.
Even when they managed to read the material, it was near impossible to find later. Frustrated, Bush noted that scientists were using the same indexing and retrieval methods "used in the days of square-rigged ships."
In 1945, Bush published the seminal article "As We May Think." In it, he proposed building "a memory extender" to help scientists deal with the "growing mountain of research" that was "bogging people down" as they struggled to find information.
70 Years Ago Sounds a Lot Like Today
An avid inventor, Bush envisioned what he called the "memex" — a "memory-extender" machine built into a standard office desk. The memex incorporated a scanner for inputting documents, a touch screen and stylus for adding annotations, a microfilm memory storage device for retaining the information and a mechanism for tagging and retrieving information.
This tagging/retrieval mechanism included a keypad to tag documents with what Bush called "trail codes," (i.e. topics) and a navigation switch to locate specific trail codes.
As Bush saw it, not only would tagging information bits help connect them into meaningful trails, but once captured and stored, trails wouldn't fade. Years later, the scientist could go back to their memex and instantly retrieve information about a trail by simply inputting the appropriate trail code.
In addition to the intrinsic problem of too much information, Bush felt contemporary technology compounded the pain of information overload. Technology was forcing people to think about information in ways that was hard to process. In Bush’s words, "Our ineptitude in getting at the record is largely caused by the artificiality of systems of indexing."
Bush noted that "the human mind does not work that way." Rather, "the human mind … operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain."
Bush’s memex would change all of that by presenting information the way the brain works.
While clearly a brilliant idea, Bush's memex was never built. Even when "updated" with newer technology during the 1950s and 1960s, he couldn't even get a prototype off the ground. Many reasons are behind the memex's failure, including several key technology hurdles, but clearly the memex was an idea whose time had not yet arrived.
Fast forward to today and we find ...
Contemporary Information Overload
Information overload is worse than ever now and it affects far more than just scientists.
Most workers deal with mounds of information on a daily basis, in the form of digital email, documents, videos, text notifications and task requests, each of which is delivered via an app.
The difficulty in processing information is compounded by the fact that each app offers a different user experience using dissimilar data representations. So it’s not only the growing amount of information that weighs on us, it’s also the difficulty piecing it all together so that it makes sense. Added to this is the challenge of storing it all in a smart way so all related pieces can be found later.
Could it be the 70-year-old memex is a solution for today’s information overload crisis?
What Would a 2017 Memex Look Like?
Bush’s intuition about how the human brain works in terms of topics and associations has been borne out by recent research. Using functional MRI scanning to map peoples’ brain operations while being exposed to objects, researchers now have empirical evidence that people think this way.
So maybe organizing information by topics, as Bush envisioned, is the right approach after all?
Certainly, many of the technological hurdles Bush faced are largely behind us. Capturing information is simple because emails, documents and apps use digital files that can be readily copied without human intervention. Memory storage in the cloud is both cheap and universally-available. Broadband networks for transferring information and smart mobile devices that can process and display information are both ubiquitous.
The missing piece of the puzzle is the method for tagging and retrieving information. Relying on humans to manually do this has proven impractical — until now.
Using recent advances in artificial intelligence in the form of natural language processing (NLP) and machine learning, it is now possible to accurately classify information into topics automatically.
For example, NLP engines can readily extract topics from an email message, a Salesforce record or a Word document. While no engine is perfect and matching topics across different information sources is still a bit of black magic, today’s NLP technology offers a great baseline which humans can modify.
Plus, using machine learning, automated decisions improve over time.
Welcome to the Age of Topic Computing
With this final piece of the puzzle in place, we are on the verge of a new age of how humans work with computers: the age of Topic Computing.
The intersection of the cloud, cheap memory storage, ubiquitous broadband data networks, smart mobile devices, network graph technologies like the Microsoft Graph, natural language processing and machine learning is creating a perfect storm.
In this new age, people will interact with computers by topics, rather than apps, regardless of where the information comes from or is stored. The complexity of putting together all the pieces will be done automatically and the most important pieces of what you need to see will spontaneously bubble up to the top of the stack.
Bush’s memex seems more prescient today than ever.
In 1945, Bush predicted there would be a "new profession of trailblazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record." While Bush predicted these trailblazers would be lawyers, physicians, chemists or historians, today, we know that almost everyone will be a trailblazer in this new age of Topic Computing.