REIsearch NEWS


Using 'Big Data' And AI To Understand The Patterns Of Our History And Tell Us About Our Future

One of the driving forces behind much of my work over the past quarter century has been how we can use massive datasets and computing platforms to help us understand global society, from the patterns that underlie our behavior to the narratives and emotions that make us human. From cataloging our past and visualizing our history to finding the patterns of history and weaving all of those narratives and stories together, I’ve long been fascinated with what becomes possible when our digital and digitized history is coupled with massive computing power and directed at the grand challenge questions around who we are, where we’ve been and where we’re heading. What might the future of data-driven exploration of humanity hold? Forbes explains perhaps the greatest contribution of the web era has been the speed with which it has ushered in the digital revolution. By creating a medium and business model for the production and curation of born-digital content at global scale, the web has reshaped the way we think about the production and consumption of information, encouraging everyone from businesses and governments down to ordinary citizens to publish their thoughts, beliefs, emotions, ideas, information and experiences to the world. The web collects all of this, serving as a single access point to society’s present thoughts and past knowledge. Simultaneously, the consumer orientation of the web and ease of publication means our past is increasingly being digitized into computerized existence. Machines can help us sift through our digitized historical record and find connections and interesting coincidences that can guide us towards at least partially restoring these long-forgotten connections. Most importantly, it doesn’t take supercomputers to do so. Most importantly, when machines allow us to search at scale across entire populations of data, we are no longer limited by human imagination. Political scientists, historians and conflict scholars have long examined the similarities in our histories, but were limited by their own creativity and imagination into examining only those connections theory suggested to them might be there. In contrast, a machine can simply cross compare every period of history against every other, performing 2.5 million correlations in 2.5 minutes and returning the results with a single line of code.

Twitter Feed