2008 Horizon Report

3

Four to Five Years: Collective Intelligence

Time-to-Adoption Horizon: Four to Five Years
Two new forms of information stores are being created in real time by thousands of people in the course of their daily activities, some explicitly collaborating to create collective knowledge stores like the Wikipedia and Freebase, some contributing implicitly through the patterns of their choices and actions. The data in these new information stores has come to be called “collective intelligence” and both forms have already proven to be compelling applications of the network. Explicit knowledge stores refine knowledge through the contributions of thousands of authors; implicit stores allow the discovery of entirely new knowledge by capturing trillions of key clicks and decisions as people use the network in the course of their everyday lives.

Overview

Collective intelligence is a term for the knowledge embedded within societies or large groups of individuals. It can be explicit, in the form of knowledge gathered and recorded by many people (for example, the Wikipedia-- www.wikipedia.org— is the result of collective intelligence); but perhaps more interesting, and more powerful, is the tacit intelligence that results from the data generated by the activities of many people over time. Discovering and harnessing the intelligence in such data—revealed through analyses of patterns, correlations, and flows—is enabling ever more accurate predictions about people’s preferences and behaviors, and helping researchers and everyday users understand and map relationships, and gauge the relative significance of ideas and events.

Examples of uses for this type of intelligence already exist in industry. Google’s PageRank system, which assigns value to a web page based on the number of other pages that link to it, uses patterns discovered in hundreds of millions of links to determine which web pages are most likely to be relevant in a list of search results. Amazon.com examines patterns in hundreds of buyer variables to recommend purchases that you might like based on your previous purchases, those of your friends, and other people who may have similar tastes or preferences.

Collective intelligence applications are an outgrowth of “open data,” the practice and philosophy that certain data should, or even must be freely available to everyone (Wikipedia, “open data,” retrieved December 2007). Collective intelligence refers to knowledge that can be uncovered by combing these open data stores, and already businesses and governments are using tools to mine these storehouses; there are obvious applications to medicine, manufacturing, and economics, just to name a few disciplines.

While the approaches that enable collective intelligence have their roots in the open source movement, there are clear distinctions between the data stores that constitute collective intelligence and other approaches to open information such as the Open Educational Resources (OER) movement. Specifically, collective intelligence is by definition highly distributed, both in its implicit and explicit forms. The data are not organized in the traditional sense, and indeed it is in part the unstructured nature of collective intelligence which allows it to be created and mined in ways that often lead to multiple levels of new insights.

Relevance for Teaching, Learning, and Creative Expression

Sources of explicit collective intelligence provide opportunities for research and self-study and give students a chance to practice the construction of knowledge—they can contribute as well as consume. Social encyclopedias like the Wikipedia and others like the Cellphedia (www.cellphedia.com), are self-correcting; they tend to be more up-to-date, especially in areas such as emerging technology or pop culture, than printed sources simply because thousands of contributors are continuously and actively engaged in adding, modifying, reviewing, and updating them.

Implicit collective intelligence is already revealing a great deal about everyday patterns of activity based on programs that mine datasets of information from huge numbers of human actions—purchases, hyperlink trails, search patterns—and the kinds of activities that can be recorded while respecting individual privacy are expansive and growing. Research projects in fields like business, economics, and cultural studies already make use of data from popular search engines, media sharing sites, e-commerce sites, and even game play. Geo-based mashups of health, commercial, and other data are easy to find, and as geotagging becomes more common, geographical data will be embedded in more and more of these data, making it possible to plot almost anything on a map or track its movement over time.

In fields like astronomy and meteorology, collective intelligence has already led to new discoveries and broadened our understanding of the world. Amateur scientists both contribute to and have access to data gathered by professionals; hundreds of millions of observations exist, and discoveries are quickly disseminated. Especially in these two fields, but also in other fields that grow by sifting through mountains of observations, amateur scientists have come to be considered valuable collaborators, adding to the body of understanding and contributing new discoveries to the field.

A sampling of applications for collective intelligence across disciplines includes the following:

• Archival Science. Tagging is an accessible form of collective intelligence that offers insight to language use and conceptual associations. The Steve Museum project is researching the effect of community tagging on access to and appreciation of museum collections (www.steve.museum).
• Environmental Studies. Researchers at the University of California, San Diego have developed a prototype personal device, Squirrel, that samples air pollutants and transmits the data to a cell phone. A program on the cell phone then sends the data to a database, providing detailed information about local air quality and conditions anywhere. One possible use of such technology is to involve the community in capturing detailed climate data related to CO2 emissions, smog, ozone and other pollutants to use in earth system science and environmental studies.
• Dynamic Systems. Currently, cell phones in major cities are transparently used to monitor traffic flow on major highways; by tracking the location of a mobile device as a caller moves from cell to cell, an accurate picture of how fast the traffic is moving can be projected and displayed on a map. When viewed over time, these data show how traffic flow is akin to other dynamic systems such as the movement of sound through air, or currents in the ocean.
• History. Created through a partnership among George Mason University, the University of New Orleans, the National Museum of American History, and others, the Hurricane Digital Memory Bank (hurricanearchive.org) is a community-created archive of stories, photographs, and other digital media that preserves and presents personal experiences of Hurricanes Katrina and Rita.
• Meteorology. Small personal weather stations installed in homes and schools augment those in public safety facilities, television stations, and official weather stations at airports and other facilities to continuously monitor local weather and atmospheric data. These data are transmitted automatically at intervals to the National Weather Service, where they are used to refine micro forecasts, especially in severe weather situations. Anyone can access the information to do research based on up-to-the-minute, real world data. Companies like WeatherBug provide easy access to this information and help connect the community around it.

Examples of Collective Intelligence

The following links provide examples of collective intelligence.

Freebase
www.freebase.com
Freebase is an open, shared online database; not only is the data in it entered by the community, but the structure of the database itself (data types, categories, and so forth) is also community-created.

Google Image Labeler
images.google.com/imagelabeler/
Google Image Labeler uses a game format to gather tags for images that are then used to improve image search.

Google Zeitgeist
www.google.com/intl/en/press/zeitgeist.html
Google Zeitgeist, a year-end report of sorts, uses implicit collective intelligence to graph search terms used throughout the year to demonstrate what topics mattered most to people.

History Commons
www.cooperativeresearch.org
The History Commons is an open-content civic journalism site. Contributors add articles about events or entities, creating detailed timelines about them (e.g. the events leading up to, during, and following Hurricane Katrina). Content is submitted, reviewed, and copyedited by volunteers.

Human Brain Cloud
www.humanbraincloud.com
The Human Brain Cloud is a game that collects word associations from thousands of “players” and creates a visual map of common associations for a given word.

For Further Reading

The following articles and resources are recommended for those who wish to learn more about collective intelligence.

10 Semantic Apps to Watch
www.readwriteweb.com/archives/
10_semantic_apps_to_watch.php
(Richard MacManus, Read/Write Web, November 29, 2007.) This blog post describes ten “semantic apps,” or applications that take advantage of the kinds of data provided by collective intelligence, that are currently in development.

Panel on Collective Intelligence
mitworld.mit.edu/video/494/
(Moderated by David Thorburn, MIT World, October 7, 2007.) This panel discussion, featuring Thomas W. Malone, Alex Pentland, and Karim R. Lakhani, discusses the question of whether a group of people working with smart machines can achieve a greater degree of intelligence than humans or machines alone. Presented as a two-hour video.

Video, Education, and Open Content: Notes Toward a New Research and Action Agenda
www.firstmonday.org/issues/
issue12_4/kaufman/
(Peter B. Kaufman, First Monday, March 16, 2007.) This paper discusses the intersection of moving images, education, and open content, and suggests areas for research.

del.icio.us: Collective Intelligence
del.icio.us/tag/hz08+collectiveintelligence
(Horizon Advisory Board and Friends, 2007.) Follow this link to find resources tagged for this topic and this edition of the Horizon Report, including the ones listed here. To add to this list, simply tag resources with “hz08” and “collectiveintelligence” when you save them to del.icio.us.