The Horizon Report is produced each fall using a carefully constructed process that is informed by both primary and secondary research. Nearly a hundred technologies, as well as dozens of meaningful trends and challenges are examined for possible inclusion in the report each year; an internationally renowned Advisory Board examines each topic in progressively more detail, reducing the set until the final listing of technologies, trends, and challenges is selected. The entire process takes place online and is fully documented at

The process of selection, a modified Delphi process now refined over several years of producing Horizon Reports, begins each summer as the Advisory Board is convened. About half of the forty or so members are newly chosen each year, and the board as a whole is intended to represent a wide range of backgrounds, nationalities, and interests. By design, at least one-third of the Advisory Board represent countries outside of North America. To date, more than 400 internationally recognized practitioners and experts have participated. Once the Advisory Board is constituted, their work begins with a systematic review of the literature — press clippings, reports, essays, and other materials — that pertain to emerging technology. Advisory Board members are provided with an extensive set of background materials when the project begins, and then are asked to comment on them, identify those which seem especially worthwhile, and also add to the set. A carefully selected set of RSS feeds from nearly 50 leading publications ensures that these resources stay current as the project progresses, and they are used to inform the thinking of the participants through the process.

Following the review of the literature, the Advisory Board engages in the process of addressing the five research questions that are at the core of the Horizon Project. These questions are the same each year, and are designed to elicit a comprehensive listing of interesting technologies, challenges, and trends from the Advisory Board:

  1. What would you list among the established technologies that learning-focused institutions should all be using broadly today to support or enhance teaching, learning, or creative inquiry?
  2. What technologies that have a solid user base in consumer, entertainment, or other industries should learning-focused institutions be actively looking for ways to apply?
  3. What are the key emerging technologies you see developing to the point that learning-focused institutions should begin to take notice during the next three to five years? What organizations or companies are the leaders in these technologies?
  4. What do you see as the key challenges related to teaching, learning, or creative inquiry that learning-focused institutions will face during the next five years?
  5. What trends do you expect to have a significant impact on the ways in which learning-focused institutions approach our core missions of teaching, research, and service?

One of the Advisory Board’s most important tasks is to answer these five questions as systematically and broadly as possible, so as to generate a large number of potential topics to consider. As the last step in this process, past Horizon Reports are revisited and the Advisory Board is asked to comment on the current state of technologies, challenges, and trends identified in previous years, and to look for metatrends that may be evident only across the results of multiple years.

To create the 2010 Horizon Report, the 47 members of this year’s Advisory Board engaged in a comprehensive review and analysis of research, articles, papers, blogs, and interviews; discussed existing applications; and brainstormed new ones. A key criterion was the potential relevance of the topics to teaching, learning, research, or creative inquiry.

Once this foundational work was completed, the Advisory Board moved to a unique consensus-building process that uses an iterative Delphi-based methodology. In the first step, the responses to the research questions were systematically ranked and placed into adoption horizons by each Advisory Board member in a multi-vote system that allowed members to weight their selections. These rankings were compiled into a collective set of responses. From the more than 110 technologies originally considered, the twelve that emerged at the top of the initial ranking process — four per adoption horizon — were further researched. Once this "short list" was identified, the potential applications of these important technologies were further explored by higher education practitioners who were either knowledgeable about them, or interested in thinking about how they might be used. A significant amount of time was spent researching applications or potential applications for each of the areas that would be of interest to practitioners.

Each of these twelve was written up in the format of the Horizon Report. With the benefit of knowing how each topic would look in the report, the “short list” was then ranked yet again, this time with a reverse ranking approach. The six technologies and applications that emerged at the top of the rankings — two per adoption horizon — are detailed in the preceding sections, and those descriptions are the final results of this process.

An ongoing component of the project involves a special set of Delicious links that have been established to help extend the findings of the project and allow new information to be shared within the community. These Delicious tags are listed under the “Further Reading” section of each of the six topic areas, and readers are invited to view the hundreds of resources used in producing the report. These links are enhanced by a vibrant community that contributes new information daily. Readers are encouraged to be part of this community and add their own examples and readings to these dynamic lists by tagging them for inclusion in each category.

Posted by NMC on January 14, 2010
Tags: chapters

Total comments on this page: 3

How to read/write comments

Comments on specific paragraphs:

Click the icon to the right of a paragraph

  • If there are no prior comments there, a comment entry form will appear automatically
  • If there are already comments, you will see them and the form will be at the bottom of the thread

Comments on the page as a whole:

Click the icon to the right of the page title (works the same as paragraphs)


No comments yet.

[...] Methodology (0) [...]

January 16, 2010 1:17 am

[...] Four to Five Years: Visual Data Analysis Methodology [...]

May 11, 2010 1:28 pm
Name (required)
E-mail (required - never shown publicly)