2009 Horizon Report: The K12 Edition

Methodology

The process used to research and create the Horizon Report: 2009 K-12 Edition is very much rooted in the methods used to develop the global edition of the Horizon Report that is released each January. All editions of the Horizon Report are produced using a carefully constructed process that is informed by both primary and secondary research. Nearly a hundred technologies, as well as dozens of meaningful trends and challenges are examined for possible inclusion in the report for each edition. Every report draws on the considerable expertise of an internationally renowned Advisory Board that first generates a broad set of important emerging technologies, challenges, and trends, and then examines each of them in progressively more detail, reducing the set until the final listing of technologies, trends, and challenges is selected.

Much of the process takes place online, but when new editions are created for the first time, a face-to-face meeting is often also part of the process, as a way to quickly build an engaged community among the advisors. All the work, wherever it occurs, is captured and placed in the Horizon Project wiki, which is the project’s home on the web. The Horizon wiki is intended to be a completely transparent window to the process, and contains the entire record of the research. The wiki for the K-12 Edition can be found at http://horizon.nmc.org/k12.

The procedure for selecting the topics that will be in the report includes a modified Delphi process now refined over years of producing Horizon Reports, and it begins with the assembly of the Advisory Board. The board as a whole is intended to represent a wide range of backgrounds, nationalities, and interests, yet each member brings a particularly relevant expertise. To date, more than 275 internationally recognized practitioners and experts have participated in a Horizon Project Advisory Board; in any given year, as many as half the members are new.

Once the Advisory Board for a particular edition is constituted, their work begins with a systematic review of the literature — press clippings, reports, essays, and other materials — that pertains to emerging technology. Advisory Board members are provided with an extensive set of background materials when the project begins, and are then asked to comment on them, identify those that seem especially worthwhile, and add to the set. The group discusses existing applications of emerging technology and brainstorms new ones. A key criterion for the inclusion of a topic is the potential relevance of the topic to teaching, learning, research, or creative expression. A carefully selected set of RSS feeds from at least a dozen relevant publications ensures that these background resources stay current as the project progresses, and they are used to inform the thinking of the participants throughout the process.

Following the review of the literature, the K-12 Advisory Board engaged in the central focus of the research — the five research questions that are at the core of the Horizon Project. These questions were designed to elicit a comprehensive listing of interesting technologies, challenges, and trends from the Advisory Board:

1. What would you list among the established technologies that schools should all be using broadly today to support or enhance teaching, learning, or creative expression?
2. What technologies that have a solid user base in consumer, entertainment, or other industries should schools be actively looking for ways to apply?
3. What are the key emerging technologies you see developing to the point that schools should begin to take notice during the next three to five years? What organizations or companies are the leaders in these technologies?
4. What do you see as the key challenges related to teaching, learning, or creative expression that schools will face during the next five years?
5. What trends do you expect to have a significant impact on the ways in which schools approach our core missions of teaching, research, and service?

One of the Advisory Board’s most important tasks is to answer these five questions as systematically and broadly as possible, so as to generate a large number of potential topics to consider. To help with this, past Horizon Reports are revisited and the Advisory Board is asked to comment on the current state of technologies, challenges, and trends identified in previous years, and to look for metatrends that that may be evident only across the results of multiple years.

At this point, the Advisory Board is asked to generate as many new responses to the questions as possible, and also to comment on the existing responses. The regional and sector-based reports add one additional step as a way to seed the responses: the topics from the short lists of the global and other regional editions for the current year are included in the list of topics to consider. Once this work is done, usually within just a few days, the Advisory Board moves to a unique consensus-building process that uses an iterative Delphi-based methodology.

In the first step of this approach, the responses to the research questions are systematically ranked and placed into adoption horizons by each Advisory Board member using a multi-vote system that allows members to weight their selections. Each member is asked to also identify the timeframe during which they feel the technology would enter mainstream use — defined for the purpose of the project as about 20% of institutions adopting it within the period discussed. (This figure is based on the research of Geoffrey A. Moore and refers to the critical mass of adoptions needed for a technology to have a chance of entering broad use.) These rankings are compiled into a collective set of responses, and inevitably, the ones around which there is the most agreement are quickly apparent.

From the more than 100 technologies originally considered for any report, the twelve that emerge at the top of the initial ranking process — four per adoption horizon — are further researched. Once this “short list” is identified, the group, working with both NMC staff and practitioners in the field, begins to explore the ways in which these twelve important technologies might be used in for teaching, learning, research, and/or creative expression. A significant amount of time is spent researching real and potential applications for each of the areas that would be of interest to practitioners.

For every edition, when that work is done, each of these twelve “short list” items is written up in the format of the Horizon Report. With the benefit of the full picture of how the topic will look in the report, the “short list” is then ranked yet again, this time in reverse. The six technologies and applications that emerge are those detailed in the Horizon Report.

For additional detail on the project methodology or to review the actual instrumentation, the ranking, and the interim products behind the report, please visit http://horizon.nmc.org/k12.