Table of Contents
Comments
Commenters1
Time-to-Adoption Horizon: Four to Five Years
New forms of analysis are making use of the visual centres in our brains and the ways of thinking described in Malcolm Gladwell’s best seller, Blink, to marshal the tremendous human capacity to discern and recognize patterns. Connections and insights that are not readily apparent in traditional tables of numbers, or in standard forms of quantitative study like correlations become obvious when portrayed visually using new techniques derived from the study of fluid dynamics and other complex systems. Visual data analysis has applications in science and engineering, and offers considerable promise for understanding complex social processes like learning that have proven difficult to explore with traditional methods.
Time-to-Adoption Horizon: Four to Five YearsNew forms of analysis are making use of the visual centres in our brains and the ways of thinking described in Malcolm Gladwell’s best seller, Blink, to marshal the tremendous human capacity to discern and recognize patterns. Connections and insights that are not readily apparent in traditional tables of numbers, or in standard forms of quantitative study like correlations become obvious when portrayed visually using new techniques derived from the study of fluid dynamics and other complex systems. Visual data analysis has applications in science and engineering, and offers considerable promise for understanding complex social processes like learning that have proven difficult to explore with traditional methods.
3
Over the past century, data collection, storage, transmission, and display has changed dramatically, and scholars have undergone a profound transformation in the way they approach data-related tasks. Data collection and compilation is no longer the tedious, manual process it once was, and tools to analyse, interpret, and display data are increasingly sophisticated, and their use routine in many disciplines. The options for illustrating trends, relationships, and cause and effect have exploded, and it is now a relatively simple matter for anyone to do the sorts of analyses that were once only the province of statisticians and engineers.
Over the past century, data collection, storage, transmission, and display has changed dramatically, and scholars have undergone a profound transformation in the way they approach data-related tasks. Data collection and compilation is no longer the tedious, manual process it once was, and tools to analyse, interpret, and display data are increasingly sophisticated, and their use routine in many disciplines. The options for illustrating trends, relationships, and cause and effect have exploded, and it is now a relatively simple matter for anyone to do the sorts of analyses that were once only the province of statisticians and engineers.4
In advanced research settings, scientists and others studying massively complex systems generate mountains of data, and have developed a wide variety of new tools and techniques to allow those data to be interpreted holistically, and to expose meaningful patterns and structure, trends and exceptions, and more. Researchers that work with data sets from experiments or simulations, such as computational fluid dynamics, astrophysics, climate study, or medicine draw on techniques from the study of visualization, data mining, and statistics to create useful ways to investigate and understand what they have found.
In advanced research settings, scientists and others studying massively complex systems generate mountains of data, and have developed a wide variety of new tools and techniques to allow those data to be interpreted holistically, and to expose meaningful patterns and structure, trends and exceptions, and more. Researchers that work with data sets from experiments or simulations, such as computational fluid dynamics, astrophysics, climate study, or medicine draw on techniques from the study of visualization, data mining, and statistics to create useful ways to investigate and understand what they have found. 5
The blending of these disciplines has given rise to the new field of visual data analysis, which is not only characterized by its focus on making use of the pattern matching skills that seem to be hard-wired into the human brain, but also in the way in which it facilitates the work of teams collaborating to tease out meaning from complex sets of information. While the most sophisticated tools are still mostly found in research settings, a variety of tools are emerging that make it possible for almost anyone with an analytical bent to easily interpret all sorts of data.
The blending of these disciplines has given rise to the new field of visual data analysis, which is not only characterized by its focus on making use of the pattern matching skills that seem to be hard-wired into the human brain, but also in the way in which it facilitates the work of teams collaborating to tease out meaning from complex sets of information. While the most sophisticated tools are still mostly found in research settings, a variety of tools are emerging that make it possible for almost anyone with an analytical bent to easily interpret all sorts of data. 6
Self-organizing maps are an approach that mimics the way our brains organize multi-faceted relationships; they create a grid of “neuronal units” such that neighbouring units recognize similar data, reinforcing important patterns so that they can be seen. Cluster analysis is a set of mathematical techniques for partitioning a series of data objects into a smaller amount of groups, or clusters, so that the data objects within one cluster are more similar to each other than to those in other clusters. Visual, interactive principal components analysis is a technique once only available to statisticians that is now commonly used to identify hidden trends and data correlations in multidimensional data sets. Gapminder (http://www.gapminder.org/), for example, uses this approach in its analysis of multivariate datasets over time.
Self-organizing maps are an approach that mimics the way our brains organize multi-faceted relationships; they create a grid of “neuronal units” such that neighbouring units recognize similar data, reinforcing important patterns so that they can be seen. Cluster analysis is a set of mathematical techniques for partitioning a series of data objects into a smaller amount of groups, or clusters, so that the data objects within one cluster are more similar to each other than to those in other clusters. Visual, interactive principal components analysis is a technique once only available to statisticians that is now commonly used to identify hidden trends and data correlations in multidimensional data sets. Gapminder (http://www.gapminder.org/), for example, uses this approach in its analysis of multivariate datasets over time.7
These sorts of tools are now finding their way into common use in many other disciplines, where the analytical needs are not necessarily computational; visualization techniques have even begun to emerge for textual analysis and basic observation. Many are free or very inexpensive, bringing the ability to engage in rich visual interpretation to virtually anyone.
These sorts of tools are now finding their way into common use in many other disciplines, where the analytical needs are not necessarily computational; visualization techniques have even begun to emerge for textual analysis and basic observation. Many are free or very inexpensive, bringing the ability to engage in rich visual interpretation to virtually anyone. 8
Online services such as Many Eyes, Wordle, Flowing Data, and Gapminder accept uploaded data and allow the user to configure the output to varying degrees. Many Eyes, for instance, allows people to learn how to create visualizations, to share and visualize their own data, and to create new visualizations from data contributed by others. Some, like Roambi, have mobile counterparts, making it easy to carry interactive, visual representations of data wherever one goes. Even quite public data, such as the posts made in Twitter, can be rendered visually to reveal creative insights. For instance, the New Political Interfaces project visualized political topics from 2009 as expressed on Twitter, charting which topics were — and were not — being discussed by politicians, news outlets, and other sources.
Online services such as Many Eyes, Wordle, Flowing Data, and Gapminder accept uploaded data and allow the user to configure the output to varying degrees. Many Eyes, for instance, allows people to learn how to create visualizations, to share and visualize their own data, and to create new visualizations from data contributed by others. Some, like Roambi, have mobile counterparts, making it easy to carry interactive, visual representations of data wherever one goes. Even quite public data, such as the posts made in Twitter, can be rendered visually to reveal creative insights. For instance, the New Political Interfaces project visualized political topics from 2009 as expressed on Twitter, charting which topics were — and were not — being discussed by politicians, news outlets, and other sources.10
As stated previously, one of the most compelling aspects of visual data analysis is in the ways it augments the natural abilities humans have to seek and find patterns in what they see. By manipulating variables, or simply seeing them change over time (as Gapminder has done so famously), it is easy to discover whether or not patterns exist. Such tools have applicability in nearly every field.
As stated previously, one of the most compelling aspects of visual data analysis is in the ways it augments the natural abilities humans have to seek and find patterns in what they see. By manipulating variables, or simply seeing them change over time (as Gapminder has done so famously), it is easy to discover whether or not patterns exist. Such tools have applicability in nearly every field. 11
As the tools, their capabilities, and their variety continue to expand, their use is already making its way out of scientific and engineering labs and into business and social research. Creative enquiry is benefiting from a wide range of new tools that are exposing trends and relationships among both qualitative and quantitative variables in real time, and making longitudinal relationships easier to find and interpret than ever. Textual analysis is an area that tools like Wordle have revealed as especially suited to visual techniques.
As the tools, their capabilities, and their variety continue to expand, their use is already making its way out of scientific and engineering labs and into business and social research. Creative enquiry is benefiting from a wide range of new tools that are exposing trends and relationships among both qualitative and quantitative variables in real time, and making longitudinal relationships easier to find and interpret than ever. Textual analysis is an area that tools like Wordle have revealed as especially suited to visual techniques. 12
The promise for teaching and learning is further afield, but because of the intuitive ways in which it can expose complex relationships to even the uninitiated, there is tremendous opportunity to integrate visual data analysis into undergraduate research, even in survey courses. Models of complex processes in quantum physics, organic chemistry, medicine, or economics are just a few of the ways in which the outcomes of visual data analysis can be applied to learning situations.
The promise for teaching and learning is further afield, but because of the intuitive ways in which it can expose complex relationships to even the uninitiated, there is tremendous opportunity to integrate visual data analysis into undergraduate research, even in survey courses. Models of complex processes in quantum physics, organic chemistry, medicine, or economics are just a few of the ways in which the outcomes of visual data analysis can be applied to learning situations. 13
Visual data analysis may help expand our understanding of learning itself. Learning is one of the most complex of social processes, with a myriad of variables interacting in highly involved ways, making it an ideal focus for the search for patterns. Related to this is the opportunity to understand the variables influencing informal learning and the social networking processes at work in the formation of learning communities. The tools for such analyses exist today; what is needed are ways to balance privacy with the kinds of data capture that can inform such work.
Visual data analysis may help expand our understanding of learning itself. Learning is one of the most complex of social processes, with a myriad of variables interacting in highly involved ways, making it an ideal focus for the search for patterns. Related to this is the opportunity to understand the variables influencing informal learning and the social networking processes at work in the formation of learning communities. The tools for such analyses exist today; what is needed are ways to balance privacy with the kinds of data capture that can inform such work.15
- Engineering. Massive amounts of data generated through research, system monitoring, plant operation, or other standard processes can be overwhelming, especially to students who are learning how to analyse and interpret such data. Visual data analysis provides a way for engineering students to come to grips with the complexities of the systems they are trying to master.
- Medicine. Created by the Commonwealth Scientific and Industrial Research Organisation (CISRO) Australia, MILXview is a rapid visualization tool for medical imagery that assists practitioners in quickly analysing and diagnosing cases.
- Research. The Visual Understanding Environment (VUE) created at Tufts University enables students and faculty to gather, sort, and make sense of large amounts of electronic content in their work. The visualizations can be annotated, and users can create and save paths through them to make guided walk-throughs.
18
Christchurch Quake Map
http://www.christchurchquakemap.co.nz
This map visualizes the 2010 Christchurch earthquake and its aftershocks, stepping through them to show how the aftershocks grow and recede over time. As new quakes and aftershocks occur, they are added to the map.
Christchurch Quake Maphttp://www.christchurchquakemap.co.nz
This map visualizes the 2010 Christchurch earthquake and its aftershocks, stepping through them to show how the aftershocks grow and recede over time. As new quakes and aftershocks occur, they are added to the map.
19
Google Public Data Explorer
http://www.google.com/publicdata/home
Built using the technology that powers Gapminder, Google Public Data Explorer enables users to create custom visualizations comparing different variables using public data.
Google Public Data Explorerhttp://www.google.com/publicdata/home
Built using the technology that powers Gapminder, Google Public Data Explorer enables users to create custom visualizations comparing different variables using public data.
20
San Francisco Crimespotting
http://sanfrancisco.crimespotting.org
Crimespotting is an interactive map of crimes in San Francisco and Oakland that visualizes crime by location, type, date, and time, allowing users to grasp patterns and trends.
San Francisco Crimespottinghttp://sanfrancisco.crimespotting.org
Crimespotting is an interactive map of crimes in San Francisco and Oakland that visualizes crime by location, type, date, and time, allowing users to grasp patterns and trends.
21
Truthy
http://truthy.indiana.edu
Truthy analyses Twitter posts to identify memes, aiding in the study of social epidemics and helping users to distinguish between actual organic memes and those planted by marketing campaigns.
Truthyhttp://truthy.indiana.edu
Truthy analyses Twitter posts to identify memes, aiding in the study of social epidemics and helping users to distinguish between actual organic memes and those planted by marketing campaigns.
22
VisualComplexity
http://www.visualcomplexity.com/vc
This project is intended to help further the field of visual data analysis by exploring and collecting visualization tools, best practices, and examples.
VisualComplexityhttp://www.visualcomplexity.com/vc
This project is intended to help further the field of visual data analysis by exploring and collecting visualization tools, best practices, and examples.
24
The following articles and resources are recommended for those who wish to learn more about visual data analysis.
The following articles and resources are recommended for those who wish to learn more about visual data analysis.25
5 Tools for Online Journalism, Exploration and Visualization
http://www.readwriteweb.com/cloud/2010/10/10-tools-for-online-journalism.php
(Alex Williams, ReadWriteWeb, 2 October 2010.) This article describes five tools that can be used to visualize data.
5 Tools for Online Journalism, Exploration and Visualization http://www.readwriteweb.com/cloud/2010/10/10-tools-for-online-journalism.php
(Alex Williams, ReadWriteWeb, 2 October 2010.) This article describes five tools that can be used to visualize data.
26
Data Visualization Usages During the Australian Federal Election
http://www.yellowfinbi.com/YFCommunityNews.i4?newsId=98610
(Lachlan James, Yellowfinbi, 26 August 2010.) This article explains how visualizations were used to display the results of recent Australian elections.
Data Visualization Usages During the Australian Federal Election http://www.yellowfinbi.com/YFCommunityNews.i4?newsId=98610
(Lachlan James, Yellowfinbi, 26 August 2010.) This article explains how visualizations were used to display the results of recent Australian elections.
27
Information Aesthetics
http://infosthetics.com
(Andrew Vande Moere, Accessed 17 October 2010.) Information Aesthetics is a collection of data visualizations — both hand-drawn information design and machine-visualized datasets — curated by a Senior Lecturer at the University of Sydney and K. U. Leuven University in Belgium.
Information Aesthetics http://infosthetics.com
(Andrew Vande Moere, Accessed 17 October 2010.) Information Aesthetics is a collection of data visualizations — both hand-drawn information design and machine-visualized datasets — curated by a Senior Lecturer at the University of Sydney and K. U. Leuven University in Belgium.
28
Report from the DOE/ASCR Workshop on Visual Analysis and Data Exploration at Extreme Scale
http://www.sci.utah.edu/vaw2007/DOE-Visualization-Report-2007.pdf
This report describes fundamental research in visualization and analysis that is enabling knowledge discovery from computational science applications at extreme scale.
Report from the DOE/ASCR Workshop on Visual Analysis and Data Exploration at Extreme Scalehttp://www.sci.utah.edu/vaw2007/DOE-Visualization-Report-2007.pdf
This report describes fundamental research in visualization and analysis that is enabling knowledge discovery from computational science applications at extreme scale.
29
Visualizing
http://www.visualizing.org
(Accessed 10 October 2010.) Visualizing is a community of practice for designers, teachers, students, researchers, and others interested in visual data analysis. The community is invited to showcase work and best practices, post sharable academic resources, and engage in dialogue about the field.
Visualizinghttp://www.visualizing.org
(Accessed 10 October 2010.) Visualizing is a community of practice for designers, teachers, students, researchers, and others interested in visual data analysis. The community is invited to showcase work and best practices, post sharable academic resources, and engage in dialogue about the field.
30
Visualizing Climate Change Impact with Ubiquitous Spatial Technologies
http://www.csdila.unimelb.edu.au/publication/conferences/Visualizing_Climate_Change_Impact_with_Ubiquitous.pdf
(Bennett, R.M., et al., Proceedings of Joint International Conference on Theory Data Handling and Modelling in Geospatial Information Science, 26-28 May 2010.) This research paper looks at the use of technologies such as Google Earth to visualize changes in weather patterns and climate, with an eye to informing decision making related to climate change.
Visualizing Climate Change Impact with Ubiquitous Spatial Technologieshttp://www.csdila.unimelb.edu.au/publication/conferences/Visualizing_Climate_Change_Impact_with_Ubiquitous.pdf
(Bennett, R.M., et al., Proceedings of Joint International Conference on Theory Data Handling and Modelling in Geospatial Information Science, 26-28 May 2010.) This research paper looks at the use of technologies such as Google Earth to visualize changes in weather patterns and climate, with an eye to informing decision making related to climate change.
31
Delicious: Visual Data Analysis
http://delicious.com/tag/hz10anz+visualization
Follow this link to find additional resources tagged for this topic and this edition of the Horizon Report. To add to this list, simply tag resources with “hz10anz” and “visualization” when you save them to Delicious.
Delicious: Visual Data Analysishttp://delicious.com/tag/hz10anz+visualization
Follow this link to find additional resources tagged for this topic and this edition of the Horizon Report. To add to this list, simply tag resources with “hz10anz” and “visualization” when you save them to Delicious.
