Time-to-Adoption Horizon: Four to Five Years
Visual data analysis blends highly advanced computational methods with sophisticated graphics engines to tap the extraordinary ability of humans to see patterns and structure in even the most complex visual presentations. Currently applied to massive, heterogeneous, and dynamic datasets, such as those generated in studies of astrophysical, fluidic, biological, and other complex processes, the techniques have become sophisticated enough to allow the interactive manipulation of variables in real time. Ultra high-resolution displays allow teams of researchers to zoom in to examine specific aspects of the renderings, or to navigate along interesting visual pathways, following their intuitions and even hunches to see where they may lead. New research is now beginning to apply these sorts of tools to the social sciences and humanities as well, and the techniques offer considerable promise in helping us understand complex social processes like learning, political and organizational change, and the diffusion of knowledge.
Over the past century, data collection, storage, transmission, and display has changed dramatically, and scholars have undergone a profound transformation in the way they approach data-related tasks. Data collection and compilation is no longer the tedious, manual process it once was, and tools to analyze, interpret, and display data are increasingly sophisticated, and their use routine in many disciplines. The options for illustrating trends, relationships, and cause and effect have exploded, and it is now a relatively simple matter for anyone to do the sorts of analyses that were once only the province of statisticians and engineers.
In advanced research settings, scientists and others studying massively complex systems generate mountains of data, and have developed a wide variety of new tools and techniques to allow those data to be interpreted holistically, and to expose meaningful patterns and structure, trends and exceptions, and more. Researchers that work with data sets from experiments or simulations, such as computational fluid dynamics, astrophysics, climate study, or medicine draw on techniques from the study of visualization, data mining, and statistics to create useful ways to investigate and understand what they have found.
The blending of these disciplines has given rise to the new field of visual data analysis, which is not only characterized by its focus on making use of the pattern matching skills that seem to be hard-wired into the human brain, but also in the way in which it facilitates the work of teams working in concert to tease out meaning from complex sets of information. While the most sophisticated tools are still mostly found in research settings, a variety of tools are emerging that make it possible for almost anyone with an analytical bent to easily interpret all sorts of data.
Self-organizing maps are an approach that mimics the way our brains organize multi-faceted relationships; they create a grid of "neuronal units" such that neighboring units recognize similar data, reinforcing important patterns so that they can be seen. Cluster analysis is a set of mathematical techniques for partitioning a series of data objects into a smaller amount of groups, or clusters, so that the data objects within one cluster are more similar to each other than to those in other clusters. Visual, interactive principal components analysis is a technique once only available to statisticians that is now commonly used to identify hidden trends and data correlations in multidimensional data sets. Gapminder (http://www.gapminder.org/), for example, uses this approach in its analysis of multivariate datasets over time.
These sorts of tools are now finding their way into common use in many other disciplines, where the analytical needs are not necessarily computational; visualization techniques have even begun to emerge for textual analysis and basic observation. Many are free or very inexpensive, bringing the ability to engage in rich visual interpretation to virtually anyone.
Online services such as Many Eyes, Wordle, Flowing Data, and Gapminder accept uploaded data and allow the user to configure the output to varying degrees. Many Eyes, for instance, allows people to learn how to create visualizations, to share and visualize their own data, and to create new visualizations from data contributed by others. Some, like Roambi, have mobile counterparts, making it easy to carry interactive, visual representations of data wherever one goes. Even quite public data, such as the posts made in Twitter, can be rendered visually to reveal creative insights. For instance, New Political Interfaces (http://newpoliticalinterfaces.org) created a visualization examining political topics as expressed on Twitter, charting which topics are — and are not — being discussed by politicians, news outlets, and other sources.
Relevance for Teaching, Learning, or Creative Inquiry
As stated previously, one of the most compelling aspects of visual data analysis is in the ways it augments the natural abilities humans have to seek and find patterns in what they see. By manipulating variables, or simply seeing them change over time (as Gapminder has done so famously) if patterns exist (or if they don’t), that fact is easily discoverable. Such tools have applicability in nearly every field.
As the tools, their capabilities, and their variety continue to expand, their use is already making its way out of scientific and engineering labs and into business and social research. Creative inquiry is benefiting from a wide range of new tools that are exposing trends and relationships among both qualitative and quantitative variables in real time, and making longitudinal relationships easier to find and interpret than ever. Textual analysis is an area that tools like Wordle have revealed as especially suited to visual techniques.
The promise for teaching and learning is further afield, but because of the intuitive ways in which it can expose complex relationships to even the uninitiated, there is tremendous opportunity to integrate visual data analysis into undergraduate research, even in survey courses. Models of complex processes in quantum physics, organic chemistry, medicine, or economics are just a few of the ways in which the outcomes of visual data analysis can be applied to learning situations.
Visual data analysis may help expand our understanding of learning itself. Learning is one of the most complex of social processes, with a myriad of variables interacting in highly complex ways, making it an ideal focus for the search for patterns. Related to this is the opportunity to understand the variables influencing informal learning and the social networking processes at work in the formation of learning communities. The tools for such analyses exist today; what is needed are ways to balance privacy with the kinds of data capture that can inform such work.
A sampling of visual data analysis applications for a variety of purposes includes the following:
- Astrophysics. Harvard scientists are using data visualization from the Chandra X-Ray Observatory to measure the expansion velocity of supernova remnants. Visual data analysis has also enabled scientists to more fully understand the effects of multiple points of explosion in a supernova.
- Fluid Dynamics and Human Physiology. Researchers working with Amira, a visual data analysis tool created originally at the Zuse Institute in Berlin, have created a range of models of biological processes from MRI data, fluid flows, and other complex datasets. Insights from the study of fluid dynamics over complex surfaces informed work that models blood flows and arterial mapping.
- Marine Geology. Published by the Lamont-Doherty Earth Observatory at Columbia University, the Virtual Ocean, similar to Google Earth, offers students a three-dimensional view of the Earth's oceans (http://www.virtualocean.org).
- Composition and Rhetoric. Using tools like Many Eyes and Wordle, students can easily analyze the contents of their papers visually for insights into what points might need further development, and whether or not certain language has been overused.
Visual Data Analysis in Practice
The following links provide examples of visual data analysis.
28 Rich Data Visualization Tools
(Theresa Neil, O’Reilly’s Inside RIA, 10 December 2009.) This article contains visual examples of dozens of data analysis displays. Listed are twenty-eight tools for creating charts, graphs, and other data displays for use by developers.
Best Science Visualization Videos of 2009
(Hadley Legget, Wired, 19 August 2009.) From simulating the way waves break against a ship to visualizing seasonal carbon dioxide accumulation in North America, these videos demonstrate the diversity of data visualization.
Brain Structure Assists in Immune Response, According to Penn Vet Study
(Jordan Reese, Media Contact, Office of University Communications, University of Pennsylvania, 28 January 2009.) Analytics and data visualization allowed researchers at the University of Pennsylvania to visually model (in real time) the response of the body's immune system to a parasitic infection.
Gapminder, a Swedish-based nonprofit organization, seeks to promote sustainable global development using data visualization as a major tool.
A wide variety of data visualization projects are featured on this site. Browse everything from changes in the text from one edition of The Origin of the Species to the next, to Cymatics, a visualization of the study of sound vibrations on matter.
Worldmapper is a visualization tool that re-draws maps based on the data being displayed. For instance, on a world map showing population, countries with more people swell while those with fewer people shrink.
For Further Reading
The following articles and resources are recommended for those who wish to learn more about visual data analysis.
7 Things You Should Know About Data Visualization II
(Educause, August 2009.) This article discusses data visualization as it relates to higher education: who's using it, why they're using it, and what to expect in the future.
New Visualization Techniques Yield Star Formation Insights: Gravity Plays Larger Role Than Thought
(Science Daily, 4 January 2009.) Early in 2009, a new computer algorithm developed at the Harvard Initiative in Innovative Computing demonstrated that data visualization is critical in the discovery of new information, not just in the final presentation of data.
The Technologies of G21: How Government Can Become a Platform for Innovation
(Gadi Ben-Yehuda, Huffington Post, 24 August 2009.) The author discusses the changes in data collection, storage, transmission, and display over the past century, noting that data visualization is now in the hands of the people for the first time.
Visualization and Knowledge Discovery: Report from the DOE/ASCR Workshop on Visual Analysis and Data Exploration at Extreme Scale
This report from the Department of Energy describes fundamental research in visualization and analysis that is enabling knowledge discovery from computational science applications at extreme scale.
Delicious: Visual Data Analysis
Follow this link to find additional resources tagged for this topic and this edition of the Horizon Report. To add to this list, simply tag resources with “hz10” and “analytics” when you save them to Delicious.
Posted by NMC on January 14, 2010