Data visualization is one of the final steps of the scientific process. As researchers, we are trained to develop a hypothesis, design the right experiment to test it, apply the correct statistical tests to evaluate the results, and finally produce data visualizations that capture the main effects. This final step is often left to the discretion of the authors, since there are many different ways to depict results visually.
For example, the two charts below depict the same exact results. Let's suppose the data represents average grades on a math exam of two groups of students, before and after particular types of instruction. In the bar chart on the left, the x-axis represents time (before or after the intervention), the y-axis represents average exam scores, and the bar colors represent the type of instruction the groups receive (blue represents video instruction and red represents a standard lecture). If you take the data at face value, the bar chart suggests that while both groups improved, the group that received the standard lecture improved more.
Now, the line graph on the right represents the same data, but visualized differently. Instead of individual bars, each instruction group is shown by a single line (blue or red) that represents the type of instruction they received, where each line connects the data from before and after the instruction manipulation. For most researchers, the choice of whether to present data in the form of a bar graph vs. a line graph feels completely arbitrary, and the decision might come down to aesthetics. For example, I might choose the representation on the left because the colors are clearer, and the data points are separated from each other, making the values easier to read out. Someone else might choose the line graph because they prefer to see the two time points connected.
A new study published in last month's issue of Journal of Vision suggests that the choice of visualization matters more than we think. Nestor Matthews and colleagues designed a series of bar graphs and line graphs depicting eight possible types of effects and interactions. For example, if we think about the exam performance data above, the different graphs could show a main effect of instruction type (e.g. the red line always above the blue line), a main effect of time (e.g. both lines go up after instruction), different types of interactions (e.g. the red line goes up while the blue line goes down), or a number of other possibilities. In the study, participants completed blocks of trials in which they had to categorize each graph as a "target" or "non-target". At the beginning of each block, participants were not informed what the target was, so initial responses were random, and participants received feedback after each response. After several trials, participants eventually improved in their ability to classify the graphs, and the rate of improvement was tracked over time across the eight different types of graphs and compared between bar graphs and line graphs.
The results were quite surprising: first, participants showed much faster learning when categorizing line graphs compared to bar graphs. This was true whether the data depicted main effects or interactions. Second, participants really struggled to categorize graphs that depicted main effects of the factor along the x-axis, regardless of graph format. For example, in the example above, the x-axis represents time (before and after the instruction manipulation). Participants were much more likely to attend to the factor shown by the two colors (blue vs. red) and their interaction (the different slopes of the lines) than the factor of time represented by the x-axis. This was most pronounced for bar graphs; even after 24 trials, participants assigned to the bar graph condition were no better than chance at identifying bar graphs that depicted the correct changes in the x-axis factor.
As a researcher, these results highlight two main issues to consider when choosing my next data visualization. The first is that bar graphs seem to be inherently less clear than line graphs. Despite my own visual preference for bar graphs, it may be more effective to communicate some results using line graphs. Why that is remains a question for further investigation. Perhaps an eye-tracking study can help pinpoint what aspects of graphs people attend to and what aspects they ignore.
The other issue is that, even when using line graphs, people might take longer to incorporate information that varies along the x-axis. In the examples above, time was always depicted along the x-axis, but I could have made a different choice. For example, the line graph below depicts the same data as the line graph above, except that the meaning of the x-axis and color assignment swapped. Now, the x-axis represents the format of instruction, and the colors represent the time of the exam (before or after instruction). Even though all the same data is depicted, the research study by Matthews and colleagues suggests people might ignore the new x-axis factor and focus more on the colors.
Scientists may also base their visualization decisions on whether the x-axis actually represents a continuous or discontinuous variable. In the second line graph, the x-axis represents the two groups, which is not a continuous variable. I would personally be reluctant to use a line graph this way, because to me, line graphs implicitly suggest the x-factor could vary continuously between the two endpoints of the line. But there is no intrinsic continuity between a video lecture and a standard lecture. Therefore, my preference would be to depict time (before and after) along the x-axis, as in the line graph in the first figure. One could imagine extending the x-axis to represent future time points, for example to show whether instruction effects are maintained over longer periods of time. Nevertheless, this choice of how to designate the meaning of the x-axis can have unintended effects on the salience and understandability of the results from the reader's perspective.
Communicating clearly is a top goal of scientific writing. The new research by Matthews and colleagues suggests that, beyond the typical factors scientists consider when deciding how to visualize results, we need to also consider how readable different graph elements are across the general population.