publications

2025

1. 

   The Anatomy of a Plea: How Uncertainty, Visualizations & Individual Differences Shape Plea Bargain Decisions

   Melanie Bancilhon, Alvitta Ottley, and Andrew Jordan

   In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025

   Abs PDF

   Plea bargains are commonly used in the criminal justice system, where they can offer potential benefits to both the prosecution and the defendant. However, research has shown that defendants often engage in poor decision-making, such as accepting the plea even when the trial sentence is likely to be less severe. While previous studies have shown some evidence that uncertainty visualizations can improve decision-making, there is a lack of research on their effectiveness in domain-specific tasks like plea bargain decision-making. In this work, we conduct a series of experiments to explore whether the presence and format of uncertainty impact plea bargain decisions, taking into account time pressure and individual differences. Our findings reveal that these factors can have a significant impact on plea bargain decisions. We also show evidence that communicating uncertainty in the form of text can elicit more optimal decisions under time-pressure conditions.

2. 

   Benchmarking Visual Language Models on Standardized Visualization Literacy Tests

   Saugat Pandey, and Alvitta Ottley

   In Computer Graphics Forum, 2025

   Abs HTML PDF

   The increasing integration of Visual Language Models (VLMs) into visualization systems demands a comprehensive understanding of their visual interpretation capabilities and constraints. While existing research has examined individual models, systematic comparisons of VLMs’ visualization literacy remain unexplored. We bridge this gap through a rigorous, first-of-its-kind evaluation of four leading VLMs (GPT-4, Claude, Gemini, and Llama) using standardized assessments: the Visualization Literacy Assessment Test (VLAT) and Critical Thinking Assessment for Literacy in Visualizations (CALVI). Our methodology uniquely combines randomized trials with structured prompting techniques to control for order effects and response variability - a critical consideration overlooked in many VLM evaluations. Our analysis reveals that while specific models demonstrate competence in basic chart interpretation (Claude achieving 67.9% accuracy on VLAT), all models exhibit substantial difficulties in identifying misleading visualization elements (maximum 30.0% accuracy on CALVI). We uncover distinct performance patterns: strong capabilities in interpreting conventional charts like line charts (76-96% accuracy) and detecting hierarchical structures (80-100% accuracy), but consistent difficulties with data-dense visualizations involving multiple encodings (bubble charts: 18.6-61.4%) and anomaly detection (25-30% accuracy). Significantly, we observe distinct uncertainty management behavior across models, with Gemini displaying heightened caution (22.5% question omission) compared to others (7-8%). These findings provide crucial insights for the visualization community by establishing reliable VLM evaluation benchmarks, identifying areas where current models fall short, and highlighting the need for targeted improvements in VLM architectures for visualization tasks. To promote reproducibility, encourage further research, and facilitate benchmarking of future VLMs, our complete evaluation framework, including code, prompts, and analysis scripts, is available at https://github.com/washuvis/VisLit-VLM-Eval

3. 

   Trustworthy by Design: The Viewer’s Perspective on Trust in Data Visualization

   Oen G McKinley, Saugat Pandey, and Alvitta Ottley

   In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025

   Abs PDF

   Despite the importance of viewers’ trust in data visualization, there is a lack of research on the viewers’ own perspective on their trust. In addition, much of the research on trust remains relatively theoretical and inaccessible for designers. This work aims to address this gap by conducting a qualitative study to explore how viewers perceive different data visualizations and how their perceptions impact their trust. Three dominant themes emerged from the data. First, users appeared to be consistent, listing similar rationale for their trust across different stimuli. Second, there were diverse opinions about what factors were most important to trust perception and about why the factors matter. Third, despite this disagreement, there were important trends to the factors that users reported as impactful. Finally, we leverage these themes to give specific and actionable guidelines for visualization designers to make more trustworthy visualizations.

4. 

   The State of the Art in User-Adaptive Visualizations

   Fernando Yanez, Cristina Conati, Alvitta Ottley, and 1 more author

   In Computer Graphics Forum, 2025

   Abs PDF

   Research shows that user traits can modulate the use of visualization systems and have a measurable influence on users’ accuracy, speed, and attention when performing visual analysis. This highlights the importance of user-adaptive visualization that can modify themselves to the characteristics and preferences of the user. However, there are very few such visualization systems, as creating them requires broad knowledge from various sub-domains of the visualization community. A user-adaptive system must consider which user traits they adapt to, their adaptation logic, and the types of (E)-Interventions they support. In this STAR, we survey a broad space of existing literature and consolidate them to structure the process of creating user-adaptive visualizations into five components: Capture (A)-Input from the user and any relevant peripheral information. Perform computational (B)-User Modeling with this input to construct a (C)-User Representation. Employ (D)-Adaptation Assignment logic to identify when and how to introduce (E)-Interventions. Our novel taxonomy provides a road map for work in this area, describing the rich space of current approaches and highlighting open areas for future work.

2024

1. 

   Guided By AI: Navigating Trust, Bias, and Data Exploration in AI-Guided Visual Analytics

   Sunwoo Ha, Shayan Monadjemi, and Alvitta Ottley

   In Computer Graphics Forum, 2024

   HONORABLE MENTION

   Abs PDF

   The increasing integration of artificial intelligence (AI) in visual analytics (VA) tools raises vital questions about the behavior of users, their trust, and the potential of induced biases when provided with guidance during data exploration. We present an experiment where participants engaged in a visual data exploration task while receiving intelligent suggestions supplemented with four different transparency levels. We also modulated the difficulty of the task (easy or hard) to simulate a more tedious scenario for the analyst. Our results indicate that participants were more inclined to accept suggestions when completing a more difficult task despite the AI’s lower suggestion accuracy. Moreover, the levels of transparency tested in this study did not significantly affect suggestion usage or subjective trust ratings of the participants. Additionally, we observed that participants who utilized suggestions throughout the task explored a greater quantity and diversity of data points. We discuss these findings and the implications of this research for improving the design and effectiveness of AI-guided VA tools.

2. 

   Toward a More Comprehensive Understanding of Visualization Literacy

   Lily W Ge, Maryam Hedayati, Yuan Cui, and 13 more authors

   ACM CHI, 2024

   Abs PDF

   Researchers have proposed many defnitions of visualization literacy, targeting various aspects of the term. But we have yet to fully capture what it really means to be literate in visualizations, which has important downstream implications, such as how to efectively teach visualization skills to younger generations. We ran a meetup at IEEE VIS 2022 that attracted over 30 researchers in the feld, who discussed aspects of visualization literacy such as how we measure it, how we can improve it, how it develops, and how it relates to other literacies. ACM CHI has a track record of attracting researchers from various felds such as visualization, learning sciences, and design, advancing research through both quantitative and qualitative approaches in and around HCI. For this year’s CHI, we propose to run a one-day workshop with the goal of further developing actionable research agendas to more comprehensively defne, understand, and improve visualization literacy. By continuing critical discussions with diverse perspectives from the CHI community, we can deepen investigations of visualization literacy through multiple lenses, such as measurement, interventions, and pedagogy.

3. 

   Building and Eroding: Exogenous and Endogenous Factors that Influence Subjective Trust in Visualization

   R Jordan Crouser, Syrine Matoussi, Lan Kung, and 3 more authors

   IEEE VIS, 2024

   Abs PDF

   Trust is a subjective yet fundamental component of human-computer interaction, and is a determining factor in shaping the efficacy of data visualizations. Prior research has identified five dimensions of trust assessment in visualizations (credibility, clarity, reliability, familiarity, and confidence), and observed that these dimensions tend to vary predictably along with certain features of the visualization being evaluated. This raises a further question: how do the design features driving viewers’ trust assessment vary with the characteristics of the viewers themselves? By reanalyzing data from these studies through the lens of individual differences, we build a more detailed map of the relationships between design features, individual characteristics, and trust behaviors. In particular, we model the distinct contributions of endogenous design features (such as visualization type, or the use of color) and exogenous user characteristics (such as visualization literacy), as well as the interactions between them. We then use these findings to make recommendations for individualized and adaptive visualization design.

4. 

   ConFides: A Visual Analytics Solution for Automated Speech Recognition Analysis and Exploration

   Sunwoo Ha, Chaehun Lim, R Jordan Crouser, and 1 more author

   IEEE VIS, 2024

   Abs PDF

   Confidence scores of automatic speech recognition (ASR) outputs are often inadequately communicated, preventing its seamless integration into analytical workflows. In this paper, we introduce ConFides, a visual analytic system developed in collaboration with intelligence analysts to address this issue. ConFides aims to aid exploration and post-AI-transcription editing by visually representing the confidence associated with the transcription. We demonstrate how our tool can assist intelligence analysts who use ASR outputs in their analytical and exploratory tasks and how it can help mitigate misinterpretation of crucial information. We also discuss opportunities for improving textual data cleaning and model transparency for human-machine collaboration.

5. 

   The Dance of Logic and Unpredictability: Examining the Predictability of User Behavior on Visual Analytics Tasks

   Alvitta Ottley

   VISIGRAPP, 2024

   Abs PDF

   The quest to develop intelligent visual analytics (VA) systems capable of collaborating and naturally interacting with humans presents a multifaceted and intriguing challenge. VA systems designed for collaboration must adeptly navigate a complex landscape filled with the subtleties and unpredictabilities that characterize human behavior. However, it is noteworthy that scenarios exist where human behavior manifests predictably. These scenarios typically involve routine actions or present a limited range of choices. This paper delves into the predictability of user behavior in the context of visual analytics tasks. It offers an evidence-based discussion on the circumstances under which predicting user behavior is feasible and those where it proves challenging. We conclude with a forward-looking discussion of the future work necessary to cultivate more synergistic and efficient partnerships between humans and the VA system. This exploration is not just about understanding our current capabilities and limitations in mirroring human behavior but also about envisioning and paving the way for a future where human-machine interaction is more intuitive and productive.

2023

1. 

   Improving Evaluation Using Visualization Decision-Making Models: A Practical Guide

   Melanie Bancilhon, Lace Padilla, and Alvitta Ottley

   In Visualization Psychology, 2023

   Abs PDF

   In visualization research, evaluation is a crucial step to assess the impact of visualization on decision-making. Existing work often gauges how good a visualization is by measuring its ability to induce accurate and fast judgment. While those measures provide some insight into the efficacy of a graph, underlying cognitive processes responsible for reasoning and judgment are often overlooked when they can have significant implications for visualization recommendation. Cognitive processes do not need to be a black box. There exist multiple models that explain decision processes, such as theories from behavioral economics and dual process theories. In this chapter, we compare and contrast different models and advocate for integrating a dual-process approach to visualization evaluation in the context of decision-making. Our goal is to guide visualization researchers to adopt a more granular approach to evaluation at the intersection of visualization and cognitive science.

2. 

   What Do We Mean When We Say “Insight”? A Formal Synthesis of Existing Theory

   Leilani Battle, and Alvitta Ottley

   IEEE Transactions on Visualization and Computer Graphics, 2023

   Abs HTML PDF

   Researchers have derived many theoretical models for specifying users’ insights as they interact with a visualization system. These representations are essential for understanding the insight discovery process, such as when inferring user interaction patterns that lead to insight or assessing the rigor of reported insights. However, theoretical models can be difficult to apply to existing tools and user studies, often due to discrepancies in how insight and its constituent parts are defined. This paper calls attention to the consistent structures that recur across the visualization literature and describes how they connect multiple theoretical representations of insight. We synthesize a unified formalism for insights using these structures, enabling a wider audience of researchers and developers to adopt the corresponding models. Through a series of theoretical case studies, we use our formalism to compare and contrast existing theories, revealing interesting research challenges in reasoning about a user’s domain knowledge and leveraging synergistic approaches in data mining and data management research.

3. 

   What Exactly is an Insight? A Literature Review

   Leilani Battle, and Alvitta Ottley

   2023 IEEE Visualization and Visual Analytics (VIS), 2023

   Abs PDF

   Insights are often considered the ideal outcome of visual analysis sessions. However, there is no single definition of what an insight is. Some scholars define insights as correlations, while others define them as hypotheses or aha moments. This lack of a clear definition can make it difficult to build visualization tools that effectively support insight discovery. In this paper, we contribute a comprehensive literature review that maps the landscape of existing insight definitions. We summarize key themes regarding how insight is defined, with the goal of helping readers identify which definitions of insight align closely with their research and tool development goals. Based on our review, we also suggest interesting research directions, such as synthesizing a unified formalism for insight and connecting theories of insight to other critical concepts in visualization research.

4. 

   Do You Trust What You See? Toward A Multidimensional Measure of Trust in Visualization

   Saugat Pandey, Oen G McKinley, R Jordan Crouser, and 1 more author

   In 2023 IEEE Visualization and Visual Analytics (VIS), 2023

   Abs PDF

   Few concepts are as ubiquitous in computational fields as trust. However, in the case of information visualization, there are several unique and complex challenges, chief among them: defining and measuring trust. In this paper, we investigate the factors that influence trust in visualizations. We draw on the literature to identify five factors likely to affect trust: credibility, clarity, reliability, familiarity, and confidence. We then conduct two studies investigating these factors’ relationship with visualization design features. In the first study, participants’ credibility, understanding, and reliability ratings depended on the visualization design and its source. In the second study, we find these factors also align with subjective trust rankings. Our findings suggest that these five factors are important considerations for the design of trustworthy visualizations.

5. 

   Mini-VLAT: A Short and Effective Measure of Visualization Literacy

   Saugat Pandey, and Alvitta Ottley

   Computer Graphics Forum, In Proceedings of the 25th EG Conference on Visualization (EuroVis), 2023

   BEST PAPER AWARD

   Abs HTML PDF

   The visualization community regards visualization literacy as a necessary skill. Yet, despite the recent increase in research into visualization literacy by the education and visualization communities, we lack practical and time-effective instruments for the widespread measurements of people’s comprehension and interpretation of visual designs. We present Mini-VLAT, a brief but practical visualization literacy test. The Mini-VLAT is a 12-item short form of the 53-item Visualization Literacy Assessment Test (VLAT). The Mini-VLAT is reliable (coefficient omega = 0.72) and strongly correlates with the VLAT. Five visualization experts validated the Mini-VLAT items, yielding an average content validity ratio (CVR) of 0.6. We further validate Mini-VLAT by demonstrating a strong positive correlation between study participants’ Mini-VLAT scores and their aptitude for learning an unfamiliar visualization using a Parallel Coordinate Plot test. Overall, the Mini-VLAT items showed a similar pattern of validity and reliability as the 53-item VLAT. The results show that Mini-VLAT is a psychometrically sound and practical short measure of visualization literacy.

6. 

   Human-Computer Collaboration for Visual Analytics: an Agent-based Framework

   Shayan Monadjemi, Mengtian Guo, David Gotz, and 2 more authors

   Computer Graphics Forum, In Proceedings of the 25th EG Conference on Visualization (EuroVis), 2023

   Abs PDF

   The visual analytics community has long aimed to understand users better and assist them in their analytic endeavors. As a result, numerous conceptual models of visual analytics aim to formalize common workflows, techniques, and goals leveraged by analysts. While many of the existing approaches are rich in detail, they each are specific to a particular aspect of the visual analytic process. Furthermore, with an ever-expanding array of novel artificial intelligence techniques and advances in visual analytic settings, existing conceptual models may not provide enough expressivity to bridge the two fields. In this work, we propose an agent-based conceptual model for the visual analytic process by drawing parallels from the artificial intelligence literature. We present three examples from the visual analytics literature as case studies and examine them in detail using our framework. Our simple yet robust framework unifies the visual analytic pipeline to enable researchers and practitioners to reason about scenarios that are becoming increasingly prominent in the field, namely mixed-initiative, guided, and collaborative analysis. Furthermore, it will allow us to characterize analysts, visual analytic settings, and guidance from the lenses of human agents, environments, and artificial agents, respectively.

7. 

   Why Combining Text and Visualization Could Improve Bayesian Reasoning: A Cognitive Load Perspective

   Melanie Bancilhon, Amanda Wright, Sunwoo Ha, and 2 more authors

   In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023

   Abs PDF

   Investigations into using visualization to improve Bayesian reasoning and advance risk communication have produced mixed results, suggesting that cognitive ability might affect how users perform with different presentation formats. Our work examines the cognitive load elicited when solving Bayesian problems using icon arrays, text, and a juxtaposition of text and icon arrays. We used a three-pronged approach to capture a nuanced picture of cognitive demand and measure differences in working memory capacity, performance under divided attention using a dual-task paradigm, and subjective ratings of self-reported effort. We found that individuals with low working memory capacity made fewer errors and experienced less subjective workload when the problem contained an icon array compared to text alone, showing that visualization improves accuracy while exerting less cognitive demand. We believe these findings can considerably impact accessible risk communication, especially for individuals with low working memory capacity.

2022

1. 

   Inferential Tasks as an Evaluation Technique for Visualization

   Ashley Suh, Ab Mosca, Shannon Robinson, and 4 more authors

   In Proceedings of the 24th EG Conference on Visualization, 2022

   BEST PAPER AWARD

   Abs PDF

   Designing suitable tasks for visualization evaluation remains challenging. Traditional evaluation techniques commonly rely on ‘low-level’ or ‘open-ended’ tasks to assess the efficacy of a proposed visualization, however, nontrivial trade-offs exist between the two. Low-level tasks allow for robust quantitative evaluations, but are not indicative of the complex usage of a visualization. Open-ended tasks, while excellent for insight-based evaluations, are typically unstructured and require time-consuming interviews. Bridging this gap, we propose inferential tasks: a complementary task category based on inferential learning in psychology. Inferential tasks produce quantitative evaluation data in which users are prompted to form and validate their own findings with a visualization. We demonstrate the use of inferential tasks through a validation experiment on two well-known visualization tools.

2. 

   Guided Data Discovery in Interactive Visualizations via Active Search

   Shayan Monadjemi, Sunwoo Ha, Quan Nguyen, and 3 more authors

   In Proceedings of IEEE Visualization and Visual Analytics (VIS), 2022

   Abs PDF

   Recent advances in visual analytics have enabled us to learn from user interactions and uncover analytic goals. These innovations set the foundation for actively guiding users during data exploration. Providing such guidance will become more critical as datasets grow in size and complexity, precluding exhaustive investigation. Meanwhile, the machine learning community also struggles with datasets growing in size and complexity, precluding exhaustive labeling. Active learning is a broad family of algorithms developed for actively guiding models during training. We will consider the intersection of these analogous research thrusts. First, we discuss the nuances of matching the choice of an active learning algorithm to the task at hand. This is critical for performance, a fact we demonstrate in a simulation study. We then present the results of a user study for the particular task of data discovery guided by an active learning algorithm specifically designed for this task.

3. 

   A Unified Comparison of User Modeling Techniques for Predicting Data Interaction and Detecting Exploration Bias

   Sunwoo Ha, Shayan Monadjemi, Roman Garnett, and 1 more author

   IEEE Transactions on Visualization and Computer Graphics, In Proceedings of IEEE Visualization and Visual Analytics (VIS), 2022

   Abs PDF

   The visual analytics community has proposed several user modeling algorithms to capture and analyze users’ interaction behavior in order to assist users in data exploration and insight generation. For example, some can detect exploration biases while others can predict data points the user will interact with before that interaction occurs. Researchers believe this collection of algorithms can help create more intelligent visual analytics tools. However, the community lacks a rigorous evaluation and comparison of these existing techniques. As a result, there is limited guidance on which method to use and when. Our paper seeks to fill this gap by comparing and ranking eight user modeling algorithms based on their performance on a diverse set of four user study datasets. We analyze exploration bias detection, data interaction prediction, and algorithmic complexity, among other measures. Based on our findings, we highlight open challenges and new directions for analyzing user interactions and visualization provenance.

4. 

   Vizxp: A visualization framework for conveying explanations to users in model reconciliation problems

   Ashwin Kumar, Stylianos Loukas Vasileiou, Melanie Bancilhon, and 2 more authors

   In Proceedings of the International Conference on Automated Planning and Scheduling, 2022

   Abs PDF

   Advancements in explanation generation for automated planning algorithms have moved us a step closer towards realizing the full potential of human-AI collaboration in real-world planning applications. Within this context, a framework called \emphmodel reconciliation has gained a lot of traction, mostly due to its deep connection with a popular theory in human psychology, known as the \emphtheory of mind. Existing literature in this setting, however, has mostly been constrained to algorithmic contributions for generating explanations. To the best of our knowledge, there has been very little work on how to effectively convey such explanations to human users, a critical component in human-AI collaboration systems. In this paper, we set out to explore to what extent visualizations are an effective candidate for conveying explanations in a way that can be easily understood. Particularly, by drawing inspiration from work done in visualization systems for classical planning, we propose a visualization framework for visualizing explanations generated from model reconciliation algorithms. We demonstrate the efficacy of our proposed system in a comprehensive user study, where we compare our framework against a text-based baseline for two types of explanations – domain-based and problem-based explanations. Results from the user study show that users, on average, understood explanations better when they are conveyed via our visualization system compared to when they are conveyed via a text-based baseline.

5. 

   A Grammar-Based Approach for Applying Visualization Taxonomies to Interaction Logs

   Sneha Gathani, Shayan Monadjemi, Alvitta Ottley, and 1 more author

   In Computer Graphics Forum, In Proceedings of the 24th EG Conference on Visualization (EuroVis), 2022

   Abs PDF

   Researchers collect large amounts of user interaction data with the goal of mapping user’s workflows and behaviors to their high-level motivations, intuitions, and goals. Although the visual analytics community has proposed numerous taxonomies to facilitate this mapping process, no formal methods exist for systematically applying these existing theories to user interaction logs. This paper seeks to bridge the gap between visualization task taxonomies and interaction log data by making the taxonomies more actionable for interaction log analysis. To achieve this, we leverage structural parallels between how people express themselves through interactions and language by reformulating existing theories as regular grammars. We represent interactions as terminals within a regular grammar, similar to the role of individual words in a language, and patterns of interactions or non-terminals as regular expressions over these terminals to capture common language patterns. To demonstrate our approach, we generate regular grammars for seven existing visualization taxonomies and develop code to apply them to three public interaction log datasets. In analyzing these regular grammars, we find that the taxonomies at the low-level (i.e., terminals) show mixed results in expressing multiple interaction log datasets, and taxonomies at the high-level (i.e., regular expressions) have limited expressiveness, due to primarily two challenges: inconsistencies in interaction log dataset granularity and structure, and under-expressiveness of certain terminals. Based on our findings, we suggest new research directions for the visualization community to augment existing taxonomies, develop new ones, and build better interaction log recording processes to facilitate the data-driven development of user behavior taxonomies.

6. 

   Testing theories of task in visual analytics

   Leilani Battle, and Alvitta Ottley

   ACM Interactions, 2022

   Abs PDF

   Theory research plays a critical role in many areas of computer science, including artificial intelligence, programming languages, and human-computer interaction. However, striving to encapsulate the human experience within a math equation or a concise model can seem reductive at best, especially when the goal is to design software that actively cooperates with people to achieve complex goals, such as in visualization research. That being said, naive reductions can also help researchers test their understanding of the complexities of human behavior. This article explores the role of theory in visualization research and our vision for the future of visualization theory work.

2021

1. 

   FairVizARD: A Visualization System for Assessing Fairness of Ride-Sharing Matching Algorithms

   Ashwin Kumar, Sanket Shah, Meghna Lowalekar, and 3 more authors

   In Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), 2021

   Abs PDF

   There is growing interest in algorithms that match passengers with drivers in ride-sharing problems and their fairness for the different parties involved (passengers, drivers, and ride-sharing companies). Researchers have proposed various fairness metrics for matching algorithms, but it is often unclear how one should balance the various parties’ fairness, given that they are often in conflict. We present FairVizARD, a visualization-based system that aids users in evaluating the fairness of ride-sharing matching algorithms. FairVizARD presents the algorithms’ results by visualizing relevant spatio-temporal information using animation and aggregated information in charts. FairVizARD also employs efficient techniques for visualizing a large amount of information in a user friendly manner, which makes it suitable for real-world settings. We conduct our experiments on a real-world large-scale taxi dataset and, through user studies and an expert interview, we show how users can use FairVizARD not only to evaluate the fairness of matching algorithms but also to expand on their notions of fairness.

2. 

   Does interaction improve bayesian reasoning with visualization?

   Ab Mosca, Alvitta Ottley, and Remco Chang

   In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

   Abs PDF

   Interaction enables users to navigate large amounts of data effectively, supports cognitive processing, and increases data representation methods. However, there have been few attempts to empirically demonstrate whether adding interaction to a static visualization improves its function beyond popular beliefs. In this paper, we address this gap. We use a classic Bayesian reasoning task as a testbed for evaluating whether allowing users to interact with a static visualization can improve their reasoning. Through two crowdsourced studies, we show that adding interaction to a static Bayesian reasoning visualization does not improve participants’ accuracy on a Bayesian reasoning task. In some cases, it can significantly detract from it. Moreover, we demonstrate that underlying visualization design modulates performance and that people with high versus low spatial ability respond differently to different interaction techniques and underlying base visualizations. Our work suggests that interaction is not as unambiguously good as we often believe; a well-designed static visualization can be as, if not more, effective than an interactive one.

2020

1. 

   DRAGON-V: Detection and Recognition of Airplane Goals with Navigational Visualization.

   Christabel Wayllace, Sunwoo Ha, Yuchen Han, and 4 more authors

   In Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), 2020

   Abs PDF

   We introduce Detection and Recognition of Airplane GOals with Navigational Visualization (DRAGON-V), a visualization system that uses probabilistic goal recognition to infer and display the most probable airport runway that a pilot is approaching. DRAGON-V is especially useful in cases of miscommunication, low visibility, or lack of airport familiarity which may result in a pilot deviating from the assigned taxiing route. The visualization system conveys relevant information, and updates according to the airplane’s current geolocation. DRAGON-V aims to assist air traffic controllers in reducing incidents of runway incursions at airports.

2. 

   Investigating the role of locus of control in moderating complex analytic workflows

   R Jordan Crouser, Alvitta Ottley, Kendra Swanson, and 1 more author

   2020

   HONARABLE MENTION AWARD

   Abs PDF

   Throughout the last decade, researchers have shown that the effectiveness of a visualization tool depends on the experience, personality, and cognitive abilities of the user. This work has also demonstrated that these individual traits can have significant implications for tools that support reasoning and decision-making with data. However, most studies in this area to date have involved only short-duration tasks performed by lay users. This short paper presents a preliminary analysis of a series of exercises with 22 trained intelligence analysts that seeks to deepen our understanding of how individual differences modulate expert behavior in complex analysis tasks.

3. 

   Competing Models: Inferring Exploration Patterns and Information Relevance via Bayesian Model Selection

   Shayan Monadjemi, Roman Garnett, and Alvitta Ottley

   IEEE Transactions on Visualization and Computer Graphics, 2020

   Abs PDF

   Analyzing interaction data provides an opportunity to learn about users, uncover their underlying goals, and create intelligent visualization systems. The first step for intelligent response in visualizations is to enable computers to infer user goals and strategies through observing their interactions with a system. Researchers have proposed multiple techniques to model users, however, their frameworks often depend on the visualization design, interaction space, and dataset. Due to these dependencies, many techniques do not provide a general algorithmic solution to user exploration modeling. In this paper, we construct a series of models based on the dataset and pose user exploration modeling as a Bayesian model selection problem where we maintain a belief over numerous competing models that could explain user interactions. Each of these competing models represent an exploration strategy the user could adopt during a session. The goal of our technique is to make high-level and in-depth inferences about the user by observing their low-level interactions. Although our proposed idea is applicable to various probabilistic model spaces, we demonstrate a specific instance of encoding exploration patterns as competing models to infer information relevance. We validate our technique’s ability to infer exploration bias, predict future interactions, and summarize an analytic session using user study datasets. Our results indicate that depending on the application, our method outperforms established baselines for bias detection and future interaction prediction. Finally, we discuss future research directions based on our proposed modeling paradigm and suggest how practitioners can use this method to build intelligent visualization systems that understand users’ goals and adapt to improve the exploration process.

4. 

   Survey on the analysis of user interactions and visualization provenance

   Kai Xu, Alvitta Ottley, Conny Walchshofer, and 3 more authors

   Computer Graphics Forum, 2020

   Abs PDF

   There is fast-growing literature on provenance-related research, covering aspects such as its theoretical framework, use cases, and techniques for capturing, visualizing, and analyzing provenance data. As a result, there is an increasing need to identify and taxonomize the existing scholarship. Such an organization of the research landscape will provide a complete picture of the current state of inquiry and identify knowledge gaps or possible avenues for further investigation. In this STAR, we aim to produce a comprehensive survey of work in the data visualization and visual analytics field that focus on the analysis of user interaction and provenance data. We structure our survey around three primary questions: (1) WHY analyze provenance data, (2) WHAT provenance data to encode and how to encode it, and (3) HOW to analyze provenance data. A concluding discussion provides evidence-based guidelines and highlights concrete opportunities for future development in this emerging area. The survey and papers discussed can be explored online interactively at https://provenance-survey.caleydo.org.

5. 

   Survey on individual differences in visualization

   Zhengliang Liu, R Jordan Crouser, and Alvitta Ottley

   In Computer Graphics Forum, 2020

   Abs PDF

   Developments in data visualization research have enabled visualization systems to achieve great general usability and application across a variety of domains. These advancements have improved not only people’s understanding of data, but also the general understanding of people themselves, and how they interact with visualization systems. In particular, researchers have gradually come to recognize the deficiency of having one-size-fits-all visualization interfaces, as well as the significance of individual differences in the use of data visualization systems. Unfortunately, the absence of comprehensive surveys of the existing literature impedes the development of this research. In this paper, we review the research perspectives, as well as the personality traits and cognitive abilities, visualizations, tasks, and measures investigated in the existing literature. We aim to provide a detailed summary of existing scholarship, produce evidence-based reviews, and spur future inquiry.

6. 

   Adaptive and Personalized Visualization

   Alvitta Ottley

   Synthesis Lectures on Visualization, 2020

   Abs

   There is ample evidence in the visualization community that individual differences matter. These prior works highlight various personality traits and cognitive abilities that can modulate the use of the visualization systems and demonstrate a measurable influence on speed, accuracy, process, and attention. Perhaps the most important implication of this body of work is that we can use individual differences as a mechanism for estimating when a design is effective or to identify when people may struggle with visualization designs. These effects can have a critical impact on consequential decision-making processes. One study that appears in this book investigated the impact of visualization on medical decision-making showed that visual aides tended to be most beneficial for people with high spatial ability, a metric that measures a person’s ability to represent and manipulate two- or three-dimensional representations of objects mentally. The results showed that participants with low spatial ability had difficulty interpreting and analyzing the underlying medical data when they use visual aids. Overall, approximately 50% of the studied population were unsupported by the visualization tools when making a potentially life-critical decision. As data fluency continues to become an essential skill for our everyday lives, we must embrace the growing need to understand the factors that may render our tools ineffective and identify concrete steps for improvement. This book presents my current understanding of how individual differences in personality interact with visualization use and draws from recent research in the Visualization, Human-Computer Interaction, and Psychology communities. We focus on the specific designs and tasks for which there is concrete evidence of performance divergence due to personality. Additionally, we highlight an exciting research agenda that is centered around creating tailored visualization systems that are aligned with people’s abilities. The purpose of this book is to underscore the need to consider individual differences when designing and evaluating visualization systems and to call attention to this critical research direction.

7. 

   Let’s Gamble: How a Poor Visualization Can Elicit Risky Behavior

   Melanie Bancilhon, Zhengliang Liu, and Alvitta Ottley

   In IEEE Visualization Conference (VIS), 2020

   Abs PDF

   Data visualizations are standard tools for assessing and communicating risks. However, it is not always clear which designs are optimal or how encoding choices might influence risk perception and decision-making. In this paper, we report the findings of a large-scale gambling game that immersed participants in an environment where their actions impacted their bonuses. Participants chose to either enter a lottery or receive guaranteed monetary gains based on five common visualization designs. By measuring risk perception and observing decision-making, we showed that icon arrays tended to elicit economically sound behavior. We also found that people were more likely to gamble when presented area proportioned triangle and circle designs. Using our results, we model risk perception and discuss how our findings can improve visualization selection.

8. 

   Did You Get The Gist Of It? Understanding How Visualization Impacts Decision-Making

   Melanie Bancilhon, and Alvitta Ottley

   In VisPych Workshop, IEEE VIS, 2020

   Abs PDF

   As visualization researchers evaluate the impact of visualization design on decision-making, they often hold a one-dimensional perspective on the cognitive processes behind making a decision. Several psychological and economic researchers have shown that to make decisions, people rely on quantitative reasoning as well as gist-based intuition – two systems that operate in parallel. In this position paper, we discuss decision theories and provide suggestions to bridge the gap between the evaluation of decision-making in visualization and psychology research. The goal is to question the limits of our knowledge and to advocate for a more nuanced understanding of decision-making with visualization.

2019

1. 

   Follow the clicks: Learning and anticipating mouse interactions during exploratory data analysis

   Alvitta Ottley, Roman Garnett, and Ran Wan

   Computer Graphics Forum, 2019

   BEST PAPER AWARD FINALIST

   Abs PDF

   The goal of visual analytics is to create a symbiosis between human and computer by leveraging their unique strengths. While this model has demonstrated immense success, we are yet to realize the full potential of such a human-computer partnership. In a perfect collaborative mixed-initiative system, the computer must possess skills for learning and anticipating the users’ needs. Addressing this gap, we propose a framework for inferring attention from passive observations of the user’s click, thereby allowing accurate predictions of future events. We demonstrate this technique with a crime map and found that users’ clicks can appear in our prediction set 92% – 97% of the time. Further analysis shows that we can achieve high prediction accuracy typically after three clicks. Altogether, we show that passive observations of interaction data can reveal valuable information that will allow the system to learn and anticipate future events.