ETK: an evaluation toolkit for visualization user studies
A 2AFC discrimination threshold experiment for video compression needs to compare the baseline (uncompressed) image (top left) with many stimuli levels (top right). In the ETK 2AFC module, the images are condensed to an array of image names. Five regions across the globe (bottom) average over multiple conditions.Abstract
This paper describes the design and features of the Evaluation Toolkit (ETK), a set of JavaScript/HTML/CSS modules leveraging the Qualtrics JavaScript API that can be used to automate image-based perceptual user evaluation studies. Automating the presentation of the images can greatly decrease the time to build and implement an evaluation study while minimizing the length and complexity of a study built within Qualtrics, along with decreasing the possibility of error in image presentation. The ETK modules each focus on automating a specific psychophysical or experimental approach. Because each module is an extension or plug-in to a Qualtrics question, the resultant study can be easily used in a laboratory setting or in a crowdsourced approach. We present the open source repository of ETK with the six modules that currently make up the toolkit and invite the community to explore, utilize, and contribute to the toolkit.
Type
Publication
In EuroVis ‘17: Proceedings of the Eurographics/IEEE VGTC Conference on Visualization: Short Papers
