The Web Experiment Method : advantages, disadvantages, and solutions

Abstract

The World Wide Web (WWW) provides a new tool for experimental research. The Web experiment method differs in fundamental aspects from traditional laboratory and field experiments; therefore it can be used to validate previous findings. Web experiments offer (1) easy access to a demographically and culturally diverse participant population, including participants from unique and previously inaccessible target populations; (2) bringing the experiment to the participant instead of the opposite; (3) high statistical power by enabling access to large samples; (4) the direct assessment of motivational confounding; and (5) cost savings of lab space, person-hours, equipment, and administration. These and 13 other advantages of Web experiments are reviewed and contrasted with 7 isadvantages, such as (1) multiple submissions, (2) lack of experimental control, (3) self-selection, and (4) drop out.Several techniques and other detailed solutions are described that avoid potential problems or even turn them into useful features of Web experimentation.