Iscience.Eu

Behavioral research and data collection via the Internet.

by Ulf-Dietrich Reips

Co-authored with Birnbaum, M. H., published 2005 In R. W. Proctor and K.-P. L. Vu (Eds.), The handbook of human factors in Web design (pp. 471-492). Mahwah, New Jersey: Erlbaum.

see 2nd edition in 2011 see 2nd edition in 2011

Design and formatting in Internet-based research

by Ulf-Dietrich Reips

Published 2010. In S. Gosling & J. Johnson, Advanced Internet Methods in the Behavioral Sciences (pp. 29-43). Washington, DC: American Psychological Association.

Web-based versus lab-based studies: A response to Kendall (2008)

by Ulf-Dietrich Reips

published in Empirical Musicology Review, 3(2), 73-77.

While in an earlier commentary (Honing & Ladinig, 2008) we stressed the potential of Web-delivered experiments for... more While in an earlier commentary (Honing & Ladinig, 2008) we stressed the potential of Web-delivered experiments for music perception research, the ongoing discussion on Web-based versus lab-based studies seems to circle around issues of method and control (Mehler, 1999; Kendall, 2008). We agree with the importance of these issues from a methodological point of view. However, we continue to stress that these issues are not essentially different for Web-based as compared to lab-based studies.

How Internet-mediated research changes science

by Ulf-Dietrich Reips

published 2008 in A. Barak (Ed.), Psychological aspects of cyberspace: Theory, research, applications (pp. 268-294). Cambridge University Press.

Internet experiments: methods, guidelines, metadata

by Ulf-Dietrich Reips

published 2009 in Human Vision and Electronic Imaging XIV 7240(1), 724008. (Invited paper by the SPIE-IS&T Science and Technology Electronic Imaging societies)

The Internet experiment is now a well-established and widely used method. The present paper describes guidelines for... more The Internet experiment is now a well-established and widely used method. The present paper describes guidelines for the proper conduct of Internet experiments, e.g. handling of dropout, unobtrusive naming of materials, and pre-testing. Several methods are presented that further increase the quality of Internet experiments and help to avoid frequent errors. These methods include the "seriousness check", "warm-up," "high hurdle," and "multiple site entry" techniques, control of multiple submissions, and control of motivational confounding. Finally, metadata from sites like WEXTOR (http://wextor.org) and the web experiment list (http://wexlist.net/) are reported that show the current state of Internet-based research in terms of the distribution of fields, topics, and research designs used.

Das psychologische Experimentieren im Internet

by Ulf-Dietrich Reips

Messinstrumente und Skalen [Measuring devices and scales].

by Ulf-Dietrich Reips

Co-authored by F. Funke, published 2007 In M. Welker & O. Wenzel (Hrsg.), Online-Forschung 2007: Grundlagen und Fallstudien (S. 52-76). Köln: Herbert von Halem.

Die fortgeschrittene, weiterhin fortschreitende und in immer mehr Bereichen alltäglich werdende Nutzung des Internets... more Die fortgeschrittene, weiterhin fortschreitende und in immer mehr Bereichen alltäglich werdende Nutzung des Internets im Allgemeinen und des Word Wide Webs im Speziellen macht das Netz gleichermaßen für Wissenschaftler, Marktforscher und Demoskopen zu einem interessanten Medium. Für computergestützte selbstadministrierte Befragungen (CAWI, computer assisted web interviewing) und andere Datenerhebungsformen im Web steht eine Vielzahl von Möglichkeiten für den Einsatz qualitativer und quantitativer Verfahren zur Verfügung.

Ein Grund für diese Popularität liegt sicherlich darin, dass der selbstadministrierte Befragungsmodus, bei dem der Untersuchungsteilnehmer die Dateneingabe mit dem Beantworten selbst übernimmt, schnelle Untersuchungen großer Stichproben verspricht. Immer mehr (frei verfügbare) Software und Onlinetools für das Erstellen von Befragungen im Web und deren Auswertung, ermöglichen es auch technisch nicht versierten Personen, Daten im Netz zu erheben. Dabei wird scheinbar oftmals vergessen, dass sich Daten zwar leicht generieren lassen, sie jedoch über bestimmte Qualitäten verfügen müssen, sollen aus ihnen valide Schlüsse gezogen werden. Sowohl Qualität, als auch Quantität von Daten werden durch die gewählten Messmethoden beeinflusst.

Die beiden Haupteinflüsse auf die Qualität der Ergebnisse einer Untersuchung, Repräsentation und Messung (Überblick in Groves/Fowler/Couper 2004), haben für Datenerhebungen im Web ebenso ihre Gültigkeit, wie für andere Erhebungsarten. Während der Bereich der Repräsentation, also der Weg von der angestrebten Grundgesamtheit über die realisierte Stichprobe und weitere Einflüsse wie Antwortverweigerung, sich nicht grundsätzlich von anderen Arten der Datenerhebung unterscheidet, gibt es, durch das Medium Computer bedingt, einige Besonderheiten im Bereich der Messung. Während Konstruktvalidität und spätere Datenverarbeitung noch unabhängig von der Erhebungsart sind, gibt es spezifische Einflüsse auf den Fehler durch den Messvorgang, die einen Einfluss auf die Ausprägung des Wertes haben, der gemessen werden soll. Die hier vorgestellten Messinstrumente und Skalen werden vor allem hinsichtlich ihrer Wirkung auf den Messfehler dargestellt.

Die internetbasierte Datenerhebung ist zwar in der Lage, einige Schwierigkeiten papierbasierter Untersuchungen zu beheben (z. B. manuelle Dateneingabe; Bewertung von Stimuli, die sich nicht abdrucken lassen), manche Probleme treten jedoch in beiden Erhebungsmodi auf (Antwort nach sozialer Erwünschtheit; Effekte der graphischen Darstellung) und es kommen online neue Herausforderungen hinzu (Browserkompatibilität; Antworten durch automatische Skripte; Mehrfachteilnahmen). Aber es ergeben sich auch neue Möglichkeiten, die in Papier-und-Bleistift-Befragungen nicht oder nur mit einem erheblichen Aufwand (sehr große Teilnehmerzahl; Messung von Antwortzeiten; komplexe Filterführung; multimediale Stimuli; völlig freiwillige Teilnahme) zu realisieren sind.

Die folgenden zwei Abschnitte zur Nutzeridentifizierung und zum Kompatibilitätsproblem sind für die spätere Datenauswertung von entscheidender Bedeutung, da stets kontrollierbar sein sollte, welcher Nutzer welche Angaben unter welchen Bedingungen gemacht hat. Der anschließende Teil über Messmethoden fokussiert auf die technische Seite der Datenerhebung im Netz. Dort werden die Vielfältigkeit der computergestützten Messungen und der damit verbundenen Einfluss auf Datenqualität und -quantität vorgestellt. Der Abschnitt über webbasierte Skalen zeigt die große Bandbreite von reaktiven Erhebungsinstrumenten in selbstadministrierten Befragungen. Letztlich richten wir - exemplarisch für Innovationen der Datenerhebung im Internet - einen Blick auf dynamische Formulare, die neben neuen Möglichkeiten der Datenerhebung auch eine Fülle von methodischen Fragen mit sich bringen.

Behavioral research and data collection via the Internet

by Ulf-Dietrich Reips

Co-Authored by Michael Birnbaum.  In R. W. Proctor and K.-P. L. Vu (Eds.)(2011), The handbook of human factors in Web design (2nd edition, pp. 563-585). Mahwah, New Jersey: Erlbaum.

http://www.crcpress.com/product/isbn/9781439825945

Sliders for the smart: Type of rating scale on the Web interacts with educational level

by Ulf-Dietrich Reips

Published 2011 in Social Science Computer Review, 29, 221-231. Co-authored with Frederik Funke and Randall Thomas.

Slider scales and radio buttons scales were experimentally compared in horizontal and vertical orientation. Slider... more Slider scales and radio buttons scales were experimentally compared in horizontal and vertical orientation. Slider scales lead to statistically significantly higher break-off rates (odds ratio 1/4 6.9) and substantially higher response times. Problems with slider scales were especially prevalent in participants with less than average education, suggesting the slider scale format is more challenging in terms of previous knowledge needed or cognitive load. An alternative explanation, technologydependent sampling (Buchanan & Reips, 2001), cannot fully account for the current results. The authors clearly advise against the use of Java-based slider scales and advocate low-tech solutions for the design of Web-based data collection. Orientation on screen had no observable effect on data quality or usability of rating scales. Implications of item format for Web-based surveys are discussed.

Scientific LogAnalyzer: A Web-based tool for analyses of server log files in psychological research.

by Ulf-Dietrich Reips

Co-autored by Stieger, S., published 2004 in Behavior Research Methods, Instruments, & Computers, 36, 304-311.

Scientific LogAnalyzer is a platform-independent interactive Web service for the analysis of log files. Scientific... more Scientific LogAnalyzer is a platform-independent interactive Web service for the analysis of log files. Scientific LogAnalyzer offers several features not available in other log file analysis tools — for example, organizational criteria and computational algorithms suited to aid behavioral and social scientists. Scientific LogAnalyzer is highly flexible on the input side (unlimited types of log file formats), while strictly keeping a scientific output format. Features include (1) free definition of log file format, (2) searching and marking dependent on any combination of strings (necessary for identifying conditions in experiment data), (3) computation of response times, (4) detection of multiple sessions, (5) speedy analysis of large log files, (6) output in HTML and/or tab-delimited form, suitable for import into statistics software, and (7) a module for analyzing and visualizing drop-out. Several methodological features specifically needed in the analysis of data collected in Internet-based experiments have been implemented in the Web-based tool and are described in this article. A regression analysis with data from 44 log file analyses shows that the size of the log file and the domain name lookup are the two main factors determining the duration of an analysis. It is less than a minute for a standard experimental study with a 2 X 2 design, a dozen Web pages, and 48 participants (ca. 800 lines, including data from drop-outs). The current version of Scientific LogAnalyzer is freely available for small log files. Its Web address is h

Mitarbeiterbefragungen per Internet oder Papier? Der Einfluss von Anonymität, Freiwilligkeit und Alter auf das Antwortverhalten [Employee surveys via Internet or paper? The influence of anonymity, voluntariness and age on answering behavior].

by Ulf-Dietrich Reips

Co-authored by Franek, L., published 2004 in Wirtschaftspsychologie, 6(1), 67-83.

Webometrics for an Open Access start-up journal.

by Ulf-Dietrich Reips

Co-authored by Uwe Matzat: published 2009 in International Journal of Internet Science: Reips, U.-D., & Matzat, U. (2009). Webometrics for an Open Access start-up journal. International Journal of Internet Science, 4, 1-3.

Why semantic differentials in Web-based research should be made from visual analogue scales and not from 5-point scales

by Ulf-Dietrich Reips

in press in Field Methods, co-authored with Frederik Funke. Featured in Visual Turn blog (http://www.visualturn.com/post/10590032374/say-bye-bye-to-the-likert-scale-a-really) and reblogged in tumblr's Science section (http://www.tumblr.com/tagged/science?before=1316868457), with >8000 notes.

In a Web experiment, participants were randomly assigned to 2 semantic differentials either made from discrete 5-point... more In a Web experiment, participants were randomly assigned to 2 semantic differentials either made from discrete 5-point ordinal rating scales or from continuous visual analogue scales (VASs) with 250 gradations. Respondents adjusted their ratings with VASs more often to maximize the precision of answers, which had a beneficial effect on data quality. No side effects like differences in means, higher dropout, more nonresponse, or higher response times were observed. Overall, the combination of semantic differentials and VASs results in a number of
advantages. Potential for further research is discussed.

Mining Twitter: Microblogging as a source for psychological wisdom of the crowds. Behavior Research Methods.

by Ulf-Dietrich Reips

Co-authored by Pablo Garaizar, published in Behavior Research Methods: Reips, U.-D., & Garaizar, P. (2011). Mining Twitter: Microblogging as a source for psychological wisdom of the crowds. Behavior Research Methods, 43, 635-642. doi http://dx.doi.org/10.3758/s13428-011-0116-6

Over the last few years, microblogging has gained prominence as a form of personal broadcasting media where... more Over the last few years, microblogging has gained prominence as a form of personal broadcasting media where information and opinion are mixed together without an established order, usually tightly linked with current reality. Location awareness and promptness provide researchers using the Internet with the opportunity to create “psychological landscapes"—that is, to detect differences and changes in voiced (twittered) emotions, cognitions, and behaviors. In our article, we present iScience Maps, a free Web service for researchers, available from http://maps.iscience.deusto.es/ and http://tweet miner.eu/. Technologically, the service is based on Twitter's streaming and search application programming interfaces (APIs), accessed through several PHP libraries, and a Java- Script frontend. This service allows researchers to assess via Twitter the effect of specific events in different places as they are happening and to make comparisons between cities, regions, or countries regarding psychological states and their evolution in the course of an event. In a step-by-step example, it is shown how to replicate a study on affective and personality characteristics inferred from first names (Mehrabian & Piercy, Personality and Social Psychology Bulletin, 19, 755–758 1993) by mining Twitter data with iScience Maps. Results from the original study are replicated in both world regions we tested (the western U.S.and the U.K.); we also discover base rate of names to be a confound that needs to be controlled for in future research.