The cost of not exper­i­ment­ing? Learn­ings from the FCA

Alexan­dra Chester­field, Natwest


By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

Par­tic­i­pants com­plet­ing a study online can­not clar­i­fy their under­stand­ing of the task with an exper­i­menter, pos­si­bly lead­ing to reduced data qual­i­ty. The absence of an exper­i­menter can be par­tic­u­lar­ly detri­men­tal to the data qual­i­ty for designs that involve com­plex cog­ni­tive tasks and mul­ti­ple test­ing ses­sions. How­ev­er, the insur­gence of video con­fer­enc­ing tech­nol­o­gy now per­mits live inter­ac­tion between a par­tic­i­pant and an exper­i­menter, facil­i­tat­ing task com­pre­hen­sive­ness and pos­si­bly improv­ing data qual­i­ty of online stud­ies. The pur­pose of the cur­rent study was to deter­mine how the deliv­ery of task instruc­tions impacts data qual­i­ty in an online cog­ni­tive study. 

In a between-sub­jects design, par­tic­i­pants com­plet­ed two test­ing ses­sions in either the Zoom con­di­tion where an exper­i­menter deliv­ered instruc­tions or in a writ­ten instruc­tion con­di­tion (no exper­i­menter). Each par­tic­i­pant com­plet­ed two cog­ni­tive tasks (spa­tial n‑back and Remote Asso­ciates Test) along with sur­veys. Data qual­i­ty was assessed through atten­tion checks, com­pre­hen­sion quizzes, task per­for­mance, and sur­vey test-retest reli­a­bil­i­ty. Data col­lec­tion was recent­ly com­plet­ed, and results will be pre­sent­ed at the con­fer­ence. As an inte­grat­ed ser­vice provider with its graph­i­cal user inter­face, help­ful sup­port team, and online com­mu­ni­ty, Goril­la has allowed us to cre­ate and run our first online study in less than a year.

Sign up for BeOnline

BeOnline is the conference to learn all about online behavioral research. It's the ideal place to discover the challenges and benefits of online research and to learn from pioneers. If that sounds interesting to you, then click the button below and register for the conference.


With thanks to our sponsors!

The cost of not exper­i­ment­ing? Learn­ings from the FCA