Jour­neys in the mind: the chal­lenges and ben­e­fits of using online meth­ods to under­stand dri­ver behav­iour and trans­port psychology

Cather­ine Naugh­tie, Uni­ver­si­ty of Bath
@CatNaughtie

YouTube

By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

Empir­i­cal dri­ver behav­iour and trans­port psy­chol­o­gy research fre­quent­ly relies on instru­ment­ed vehi­cles, sim­u­la­tors, or nat­u­ral­is­tic obser­va­tion stud­ies, but what can we do when these options are not avail­able? I will pro­vide insights from my expe­ri­ence design­ing, devel­op­ing, and pilot­ing an online study into dri­ver behav­iour and the effect of vehi­cle con­trol per­tur­ba­tions on dri­ver effi­cien­cy and per­for­mance in a sec­ondary atten­tion­al load­ing task. This exper­i­ment was designed to pro­vide a real­is­tic ana­logue of the dri­ving task to explore the psy­cho­log­i­cal under­pin­nings of behaviours.

In less than two months, it was pos­si­ble to learn JavaScript and devel­op a game as an ana­logue of the dri­ving task and run a pilot study (N = 35) that suc­cess­ful­ly val­i­dat­ed the exper­i­men­tal pro­to­col. Pilot results sup­port­ed the hypoth­e­sised neg­a­tive effect of per­tur­ba­tion on per­for­mance and indi­cat­ed inter­est­ing dynam­ics relat­ing to con­text, degree, and direc­tion of per­tur­ba­tions that are being explored in a fol­low up study. Here, I will out­line the main chal­lenges involved in this process, the ben­e­fits asso­ci­at­ed with it, and show that it is pos­si­ble for a cod­ing novice to devel­op inter­ac­tive, bespoke behav­iour­al exper­i­ments quick­ly and suc­cess­ful­ly. This online approach can facil­i­tate inno­v­a­tive approach­es to dri­ver behav­iour and trans­port psy­chol­o­gy research.

Full Tran­script:

Cather­ine Naugh­tie:
Well, good after­noon. I’m Cather­ine Naugh­tie, a PhD research at the Uni­ver­si­ty of Bath. Today, I’m going to talk to you about my expe­ri­ence using online research in dri­ver behav­ior. And I recent­ly faced a chal­lenge that is doubt­less famil­iar to you, a ban on in-per­son research.

Cather­ine Naugh­tie:
Now, study­ing road user and dri­ver behav­ior in lock­down is real­ly chal­leng­ing as research in the field fre­quent­ly relies on instru­ment­ed vehi­cles, sim­u­la­tors, and nat­u­ral­is­tic obser­va­tion. So, what do we do when these options are not avail­able? But online meth­ods pro­vide a life­line, allow­ing us to con­tin­ue our research and more­over to lever­age the pow­er of mov­ing it online, to approach research dif­fer­ent­ly and engage with par­tic­i­pants in new ways.

Cather­ine Naugh­tie:
Today, I’m going to talk to you about my expe­ri­ence, adapt­ing my research and the approach I took, its ben­e­fits, and some of the chal­lenges I had to over­come. I want­ed to high­light three tricks I learned through this expe­ri­ence that helped me go from being a cod­ing novice to suc­cess­ful­ly build­ing and run­ning a code based exper­i­ment. One, to do online research, not sim­ply to do research online. Two, to be ambi­tious. And three, to use the com­mu­ni­ty. Hang on. The thing isn’t work­ing. Okay, there we go.

Cather­ine Naugh­tie:
So, what do I mean by do online research, not sim­ply doing research online? Well, in dri­ver behav­ior and trans­port research, there are some ques­tions that are best under­stood through high fideli­ty or nat­u­ral­is­tic meth­ods, such as eval­u­at­ing respons­es to a spe­cif­ic vehi­cle sys­tem or obser­va­tion­al stud­ies of ease good to use, and some ques­tions that can be answered using tra­di­tion­al in-per­son meth­ods, such as atti­tudes sur­veys, or road user interviews.

Cather­ine Naugh­tie:
How­ev­er, where there’s evi­dence for a behav­ioral effect in lit­er­a­ture, we can explore larg­er and more diverse sam­ples avail­able online to inves­ti­gate the under­ly­ing cog­ni­tive process­es involved, ask­ing ques­tions that are not so fun­da­men­tal, that it’s unclear how the mech­a­nism would oper­ate in a dri­ving task, or so spe­cif­ic that it would require a real­is­tic exper­i­ment set­up. But they’re just right, where enough is known to devel­op a sim­ple exper­i­ment that can form the basis of lat­er imple­men­ta­tions and eco­log­i­cal­ly valid con­text. This is where online meth­ods can add real value.

Cather­ine Naugh­tie:
To put this in con­text, con­sid­er my research project. This emerged from an oppor­tu­ni­ty. Most mod­ern vehi­cle con­trol sys­tems are elec­tron­ic, mean­ing it’s pos­si­ble to change how they respond to dri­ver inputs on the fly. There­fore, it’s the­o­ret­i­cal­ly pos­si­ble to cre­ate per­son­al­ized adap­tive vehi­cle con­trols. This could help reduce emis­sions, improve safe­ty and just make cars nicer to dri­ve, which is great, in theory.

Cather­ine Naugh­tie:
How­ev­er, research has shown that chang­ing the way that a vehi­cle behaves in response to dri­ver inputs can increase men­tal work­load, impaired atten­tion and dam­age per­for­mance. There­fore, to ful­ly real­ize the ben­e­fits of adap­tive sys­tems, we need to under­stand the cog­ni­tive and behav­ioral con­se­quences of actu­al­ly imple­ment­ing them.

Cather­ine Naugh­tie:
Now, pre­vi­ous stud­ies inves­ti­gat­ing per­tur­ba­tions in dri­ver vehi­cle inter­ac­tions used instru­ment­ed vehi­cles and sim­u­la­tors, as you can see on the image on the right. Now, this approach is cost­ly and it’s time con­sum­ing, and it’s def­i­nite­ly not some­thing you can do online. How­ev­er, from a behav­ioral per­spec­tive, the process­es that these stud­ies were inter­ro­gat­ing could be inves­ti­gat­ed using online meth­ods, includ­ing some of the most press­ing unan­swered ques­tions. Are there thresh­olds where changes become notice­able or impact per­for­mance? Are there times when adap­ta­tions are more or less dan­ger­ous? Does prim­ing influ­ence the ways that peo­ple respond to these changes?

Cather­ine Naugh­tie:
As pre­vi­ous stud­ies sup­port­ed the assump­tion that the­o­ret­i­cal under­stand­ings of atten­tion and work­load, and the find­ings of more gen­er­al per­tur­ba­tions stud­ies would apply in a dri­ving con­text, it would be rea­son­able to hypoth­e­size that find­ings from stud­ies elic­it­ing sim­i­lar atten­tion­al demands could be trans­fer­able to an auto­mo­tive domain. This meant that the ques­tions could be addressed with­out cost­ly sim­u­la­tion stud­ies and ben­e­fit from the diverse par­tic­i­pant pool avail­able through online experimentation.

Cather­ine Naugh­tie:
How­ev­er, though it was­n’t nec­es­sary to imple­ment this exper­i­ment on vehi­cle, it was impor­tant to design a task that repli­cat­ed the cog­ni­tive process­es that are asso­ci­at­ed with dri­ving. And this moves me on to point two, be ambitious.

Cather­ine Naugh­tie:
I devel­oped an exper­i­men­tal pro­to­col adapt­ing the Atari game, Pong, where I can manip­u­late the respon­sive­ness of the user con­trols dur­ing game play, incor­po­rat­ing an audi­to­ry dis­trac­tor task and pre­sent­ing visu­al primes. This effec­tive­ly address­es the research ques­tions that I was deal­ing with. How­ev­er, I quick­ly found out that imple­ment­ing the plan meant devel­op­ing it using JavaScript.

Cather­ine Naugh­tie:
Now, back then I saw cod­ing as some­thing oth­er peo­ple did. It was a for­eign syn­tax, com­plete­ly opaque to me. That’s changed now. And with­in weeks I was able to learn to code and build an online game for my exper­i­ment. And if I can, you can too.

Cather­ine Naugh­tie:
Learn­ing to code meant I could explode the true pow­er of Goril­la, not only to devel­op the games, but also design bespoke met­rics that could dra­mat­i­cal­ly sim­pli­fy my data analy­sis. For instance, I need­ed to use a for­mu­la to com­pute effi­cien­cy scores. Cre­at­ing a met­ric meant that I could cal­cu­late it auto­mat­i­cal­ly in the code and log the com­put­ed val­ue at each time step in the data out­put. This flex­i­bil­i­ty, sim­ply isn’t avail­able with stan­dard off the shelf approach­es. And nest­ing a code task in an exper­i­ment gives you the best of both worlds, the pow­er to build bespoke exper­i­ments and the ease of off the shelf tem­plates for sim­pler com­po­nents, such as demo­graph­ics or stim­uli pre­sen­ta­tion. But fig­ur­ing out how to do all this is still chal­leng­ing. And here I want­ed to stress the ben­e­fits of just using the community.

Cather­ine Naugh­tie:
In acad­e­mia, researchers often present meth­ods in a very san­i­tized way, that smooth over glitch­es, false starts and errors. And this can make learn­ing new meth­ods, incred­i­bly daunt­ing. Forums like the Goril­la Face­book group and Stack Over­flow are places where you can real­ly ben­e­fit from insights into run­ning exper­i­ments and sup­port solv­ing cod­ing prob­lems. I ben­e­fit­ed from post­ing and read­ing ques­tions in these forums, and from real­iz­ing that so many oth­er peo­ple had the same ques­tions that I did, even sea­soned pro­gram­mers. And open mate­ri­als, code libraries and jump­ing head first into these sup­port net­works will save you hours of frus­tra­tion and make the learn­ing process faster and much, much more enjoyable.

Cather­ine Naugh­tie:
At var­i­ous points in my pilot, I came across bugs. For instance, one, I called the nev­er nev­er-end­ing task, where errors in my code meant par­tic­i­pants end­ed up play­ing a game that last­ed for­ev­er rather than five min­utes, but all these bugs were solv­able. And in fact, the whole exper­i­ment was stream­lined in the process of debug­ging. And rec­og­niz­ing that errors are a nor­mal part of the process helped me gain con­fi­dence, and go from feel­ing over­whelmed to most­ly com­fort­ably devel­op­ing and debug­ging my own code. And most impor­tant­ly, run­ning a suc­cess­ful study.

Cather­ine Naugh­tie:
So, in sum­ma­ry, using online meth­ods and dri­ver behav­ior research has immense poten­tial. And my top tips for mak­ing the most of it are to for­mu­late online ques­tions, to har­ness all the ben­e­fits of the par­a­digm, to be ambi­tious and chal­lenge your­self, to devel­op new skills, and to lean in to the vibrant com­mu­ni­ty, grow­ing around online research to help you do that. Thank you very much for your attention.

Speak­er 2:
Thank you very much, Cather­ine. So, we have a ques­tion, I was think­ing of as well, from Camil­la, how did you go about learn­ing JavaScript from scratch in such a short time? Tell us of your wisdom.

Cather­ine Naugh­tie:
To be hon­est, it was seek­ing out help a lot. And I spent an awful lot of time on Stack Over­flow and I found some bril­liant YouTube videos, where peo­ple would walk through the process­es. And I basi­cal­ly spent a lot of time look­ing at oth­er peo­ple’s code and try­ing to work out how they were struc­tur­ing the syn­tax to do par­tic­u­lar things that I knew I need­ed to do. And I found games or dif­fer­ent appli­ca­tions that were doing… Because essen­tial­ly, I need­ed to make the pud­dles move up and down, and change the speed at which they did that based on when peo­ple were clicking.

Cather­ine Naugh­tie:
And so, I looked at games where they were doing sim­i­lar things and said, “How are they doing it?” When I did­n’t under­stand, I’d annoy devel­op­ers on Stack Over­flow say­ing, “How do you do this? How is that work­ing? How’s this work­ing?” And fail­ing a lot. And then through that, I start­ed work­ing out the log­ic behind it. And then I think once you’re doing it immer­sive­ly over that peri­od of time, you can actu­al­ly learn quite quick­ly. But yeah, it was a challenge.

Speak­er 2:
Fan­tas­tic. Thank you very much.

 

Get on the Waitlist

BeOnline is the conference to learn all about online behavioral research. It's the ideal place to discover the challenges and benefits of online research and to learn from pioneers. If that sounds interesting to you, then click the button below and sign up for our newsletter. You will be the first to know when we release new content and open applications for BeOnline 2022.

With thanks to our sponsors!

Jour­neys in the mind: the chal­lenges and ben­e­fits of using online meth­ods to under­stand dri­ver behav­iour and trans­port psychology