Jour­neys in the mind: the chal­lenges and ben­e­fits of using online meth­ods to under­stand dri­ver behav­iour and trans­port psychology

Cather­ine Naugh­tie:
Well, good after­noon. I’m Cather­ine Naugh­tie, a PhD research at the Uni­ver­si­ty of Bath. Today, I’m going to talk to you about my expe­ri­ence using online research in dri­ver behav­ior. And I recent­ly faced a chal­lenge that is doubt­less famil­iar to you, a ban on in-per­son research.

Cather­ine Naugh­tie:
Now, study­ing road user and dri­ver behav­ior in lock­down is real­ly chal­leng­ing as research in the field fre­quent­ly relies on instru­ment­ed vehi­cles, sim­u­la­tors, and nat­u­ral­is­tic obser­va­tion. So, what do we do when these options are not avail­able? But online meth­ods pro­vide a life­line, allow­ing us to con­tin­ue our research and more­over to lever­age the pow­er of mov­ing it online, to approach research dif­fer­ent­ly and engage with par­tic­i­pants in new ways.

Cather­ine Naugh­tie:
Today, I’m going to talk to you about my expe­ri­ence, adapt­ing my research and the approach I took, its ben­e­fits, and some of the chal­lenges I had to over­come. I want­ed to high­light three tricks I learned through this expe­ri­ence that helped me go from being a cod­ing novice to suc­cess­ful­ly build­ing and run­ning a code based exper­i­ment. One, to do online research, not sim­ply to do research online. Two, to be ambi­tious. And three, to use the com­mu­ni­ty. Hang on. The thing isn’t work­ing. Okay, there we go.

Cather­ine Naugh­tie:
So, what do I mean by do online research, not sim­ply doing research online? Well, in dri­ver behav­ior and trans­port research, there are some ques­tions that are best under­stood through high fideli­ty or nat­u­ral­is­tic meth­ods, such as eval­u­at­ing respons­es to a spe­cif­ic vehi­cle sys­tem or obser­va­tion­al stud­ies of ease good to use, and some ques­tions that can be answered using tra­di­tion­al in-per­son meth­ods, such as atti­tudes sur­veys, or road user interviews.

Cather­ine Naugh­tie:
How­ev­er, where there’s evi­dence for a behav­ioral effect in lit­er­a­ture, we can explore larg­er and more diverse sam­ples avail­able online to inves­ti­gate the under­ly­ing cog­ni­tive process­es involved, ask­ing ques­tions that are not so fun­da­men­tal, that it’s unclear how the mech­a­nism would oper­ate in a dri­ving task, or so spe­cif­ic that it would require a real­is­tic exper­i­ment set­up. But they’re just right, where enough is known to devel­op a sim­ple exper­i­ment that can form the basis of lat­er imple­men­ta­tions and eco­log­i­cal­ly valid con­text. This is where online meth­ods can add real value.

Cather­ine Naugh­tie:
To put this in con­text, con­sid­er my research project. This emerged from an oppor­tu­ni­ty. Most mod­ern vehi­cle con­trol sys­tems are elec­tron­ic, mean­ing it’s pos­si­ble to change how they respond to dri­ver inputs on the fly. There­fore, it’s the­o­ret­i­cal­ly pos­si­ble to cre­ate per­son­al­ized adap­tive vehi­cle con­trols. This could help reduce emis­sions, improve safe­ty and just make cars nicer to dri­ve, which is great, in theory.

Cather­ine Naugh­tie:
How­ev­er, research has shown that chang­ing the way that a vehi­cle behaves in response to dri­ver inputs can increase men­tal work­load, impaired atten­tion and dam­age per­for­mance. There­fore, to ful­ly real­ize the ben­e­fits of adap­tive sys­tems, we need to under­stand the cog­ni­tive and behav­ioral con­se­quences of actu­al­ly imple­ment­ing them.

Cather­ine Naugh­tie:
Now, pre­vi­ous stud­ies inves­ti­gat­ing per­tur­ba­tions in dri­ver vehi­cle inter­ac­tions used instru­ment­ed vehi­cles and sim­u­la­tors, as you can see on the image on the right. Now, this approach is cost­ly and it’s time con­sum­ing, and it’s def­i­nite­ly not some­thing you can do online. How­ev­er, from a behav­ioral per­spec­tive, the process­es that these stud­ies were inter­ro­gat­ing could be inves­ti­gat­ed using online meth­ods, includ­ing some of the most press­ing unan­swered ques­tions. Are there thresh­olds where changes become notice­able or impact per­for­mance? Are there times when adap­ta­tions are more or less dan­ger­ous? Does prim­ing influ­ence the ways that peo­ple respond to these changes?

Cather­ine Naugh­tie:
As pre­vi­ous stud­ies sup­port­ed the assump­tion that the­o­ret­i­cal under­stand­ings of atten­tion and work­load, and the find­ings of more gen­er­al per­tur­ba­tions stud­ies would apply in a dri­ving con­text, it would be rea­son­able to hypoth­e­size that find­ings from stud­ies elic­it­ing sim­i­lar atten­tion­al demands could be trans­fer­able to an auto­mo­tive domain. This meant that the ques­tions could be addressed with­out cost­ly sim­u­la­tion stud­ies and ben­e­fit from the diverse par­tic­i­pant pool avail­able through online experimentation.

Cather­ine Naugh­tie:
How­ev­er, though it was­n’t nec­es­sary to imple­ment this exper­i­ment on vehi­cle, it was impor­tant to design a task that repli­cat­ed the cog­ni­tive process­es that are asso­ci­at­ed with dri­ving. And this moves me on to point two, be ambitious.

Cather­ine Naugh­tie:
I devel­oped an exper­i­men­tal pro­to­col adapt­ing the Atari game, Pong, where I can manip­u­late the respon­sive­ness of the user con­trols dur­ing game play, incor­po­rat­ing an audi­to­ry dis­trac­tor task and pre­sent­ing visu­al primes. This effec­tive­ly address­es the research ques­tions that I was deal­ing with. How­ev­er, I quick­ly found out that imple­ment­ing the plan meant devel­op­ing it using JavaScript.

Cather­ine Naugh­tie:
Now, back then I saw cod­ing as some­thing oth­er peo­ple did. It was a for­eign syn­tax, com­plete­ly opaque to me. That’s changed now. And with­in weeks I was able to learn to code and build an online game for my exper­i­ment. And if I can, you can too.

Cather­ine Naugh­tie:
Learn­ing to code meant I could explode the true pow­er of Goril­la, not only to devel­op the games, but also design bespoke met­rics that could dra­mat­i­cal­ly sim­pli­fy my data analy­sis. For instance, I need­ed to use a for­mu­la to com­pute effi­cien­cy scores. Cre­at­ing a met­ric meant that I could cal­cu­late it auto­mat­i­cal­ly in the code and log the com­put­ed val­ue at each time step in the data out­put. This flex­i­bil­i­ty, sim­ply isn’t avail­able with stan­dard off the shelf approach­es. And nest­ing a code task in an exper­i­ment gives you the best of both worlds, the pow­er to build bespoke exper­i­ments and the ease of off the shelf tem­plates for sim­pler com­po­nents, such as demo­graph­ics or stim­uli pre­sen­ta­tion. But fig­ur­ing out how to do all this is still chal­leng­ing. And here I want­ed to stress the ben­e­fits of just using the community.

Cather­ine Naugh­tie:
In acad­e­mia, researchers often present meth­ods in a very san­i­tized way, that smooth over glitch­es, false starts and errors. And this can make learn­ing new meth­ods, incred­i­bly daunt­ing. Forums like the Goril­la Face­book group and Stack Over­flow are places where you can real­ly ben­e­fit from insights into run­ning exper­i­ments and sup­port solv­ing cod­ing prob­lems. I ben­e­fit­ed from post­ing and read­ing ques­tions in these forums, and from real­iz­ing that so many oth­er peo­ple had the same ques­tions that I did, even sea­soned pro­gram­mers. And open mate­ri­als, code libraries and jump­ing head first into these sup­port net­works will save you hours of frus­tra­tion and make the learn­ing process faster and much, much more enjoyable.

Cather­ine Naugh­tie:
At var­i­ous points in my pilot, I came across bugs. For instance, one, I called the nev­er nev­er-end­ing task, where errors in my code meant par­tic­i­pants end­ed up play­ing a game that last­ed for­ev­er rather than five min­utes, but all these bugs were solv­able. And in fact, the whole exper­i­ment was stream­lined in the process of debug­ging. And rec­og­niz­ing that errors are a nor­mal part of the process helped me gain con­fi­dence, and go from feel­ing over­whelmed to most­ly com­fort­ably devel­op­ing and debug­ging my own code. And most impor­tant­ly, run­ning a suc­cess­ful study.

Cather­ine Naugh­tie:
So, in sum­ma­ry, using online meth­ods and dri­ver behav­ior research has immense poten­tial. And my top tips for mak­ing the most of it are to for­mu­late online ques­tions, to har­ness all the ben­e­fits of the par­a­digm, to be ambi­tious and chal­lenge your­self, to devel­op new skills, and to lean in to the vibrant com­mu­ni­ty, grow­ing around online research to help you do that. Thank you very much for your attention.

Speak­er 2:
Thank you very much, Cather­ine. So, we have a ques­tion, I was think­ing of as well, from Camil­la, how did you go about learn­ing JavaScript from scratch in such a short time? Tell us of your wisdom.

Cather­ine Naugh­tie:
To be hon­est, it was seek­ing out help a lot. And I spent an awful lot of time on Stack Over­flow and I found some bril­liant YouTube videos, where peo­ple would walk through the process­es. And I basi­cal­ly spent a lot of time look­ing at oth­er peo­ple’s code and try­ing to work out how they were struc­tur­ing the syn­tax to do par­tic­u­lar things that I knew I need­ed to do. And I found games or dif­fer­ent appli­ca­tions that were doing… Because essen­tial­ly, I need­ed to make the pud­dles move up and down, and change the speed at which they did that based on when peo­ple were clicking.

Cather­ine Naugh­tie:
And so, I looked at games where they were doing sim­i­lar things and said, “How are they doing it?” When I did­n’t under­stand, I’d annoy devel­op­ers on Stack Over­flow say­ing, “How do you do this? How is that work­ing? How’s this work­ing?” And fail­ing a lot. And then through that, I start­ed work­ing out the log­ic behind it. And then I think once you’re doing it immer­sive­ly over that peri­od of time, you can actu­al­ly learn quite quick­ly. But yeah, it was a challenge.

Speak­er 2:
Fan­tas­tic. Thank you very much.