A Case Study of Repro­ducible Addic­tion Sci­ence Online

Eric Thrailkill 0:00
I have a laser point, great. So yes, thank you so much for the invi­ta­tion to talk about some of my work here. I will go through sort of like a sto­ry I tried to put togeth­er about it. So that’s gonna go some­what like how I found Goril­la exper­i­ment builder, which is sort of changed my research life. And, and then how just just doing a lit­tle back­ground on online research on sub­stance abuse risk fac­tors, which is what I’ve been doing.

And then pro­vide a case of that dri­ve look­ing at indi­vid­ual dif­fer­ences in loss aver­sion, and risk for cig­a­rette smok­ing and oth­er sub­stance abuse prob­lems. And then talk about some cur­rent direc­tions, I’m tak­ing this research and con­sid­er­a­tions and sort of ways I want to devel­op things. So thank you very much. I’ll proceed.

So dur­ing the pan­dem­ic, I was spend­ing a lot of time, well, ear­li­er in the pan­dem­ic, I should say, I was spend­ing a lot of time try­ing to fig­ure out how to do research online. I’ve been doing that a lit­tle bit before that, but the pan­dem­ic kick start­ed that. And so I was doing all the pro­gram­ming stuff and that sort of thing and try­ing to fig­ure every­thing out. And even­tu­al­ly, some­how, some­way, I came across this pod­cast online. And it was a webi­nar, actu­al­ly, by Joe Devlin, who was putting on a series of inter­views with aca­d­e­mics in psy­chol­o­gy, neu­ro­science, behav­iour­al sci­ences at Uni­ver­si­ty Col­lege Lon­don, on dif­fer­ent top­ics, and one of the top­ics was Goril­la exper­i­ment builder. So this is what intro­duced me to this tool, which I even­tu­al­ly got to use.

And I’ve been using ever since. And so in, in the venue I’ve been using, it is, is try­ing to study risk fac­tors asso­ci­at­ed with sub­stance abuse. And since well, before the pan­dem­ic, peo­ple in the sub­stance abuse research field have been inter­est­ed in online research. And so this is a paper, which seems kind of old now. But it came out 2019 pre pan­dem­ic, of course, on the use of crowd­sourc­ing addic­tion sci­ence, and what this paper goes over are, are a num­ber of areas where online research has been par­tic­u­lar­ly help­ful for set­ting sub­stance abuse prob­lems and peo­ple and so it goes over the the use of crowd sourc­ing to to get gath­er large, large sam­ples to basi­cal­ly repli­cate what we’ve seen in lab­o­ra­to­ry stud­ies, look­ing at case con­trol type designs, folks who are using sub­stances ver­sus folks who do not use sub­stances and repli­cat­ing those dif­fer­ent scales are a good place to start.

But oth­er users have entered devel­op more inter­ven­tion tools, and pilot test tools before using them in per­son with clin­i­cal sam­ples, and also devel­op new mea­sures and val­i­date them rapid­ly. And then they also are able to con­tact peo­ple over and over and not have them come into the lab­o­ra­to­ry for lon­gi­tu­di­nal mea­sure mea­sure­ment, which is extreme­ly use­ful. So what this paper end­ed up con­clud­ing was that online addic­tion research is going to be com­ple­men­tary to clin­i­cal tri­als, human lab­o­ra­to­ry stud­ies, epi­demi­o­log­i­cal stud­ies, and it’s going to help over­all ben­e­fit the field by improv­ing repro­ducibil­i­ty, rigour and expand­ing pos­si­bil­i­ties, the study fac­tors relat­ed to sub­stance sub­stance abuse, and over­all health relat­ed behaviours.

And so with that sort of back­ground, I was inter­est­ed in in using this and so this is just a fig­ure show­ing that if you search, the num­ber of papers being pub­lished and num­ber cita­tions in most papers, it seems to be increas­ing expo­nen­tial­ly and just sort of enter­ing easy terms like Mechan­i­cal Turk, addiction

4:59
and so on. Here’s this paper that was the even­tu­al result of me get­ting inter­est­ed in goril­la exper­i­ment builder. And so I was inter­est­ed in loss aver­sion. And look­ing at it, and cig­a­rette smok­ers. And so just to give a lit­tle bit of back­ground, as we are all pret­ty famil­iar with from all the talks today, behav­iour­al eco­nom­ics is just inte­grat­ing psy­chol­o­gy into study of choice and deci­sion mak­ing that peo­ple make. And it’s, it’s been pret­ty obvi­ous for a long time for many, many peo­ple that real life behav­iour is not con­form­ing to eco­nom­ic predictions.

And so one exam­ple of this this was just described in the pre­vi­ous talk is that poten­tial loss­es have a larg­er impact on our choic­es that poten­tial gains that are oth­er­wise equiv­a­lent. And this is what I call loss­es or what’s called loss aver­sion loss averse behav­iour, we behave as if the loss­es are hav­ing a larg­er effect on our behav­iour, they are on our on our val­u­a­tion than gains.

And you might be able to think about loss aver­sion as a poten­tial pro­tec­tive fac­tor against the loss­es that inevitably inevitably hap­pen in rela­tion to our health from engag­ing in behav­iours such as sub­stance abuse, and there are some in per­son stud­ies that actu­al­ly sug­gest this and so with peo­ple who are drink­ing in excess or using cocaine prob­lems who have prob­lems with these types of behav­iours, stan­dard mea­sures of loss aver­sion have found low­er lev­els of loss aver­sion among these groups in com­par­isons to matched con­trol groups are groups of peo­ple who are oth­er­wise matched on socio demo­graph­ic vari­ables such as edu­ca­tion­al attain­ment, gender.

They’re show­ing, in com­par­i­son, low­er lev­els of loss aver­sion, mean­ing that they are behav­ing as if poten­tial loss­es are hav­ing less effect or a sim­i­lar amount of effect on their behav­iour as poten­tial gains. And so, in addi­tion to loss aver­sion, or oth­er impor­tant deci­sion mak­ing fac­tors is one of them that’s par­tic­u­lar­ly well stud­ied in sub­stance abuse research is delay dis­count­ing or the deval­u­a­tion of rewards with the delay to their fea­ture receipt.

And so it’s been doc­u­ment­ed since in the 1990s, that indi­vid­u­als who are using hero­in or smok­ing cig­a­rettes or using cocaine, so on and so forth, lots of unhealthy behav­iours have steep­er or high­er delay dis­count­ing of the future rewards asso­ci­at­ed with them in com­par­i­son to peo­ple who are oth­er­wise matched, but are not using these substances.

So the study that I did was was that after look­ing at the research on loss aver­sion that was out there, it was com­par­i­son, com­par­a­tive­ly less devel­oped research on delay dis­count­ing. Loss aver­sion stud­ies, sub­stance abuse dis­or­ders, are not had not account­ed for delayed dis­count­ing, you know, because, you know, peo­ple are, these are, you did­n’t know whether these fac­tors are account­ing for one anoth­er, or sep­a­rate from one anoth­er, are going on inde­pen­dent­ly in influ­enc­ing behav­iour. And none of the stud­ies on loss aver­sion had exam­ined cig­a­rette smok­ing, which we know is high­ly comor­bid with these oth­er sub­stance use prob­lems, but had not been exam­ined by itself when it is, of course, relat­ed to hun­dreds and hun­dreds of 1000s of deaths every year. And so it’s very impor­tant to under­stand cigarettes.

9:08
So we do this study, using real exper­i­ment build­ing. So we set this up in a pret­ty straight­for­ward way. We had some basic demo­graph­ic and health ques­tions peo­ple accept­ed the study on mechan­i­cal Turk. We did not tell them that it was about smok­ing we told them that it was about gen­er­al health and choic­es. And so we had asked them ques­tions about cig­a­rette smok­ing, but also about drink­ing about drug use about whether they sleep well at night, whether they have prob­lems with being depressed. We did­n’t make it clear to them up front that this would be about smok­ing but sep­a­rat­ed them based on their answer to the smok­ing question.

And then after doing that, they com­plet­ed tasks we had a sim­ple mixed gam­ble task, which was a hypo­thet­i­cal coin flip between a poten­tial loss or poten­tial gain. It was not con­se­crat­ed mean­ing that they did­n’t actu­al­ly get shown whether they won the gain amount or loss amount. It was just would you accept this gam­ble as a yes or no question.

And then we also use the stan­dard mea­sure to mea­sure delay dis­count­ing this mon­e­tary choice ques­tion­naire, which has been stud­ied in many dif­fer­ent set­tings, and many, many stud­ies in the past. So we’re able to mea­sure both of these fac­tors. And we includ­ed delay dis­count­ing, because we know already that smok­ers have steep­er delayed dis­count­ing than non smok­ers or nev­er smok­ers, that’s well estab­lished. So this pro­vid­ed a pos­i­tive con­trol to tell us that we’re actu­al­ly get­ting peo­ple who are cig­a­rette smokers.

And then, we tar­get­ed to get 200 peo­ple in each group, Mechan­i­cal Turk, the two groups were peo­ple who are cur­rent­ly smok­ing cig­a­rettes, or peo­ple who had nev­er smoked cig­a­rettes, as defined as hav­ing smoked less than 100 cig­a­rettes in their life­time. And they’re not cur­rent­ly smok­ing or using oth­er tobac­co prod­ucts. And our cur­rent cig­a­rette smok­ers were required to say that they’re also not using cur­rent­ly, tobac­co prod­ucts oth­er than cigarettes.

Okay, and then we, we attempt­ed to strat­i­fy the groups on gen­der and edu­ca­tion­al attain­ment. And then we includ­ed stan­dard bot checks tak­en from the sam­ple mate­ri­als on Goril­las web­site. And we had a sort of infor­ma­tion sheet that had to be checked that if they did­n’t check it, they weren’t able to move for­ward in the study. And so this is anoth­er sort of point where peo­ple can be select­ed for.

So we use Goril­la exper­i­ment builder to do this. The basic design of the study was that we had fac­tors right so smok­ing sta­tus, cur­rent­ly smok­ing, nev­er smoked and got a task order, whether they got the delay dis­count­ing first or loss aver­sion first. And then we had two ver­sions of the loss aver­sion task at that point to real­ly get into but we had peo­ple com­plete dif­fer­ent con­di­tions of it, in order to pro­vide a more rig­or­ous mea­sure of their loss averse behav­iour, or lack thereof.

And so what this actu­al­ly end­ed up look­ing like, I’m not gonna include the actu­al pic­ture, and it’s even more com­pli­cat­ed, some­thing like this, if you want to see the actu­al exper­i­men­tal exper­i­ment on goril­la, you can go to this QR code and all these mate­ri­als are avail­able. For free, freely avail­able on open mate­ri­als, all the tasks and exper­i­ment design are avail­able for any­body to look at.

So we had our ini­tial ques­tion­naires. And based on that there is sep­a­rat­ed into groups based on smok­ing sta­tus. And then we have three lev­els of edu­ca­tion­al attain­ment, high school or low­er, some col­lege, or col­lege grad­u­ates. And then we had three lev­els of gen­der, male, female or oth­er iden­ti­fy­ing. So you can see sort of com­plex the com­plex­i­ty increas­es. And to the point where we’ll just skip to it that we had 56 quo­tas over I’m sort of proud of that, because it seems like a lot, but it actu­al­ly was actu­al­ly it was very neat­ly organ­ised and easy to work with. So it’s very cool. And out of this, we got data. So we screened lots and lots of peo­ple exclud­ed lots and lots of people

14:01
based on our quo­tas, require­ments, but we’re able to keep track of all that pret­ty eas­i­ly. And we even­tu­al­ly got pret­ty close to meet­ing our goals in terms of the size of this groups, for smok­ers and nev­er smokers.

So here’s a sort of sam­ple demo­graph­ic table on this side. We tried our best to match on gen­der and edu­ca­tion­al attain­ment. We did­n’t quite get there because the preva­lence of peo­ple who who report that they have a high school diplo­ma or less on Mechan­i­cal Turk is very low. So it’s very dif­fi­cult to find peo­ple who have a low lev­el of edu­ca­tion­al attain­ment on Mechan­i­cal Turk, just sort of a quirk of the plat­form. But any­ways, we got pret­ty close we includ­ed these vari­ables in our analyses.

Any­ways, So here are the actu­al results. So I’m show­ing here on the left the screen­shots of what’s some­body would get these two dif­fer­ent tasks. So it’s, it’s very sim­ple. And the data are show­ing here that peo­ple who on this gam­bling task peo­ple who report­ed that they’re cur­rent­ly smok­ing are accept­ing more gam­bles. And that’s our sim­ple mea­sure of loss, of behav­iour. They’re less loss averse. They’re accept­ing more risky gam­ble’s that are giv­ing them poten­tial for worse out­comes on their, on their gam­bles, so.

Nev­er smokeds, they are when the gain the poten­tial gain amount is twice as much poten­tial loss and now they’re accept­ing about exact­ly half those gam­bles that’s con­sis­tent with loss aver­sion. So they’re per­fect­ly loss averse in the nev­er smoked when cur­rent smok­ing peo­ple are less loss averse. We find also in our con­trol mea­sures like dis­count­ing, we are see­ing that we have steep­er delayed dis­count­ing and high­er dis­count­ing rate in the cur­rent­ly smok­ing indi­vid­u­als per­son, ver­sus those are not or had nev­er smoked. So this was our main result of the study.

And so what have I done since this, we’ve repli­cat­ed this find­ing using the same sort of approach, exper­i­ment. Design, we’re includ­ing more detail on sub­stance use oth­er risk fac­tors, look­ing into alco­hol use and drug use in more detail. We’re look­ing at oth­er health rel­e­vant behav­iours such as seden­tary lifestyle, ask­ing ques­tions about peo­ple’s phys­i­cal activ­i­ty and look­ing at behav­iour­al eco­nom­ic risk fac­tors and those whether those increased risk for being seden­tary. Refin­ing our meth­ods, includ­ing oth­er sorts of tools, such as IP hub, Qualtrics screen­ing tools, and I’m just attempt­ing to keep up with real­is­ing fea­tures that are com­ing out and try­ing to train oth­er peo­ple that I work with, on how to use these tools. And going forward.

We’ve heard a lot about gam­i­fi­ca­tion ear­ly on, in these ses­sions. And I think gam­i­fi­ca­tion is not just for kids, I think we should def­i­nite­ly use it to include par­tic­i­pant engage­ment, next types of stud­ies, and just try­ing to inte­grate more of the tools and keep up with what’s going on.

And I’m in par­tic­u­lar­ly just want to point out that going for­ward, I’m very much inter­est­ed in lon­gi­tu­di­nal mea­sure­ment with this sorts of thing, keep­ing in touch with peo­ple, because that’s also going to increase the qual­i­ty of respons­es. You know, if you have some­body that’s com­ing back time and time again, to com­plete these ques­tion­naires, tasks, then it’s pret­ty like­ly that their qual­i­ty par­tic­i­pants, not being a robot. So with that, I want to thank every­body who’s involved in the stud­ies, par­tic­u­lar­ly Jo, for invit­ing me and fund­ing if you want to get in touch with me here’s my email address, also get in touch with me on Twit­ter. Thank you very much.

Jo Ever­shed 18:21
Eric, that was fan­tas­tic. Thank you. I think you’re absolute­ly right, that gam­i­fi­ca­tion is a way that we can build as each researcher could build them­selves a library of par­tic­i­pants that’s inter­est­ed in their in their research and bring them again and again, to come back in a way that does­n’t feel so hor­ri­bly demand­ing. Because you know, it. We need to find our tribe, both in research, but also in terms of the par­tic­i­pants who wants to take part in our research and con­tribute to it. I mean, I’m sure we also need sep­a­rate sam­ples occa­sion­al­ly to val­i­date it just to make sure that’s not a weird sam­ple. But I think there’s def­i­nite­ly place for that there. Have you done any lon­gi­tu­di­nal research already or not?

Eric Thrailkill 19:04
Not yet, I’m sort of get­ting ready to do to launch study on that pret­ty soon. But just start­ing very sim­ply, 2 measurements.

Jo Ever­shed 19:16
So I did have anoth­er ques­tion, because I maybe you said it in your talk. Is it that peo­ple with every­body else while I’m ask­ing this ques­tion, if you’ve got a ques­tion for Eric, there was a high­ly tech­ni­cal one, but oth­ers wel­come as well, in the q&a, please now. Do you? Is it sort of ask­ing about causal­i­ty here? Is it that peo­ple with risky deci­sion mak­ing are more like­ly to smoke or is it that smok­ers are riski­er deci­sion mak­ing? Do we know which order that hap­pens in and is that a ques­tion? And if not, how could we answer it?

Eric Thrailkill 19:48
So that’s one of the things I’m try­ing to pur­sue and so there are two ways or a cou­ple of ways to do it. There’s prob­a­bly more than what I’m going to men­tion and and the first is to start with peo­ple who’ve nev­er had, who nev­er had any expo­sure to smok­ing or sub­stance use or such as, as ado­les­cents and fol­low them, as they sort of devel­op into, you know, try­ing cer­tain sub­stances and, you know, keep track of whether they’re, you know, deci­sion mak­ing, pre­dicts that those tran­si­tions or pre­dicts their their like­li­hood of sam­pling substances.

Anoth­er way to approach this exper­i­men­tal­ly, see what if you whether you have a sort of method for chang­ing deci­sion mak­ing, whether that trans­lates to an imme­di­ate change in their sort of sub­stance use behav­iour. Those, again, in in the delay dis­count­ing area, those have been both pur­sued. And it seems like if you can influ­ence peo­ple’s episod­ic future think­ing, you know, they’re think­ing about the future self, and things they would like to do. And you have them doing that, while they have an oppor­tu­ni­ty to smoke, those sorts of cues can result in less smoking.

And so that’s been done lab­o­ra­to­ry stud­ies, peo­ple are try­ing to devel­op that for more sort of appli­ca­tion in real life, to present cues to peo­ple in real life to get them to when they’re about to smoke with very high lev­els of crav­ing sweets of things. It’s less well devel­oped with loss aver­sion there as well, but seems like you could pic­ture some ways to bring peo­ple’s atten­tion to the poten­tial loss­es that could be there are going to hap­pen to them if they con­tin­ue behav­ing a cer­tain way, such as fam­i­ly, friends, and those sorts of things, longevi­ty, that might help them inhib­it cer­tain behav­iours in the moment, but it’s just less well developed.

And so one thing that ear­ly on was shown with delay dis­count­ing and smok­ers that’s a for­mer smok­ers dis­count future works in the same way as nev­er smok­ers. And so it seems like quit­ting might be relat­ed to sort of repair. And these sorts of deci­sion mak­ing areas we’ve seen recent­ly. It’s not pub­lished yet but the same sort of things going on with loss aver­sion, sort of for­mer smok­ers or, or loss averse in the same way as nev­er smok­ers are.

Jo Ever­shed 22:59
Fan­tas­tic. That was a very com­pre­hen­sive answer, which was fas­ci­nat­ing. There are two ques­tions in the chat in the q&a for you, which I’m going to leave you to answer inde­pen­dent­ly. Because we in the inter­est of time, we now need to move on to Casey Roark. Eric, thank you so much for join­ing us here today. Please stay for the rest of the session.