Online research with chil­dren with and with­out an intel­lec­tu­al disability

Kather­ine Ellis, Uni­ver­si­ty of Sur­rey
@DrKrellis

YouTube

By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

The pan­dem­ic has meant that face-to-face research with spe­cial pop­u­la­tions is not viable and there remains uncer­tain­ty as to when direct assess­ments with these under­rep­re­sent­ed groups can restart. Our research team has embarked on apply­ing our pre­vi­ous face-to-face pro­to­col, which includes eye-track­ing, cog­ni­tive and behav­iour­al assess­ments, to online meth­ods with neu­rotyp­i­cal chil­dren, and chil­dren with genet­ic syn­dromes and intel­lec­tu­al disability.

I will dis­cuss our expe­ri­ences of mov­ing research online and work­ing with care­givers to sup­port data col­lec­tion with their child over a video-link. I will focus on the ben­e­fits and chal­lenges of online research, out­lin­ing tech­niques on how to col­lect online data from groups that are tra­di­tion­al­ly dif­fi­cult to include in research more broad­ly. I will com­pare the qual­i­ty of the data and find­ings with pre­vi­ous­ly col­lect­ed face-to-face data from the same groups.

Online research reduces bur­den for fam­i­lies with addi­tion­al needs by reduc­ing trav­el and enabling greater flex­i­bil­i­ty. It also enables researchers to reach a greater num­ber of peo­ple from rare and/or dif­fi­cult to reach groups. How­ev­er, we need to con­sid­er these ben­e­fits rel­a­tive to the integri­ty of the data. This work will con­tribute to cost/benefit analy­ses on when to apply online tech­niques in spe­cial pop­u­la­tions in the future.

Full Tran­script:

Kather­ine Ellis:
Hi, every­one. My name is Kather­ine Ellis. I’m a research fel­low in the Neu­rode­vel­op­men­tal Dis­or­ders Lab, which is led by Dr. Jo Moss at the Uni­ver­si­ty of Sur­rey. I’m going to be talk­ing today about how to run research online with a par­tic­u­lar focus on chil­dren with an intel­lec­tu­al dis­abil­i­ty using eye-track­ing studies.

Kather­ine Ellis:
A bit of back­ground about what we do in the lab. So we are inter­est­ed in genet­ic syn­dromes that are asso­ci­at­ed with an intel­lec­tu­al dis­abil­i­ty, but we’re more inter­est­ed in them because of their dis­tinct pro­files of autis­tic traits and these dif­fer between one anoth­er and also those with [broad 00:00:39] syn­dromic autism. So what we do is look at vari­abil­i­ty at the social cog­ni­tive lev­el across these groups, which might explain per­haps the dif­fer­ences in autis­tic traits, with the hope that this work is going to help to lead to the devel­op­ment of more indi­vid­u­al­ized sup­port for these groups.

Kather­ine Ellis:
So one of the big chal­lenges of work­ing with these pop­u­la­tions, which are already real­ly under­rep­re­sent­ed in research, is find­ing tasks that can give us a real­ly good deep phe­no­typ­ing, social cog­ni­tive, phe­no­typ­ing infor­ma­tion, but also reli­able and valid, but are appro­pri­ate for peo­ple with quite lim­it­ed lan­guage, atten­tion and motor skills. And this is where eye-track­ing has been real­ly use­ful. So we’ve got a cou­ple of eye-track­ing tasks that look at social cog­ni­tive abil­i­ties, such as today’s fol­low­ing task. And this works real­ly well in all groups because it’s a com­plete­ly pas­sive par­a­digm. They don’t have to do any par­tic­u­lar tasks, but we can get real­ly nice, detailed infor­ma­tion about what they’re look­ing at. And this helps us to under­stand how they might be pro­cess­ing the world around them with­out them hav­ing to tell us in language.

Kather­ine Ellis:
Pri­or to the pan­dem­ic, we were doing this face to face using a Tobii lap­top, and this was work­ing real­ly well. It was trans­portable. We could take it to peo­ple’s homes. We could sit on the floor with the kids. And we were find­ing that the par­a­digms that we were using were work­ing with the neu­ro-typ­i­cal con­trol groups, and we were start­ing to find some real­ly nice, inter­est­ing ear­ly online research in these groups. Obvi­ous­ly, with the pan­dem­ic, it’s just not eth­i­cal to do these sorts of pop­u­la­tions, so this is what gave us the push, as many peo­ple to work and see whether we could do this online. So we’ve been using the Goril­la web eye-track­ing zone, which, for those of you who don’t know, uti­lizes WebGaz­er. So this uses par­tic­i­pants’ own web cam­era to detect their face, and then uses this infor­ma­tion to give an esti­mate of where peo­ple are look­ing on the screen.

Kather­ine Ellis:
And what you might notice is that I haven’t just got that icon for Goril­la, I’ve also got an icon for Teams, and this is because for these pop­u­la­tions, being able to com­mu­ni­cate with par­ents is real­ly, real­ly crit­i­cal. So yes, in the­o­ry, we could just send links off to par­ents and say, “Give it go with your kids. See if you can get them to watch a video and do the cal­i­bra­tion.” But for these par­tic­u­lar pop­u­la­tions who have a lot of addi­tion­al needs, and often you need quite skilled researchers to get data face to face, when you’re mov­ing every­thing online.

Kather­ine Ellis:
Actu­al­ly, what our role as researchers has moved to is actu­al­ly being able to coach the par­ents to do our job essen­tial­ly, to be the researchers. And this is sort of the key point that I real­ly want to empha­size. So what we ask par­ents to do, if they’re lucky enough to have a sec­ond device, is to set that up with a Teams link, and then what we’ll do is coach them through set­ting up every­thing on their lap­top, come up with a plan before the child has even come into the room, to make sure that we try and make every­thing run smooth­ly as pos­si­ble, because the aim is to make sure that the child is sit­ting for as lim­it­ed amount as pos­si­ble. And we don’t want them to sit there for any longer than is necessary.

Kather­ine Ellis:
But it’s not just sort of dur­ing the test­ing ses­sion where we will do this, actu­al­ly com­mu­ni­ca­tion is real­ly vital through­out the whole research process. And it starts from per­haps your first recruit­ment call. And this is where you’re real­ly start­ing to estab­lish a part­ner­ship with that par­ent, and you estab­lished a strat­e­gy that you’re going to work with before you actu­al­ly have the actu­al test­ing ses­sion. Every child is dif­fer­ent, and it’s talk­ing through, “What’s going to moti­vate them? What’s going to make them feel the most com­fort­able? And what’s going to help the par­ent in that situation?”

Kather­ine Ellis:
Fol­low­ing for that, you want to pro­vide some resources over email, so this is where you might pro­vide some more detailed infor­ma­tion about things like inter­net speeds. But what’s real­ly been help­ful at this point has been send­ing par­ents an eye-track­ing dum­my link. So actu­al­ly they can have a go at the eye-track­ing them­selves, see how it works, see what does­n’t work. And this helps par­ents feel a lot less anx­ious about what hap­pens in the actu­al test, say, because they feel they under­stand what’s going to hap­pen, even if it is rel­a­tive­ly straightforward.

Kather­ine Ellis:
And then hope­ful­ly by that point, by the time that you get to the test­ing ses­sions, the par­ents feel sup­port­ive, you’re there for any tech issues, and it’s just ensur­ing that there’s a calm envi­ron­ment to make sure kids feel relaxed and par­ents feel relaxed, because if you have a stressed par­ent or stressed child, you’re not going to get good data.

Kather­ine Ellis:
And I just want­ed to see high­light here one of the biggest chal­lenges that we have, which is achiev­ing cal­i­bra­tion. And this is because I think in Goril­la, WebGaz­er is in beta mode at the moment. It’s not as for­giv­ing as per­haps the cal­i­bra­tion pro­ce­dure as in the Tobii lap­top. And this has been one of the major bar­ri­ers that have become, not just in child research, but I think in adult research as well. And I just want­ed to show you some of the infor­ma­tion that we send to par­ents, which we were giv­en from Sarah White’s lab, who very kind­ly passed this onto us, and just high­light all the num­ber of things that you have to think about before even get­ting start­ed to make sure that the sit­u­a­tion is set up as best as pos­si­ble, to make sure every­thing runs real­ly, real­ly smoothly.

Kather­ine Ellis:
So you might be think­ing at this point, “Gosh, this is a lot of effort. Is it worth it with these pop­u­la­tions?” And if it works, actu­al­ly it def­i­nite­ly is and has some advan­tages. And I think hope­ful­ly our work will lead to some reflec­tions on a cost ben­e­fit analy­sis of who and when to do this research with. So one of the real­ly good things about online research with these par­tic­u­lar groups is the amount of bur­den that it reduces for fam­i­lies who often have quite a lot of addi­tion­al bur­dens for car­ing with a child with addi­tion­al needs. The main thing being lack of trav­el, extra flex­i­bil­i­ty and less pres­sure to get every­thing done, con­sid­er­ing that the groups that we work with are often spread around the coun­try because they’re quite rare. And yes, there are chal­lenges, but there are ways of get­ting around it.

Kather­ine Ellis:
So of course, any sort of good cost ben­e­fit analy­sis is going to be depen­dent on the data. And we are get­ting some pos­i­tive results. We’re find­ing that actu­al­ly kids are able to get past that cal­i­bra­tion, and that’s a huge achieve­ment, I think. So a lot of the typ­i­cal­ly devel­op­ing kids are cal­i­brat­ing, and we’re get­ting use­ful infor­ma­tion out of them. And then for the syn­drome groups, actu­al­ly, about two thirds are man­ag­ing to cal­i­brate, and I’m absolute­ly thrilled about this because although it might not be as high as the face-to-face cal­i­bra­tion, that’s eight out of 12 kids who haven’t had to come to the lab and have addi­tion­al bur­dens. I think this is real­ly, real­ly great news.

Kather­ine Ellis:
And what I want to do is I’ve start­ed to draw out the descrip­tive char­ac­ter­is­tics of the syn­drome chil­dren who are being suc­cess­ful and who aren’t. And there’s no clear pic­ture at the moment because there’s a lot of over­lap in these char­ac­ter­is­tics. So my mes­sage is, “Just give it a go at this moment.”

Kather­ine Ellis:
And then, of course, we want to find out, “Is the data mean­ing­ful? Is it valid?” We don’t have the sam­ple size yet to say for cer­tain whether the par­a­digms that we are using are work­ing online, but we know that they are work­ing in adult pop­u­la­tions. And in terms of a sim­ple social video view­ing par­a­digm, we’re see­ing pat­terns of behav­ior which is sim­i­lar to pre­vi­ous find­ings, par­tic­u­lar­ly in the Frag­ile X group. So they are watch­ing the social scenes, they’re just doing same times with short­er episodes, and this is per­haps what we would expect with this group, so that’s [inaudi­ble 00:08:22]. So thank you for lis­ten­ing. Do let me know if you have any ques­tions, drop me an email. I’ve got heaps of sto­ries that I can’t share in eight minutes.

Sophie Scott:
Thank you very much. Thank you. Well, the ques­tion I have is about [inaudi­ble 00:08:41], which is about the sam­ples cal­i­bra­tion data, which did look real­ly inter­est­ing, but how often did you find that you need­ed to recal­i­brate? And how long were your ses­sions in total?

Kather­ine Ellis:
So our videos are real­ly short, so they were only about four to five min­utes, and we haven’t had any reports. I’ve watched them. We haven’t had any reports of actu­al­ly when they’ve got past the cal­i­bra­tion that they’ve had to recal­i­brate. That’s real­ly, real­ly promis­ing. So that’s a real­ly good sign, because I think there was a huge ques­tion of whether you’d be able to do any­thing like this with this population.

Sophie Scott:
Yeah.

Kather­ine Ellis:
And then, sor­ry, was there a sec­ond part of the question?

Sophie Scott:
Yeah. How long are your ses­sions in total?

Kather­ine Ellis:
So as I said, the actu­al video is real­ly, real­ly short. The range of ses­sions real­ly depends on the child. And as I said, the good thing is we can be flex­i­ble. So for some kids, 20 min­utes is the absolute lim­it, and that’s the range of behav­ioral and cog­ni­tive tasks. For some, they can do it a lit­tle bit longer. And you just have to be very flex­i­ble in terms of how long or how lit­tle you do. And one of the things that I did­n’t men­tion is some­times you find that par­ents tend to be a bit more con­cerned about whether kids do it or not. So as a research, what’s real­ly impor­tant is mak­ing sure when to jump in and say, “Do you know what? I think we’ve done a lot. Why don’t we try it anoth­er day?”

Sophie Scott:
Yeah. Well, thank you very much. And don’t for­get, you can con­tin­ue ask­ing ques­tions in the Q&A, and those be answered by the speak­ers like Kather­ine after they fin­ish talk­ing. Thank you very much, Katherine.

 

Get on the Waitlist

BeOnline is the conference to learn all about online behavioral research. It's the ideal place to discover the challenges and benefits of online research and to learn from pioneers. If that sounds interesting to you, then click the button below and sign up for our newsletter. You will be the first to know when we release new content and open applications for BeOnline 2022.

With thanks to our sponsors!

Online research with chil­dren with and with­out an intel­lec­tu­al disability