PONT: A Pro­to­col for Online Neu­ropsy­cho­log­i­cal Testing

Will Saban, Berke­ley Uni­ver­si­ty of California

YouTube

By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

Full Tran­script:

Will:
Let me move this. Okay.

Will:
My name is Will. I’m a post­doc at Berke­ley, and I will talk today about PONT, a pro­to­col for online neu­ropsy­cho­log­i­cal testing.

Will:
A lit­tle bit of an intro­duc­tion. The lit­er­a­ture is focused main­ly on the cor­tex when we look con­di­tion domains of stud­ies from atten­tion to arith­metic to motor sequence learn­ing. While the con­tri­bu­tion of sub-cor­ti­cal brain mech­a­nisms to [inaudi­ble 00:00:29] remain unclear, we see from Stan­ford the “cor­tic­o­cen­tric” neu­ro­science bias. Sub­cor­ti­cal struc­tures such as the basal gan­glia, the cere­bel­lum, the thal­a­mus and more might also be involved in cognition.

Will:
And one solu­tion to this prob­lem is to do more stud­ies involv­ing patients with sub­cor­ti­cal pathology.

Will:
Cur­rent­ly, I’m focus­ing on two sub­cor­ti­cal regions. One is the cere­bel­lum, using patients with spin­ocere­bel­lar atax­ia, SCA, and the oth­er sub-cor­ti­cal region is the basal gan­glia, BG, using patients with Parkin­son’s disease.

Will:
But we have anoth­er prob­lem with this solu­tion. We are deal­ing with a very lim­it­ed resource. We are deal­ing with the patients themselves.

Will:
First, it’s very dif­fi­cult to iden­ti­fy and recruit these indi­vid­u­als for projects. But let’s say you suc­ceed to recruit them into your exper­i­ment. Their abil­i­ty to par­tic­i­pate in a research project is very restrict­ed by their med­ical con­di­tions, for exam­ple, their motor impair­ments. And as a result, if you will do a lit­er­a­ture search, you will find papers with very small sam­ple sizes, such as 6, 7, 8 par­tic­i­pants. But not only that, it takes quite a bit of time to com­plete a sin­gle study, let alone a pack­age of stud­ies that might allow a more com­pre­hen­sive sto­ry with the appro­pri­ate con­trol participants.

Will:
Just the indi­ca­tion to show you how much the prob­lem is seri­ous. If you will do a PubMed search result and you will enter two key terms in PubMed, one is neu­ropsy­cho­log­i­cal study, the oth­er is fMRI stud­ies, this is what you will get. So, Y axis is the count, the num­ber of results. The X axis, the num­ber of the year.

Will:
And you can see how many more results we get for fMRI com­pared to neu­ropsy­cho­log­i­cal study. And I think the trend is also very clear.

Will:
To address is prob­lems, in my first year in my post-doc, I’ve devel­oped this pro­to­col, a pro­to­col for online neu­ropsy­cho­log­i­cal test­ing. I like to call it PONT. Pont is a bridge in France, by the way.

Will:
And as a first step, I recruit­ed a team of research assis­tants that helped me to build protocol.

Will:
And the PONT pro­to­col involves five main steps.

Will:
Step num­ber one, we reach out to sup­port groups from all across the Unit­ed States, from LA to New York. And we wrote an email to the group man­agers ask­ing to trans­fer the infor­ma­tion to the group mem­bers about the project. And step num­ber two, each one of the group mem­bers reach out to us say­ing that they want to par­tic­i­pate in our project. In step num­ber three, each indi­vid­ual is doing an online neu­ropsy­cho­log­i­cal assess­ment in which we assess med­ical con­di­tions assess­ment, we assess motor abil­i­ties, cog­ni­tive abil­i­ties. It’s about a one hour ses­sion. And step num­ber four, each indi­vid­ual is get­ting an email with a spe­cif­ic link and a spe­cif­ic ID. And they can do the exper­i­ment when­ev­er they want. When they com­plete the exper­i­ment, we asked for some feed­back and we paid them for the participation.

Will:
These are the five main steps of PONT.

Will:
In the first 10 months of the project, we were able to recruit about 420 par­tic­i­pants for our data­base, 150 with Parkin­son’s dis­ease (PD), 150 SCA, spin­ocere­bel­lar atax­ia, and 120 con­trol participants.

Will:
And the goal of the project was to do dif­fer­ent exper­i­ments to assess both cog­ni­tive abil­i­ties and motor abil­i­ties. In using PONT, the idea is to assess dif­fer­ent neur­al mech­a­nisms in a wide range of abilities.

Will:
And now we’ll focus on the motor project.

Will:
So, this is a con­crete exam­ple of a DSP task, a dis­crete seg­ments pro­duc­tion test, in which par­tic­i­pants are asked to put their fin­gers on the key­board, such that each fin­ger is on the appro­pri­ate but­ton as fol­low­ing. And then they need to exe­cute the sequence as fast as possible.

Will:
A lit­tle bit of back­ground about sequence learn­ing. Most research has focused on the effect of mem­o­ry on sequence exe­cu­tion, basi­cal­ly by com­par­ing between repeat­ing sequences to ran­dom sequences. And there is a lot of stud­ies. Large­ly, they touch on the involve­ment of basal gan­glia (BG) and cere­bel­lum on sequence exe­cu­tion, though there is one caveat here as two stud­ies by [Wat­ten­berg 00:05:15] recent­ly have sug­gest­ed that the activ­i­ty of the Parkin­son’s dis­ease lit­er­a­ture is equiv­o­cal, mean­ing peo­ple saw mixed results. Some­times peo­ple with Parkin­son’s were unim­paired and some­times not.

Will:
And as I see it, there is five main cycles involved in sequence exe­cu­tion. One is per­cep­tion. You need visu­al abil­i­ty to have visu­al pat­tern recog­ni­tion of the dig­its. Then you need to cre­ate S‑R map­ping, stim­u­lus response. So, you need to cre­ate the form, the asso­ci­a­tion between the dig­it and your motor response. Then you have mem­o­ry abil­i­ties. You can retrieve pre­vi­ous instances of the sequence. Then you have oth­er more gen­er­al algo­rithm abil­i­ties, such as plan­ning of the sequence. And, in the end, you need to have motor exe­cu­tion to exe­cute the sequence.

Will:
And we asked two ques­tions, two main questions.

Will:
One: is it pos­si­ble to con­duct rel­a­tive­ly com­plex motor sequence learn­ing tests online on patients with neu­ro­log­i­cal dis­or­ders? It was not clear because this is the first project as far as we know that has tried to do this kind of com­plex texts online on patients. And sec­ond was a more the­o­ret­i­cal ques­tion: what is the con­tri­bu­tion of the basal gan­glia and cere­bel­lum to mem­o­ry and algo­rithm process­es involved with sequence execution?

Will:
And the method is the following.

Will:
They viewed the dis­crete sequence pro­duc­tion test, the DSP test in whose par­tic­i­pants need to exe­cute four key but­tons in a spe­cif­ic order using two hands, so eight fingers.

Will:
And we had two con­di­tions. One is the rep­e­ti­tion con­di­tion in which par­tic­i­pants saw eight dif­fer­ent sequences that repeat 24 times. In this con­di­tion, we’ll assess mem­o­ry-based learn­ing. In the oth­er, the sec­ond con­di­tion is the no rep­e­ti­tion con­di­tion in which par­tic­i­pants saw 192 dif­fer­ent sequences. And this con­di­tion was just more gen­er­al algo­rithm based learn­ing abil­i­ties. And the main depen­dent vari­able is the exe­cu­tion time, or what peo­ple call ET. And this is the time from the first key press to the release of the last key press.

Will:
This is a slide from the real test, so you can see the instruc­tion. This is the way it looks for the par­tic­i­pants. So, they see the sequence they need to exe­cute and they got feed­back if it’s cor­rect or incor­rect exe­cu­tion of the sequence.

Will:
And in about one month from the moment we start­ed this exper­i­ment, we were able to col­lect data from 17 indi­vid­ual with spin­ocere­bel­lar atax­ia (SCA), 21 Parkin­son’s dis­ease and 20 con­trol participants.

Will:
This is the data. The y‑axis is the ET, exe­cu­tion time of the sequence. The x‑axis the cycle of learning.

Will:
And I think you can see four main phenomena.

Will:
One, you can see both patients with the Parkin­son dis­ease and the SCA in gen­er­al is slow­er than the con­trol par­tic­i­pants. And this is makes sense, right? Because they have motor impairments.

Will:
Anoth­er phe­nom­e­na, you can see a nice learn­ing curve. So, con­trol par­tic­i­pants are able to learn the sequences.

Will:
Third phe­nom­e­na, you can see more ben­e­fits from learn­ing in the repetition/blue con­di­tion com­pared to the no repetition/green con­di­tion. And this would make sense because in the rep­e­ti­tions, they have mem­o­ry benefits.

Will:
And you can also see already in this fig­ure that the atax­ia, the SCA group, have seri­ous learn­ing impair­ments. They are not real­ly able to learn this task.

Will:
I am men­tion­ing the obvi­ous here, because I think when some­one is doing a new approach in doing patient stud­ies online, it’s impor­tant to show a good san­i­ty test. So the data makes sense. And I think this is quite convincing.

Will:
After­wards, we want­ed to assess the learn­ing ben­e­fits. So, we cal­cu­lat­ed for each par­tic­i­pant the slope for each indi­vid­ual, for each con­di­tion. The y‑axis is the slope, x‑axis their groups. You can see first that the SCA group in green has less learn­ing ben­e­fits com­pared to the con­trol in blue. And the Parkin­son’s group in pink have less ben­e­fits only in the no rep­e­ti­tion con­di­tion selectively.

Will:
We also want­ed to exclude poten­tial con­founds. So, you can see that there is no exe­cu­tion time, ET, and RT, response time, for the first key press trade off. And there is no ET and accu­ra­cy trade off. But I think a nice fea­ture of this fig­ure is to see how much patient data we can col­lect using PONT.

Will:
So, just to sum­ma­rize these results, the lit­er­a­ture sug­gests strong evi­dence for impair­ments in the SCA group and the PD, the data show mixed results. And our added val­ue the­o­ret­i­cal­ly is that indeed, SCA showed the clas­sic impair­ment but also when we do the exper­i­ments online. And the PD impair­ment is selec­tive and depends on the algo­rithm need­ed, maybe some kind of prepa­ra­tion processes.

Will:
I start­ed with two main questions.

Will:
One, is it pos­si­ble to con­duct com­plex motor sequence learn­ing tests on patients online. And the answer is def­i­nite­ly yes. Sec­ond, I asked, what is the con­tri­bu­tion of the basal gan­glia (BG) and cere­bel­lum to mem­o­ry and algo­rithm process­es involved in sequence exe­cu­tion? And the answer is, PG is prob­a­bly is involved in motor algo­rithm, maybe plan­ning of sequences and cere­bel­lum is involved in motor mem­o­ry and algo­rithm process­es of sequence learning.

Will:
I would like to look to end, to fin­ish the pre­sen­ta­tion, with the advan­tages of PONT, this pro­to­col. And I think the key term here is acces­si­bil­i­ty. Using PONT, we have much more access to patient pop­u­la­tions. And I think it’s super impor­tant in the long run because it will allow first big­ger sam­ple size on patient pop­u­la­tion. Big­ger sam­ple size allows bet­ter rep­re­sen­ta­tion of these pop­u­la­tions. A lot of times, there’s bias in the literature.

Will:
Of course, it will allow also more stud­ies, as in all online stud­ies, which will allow bet­ter under­stand­ing of dis­eases such as Parkin­son’s, but also bet­ter under­stand­ing of the brain, such as the cerebellum.

Will:
And I think a nice fea­ture, at least from the feed­back that we are get­ting from the par­tic­i­pants, it’s quite engag­ing. It’s much more engag­ing for the par­tic­i­pants from in-per­son stud­ies. And in gen­er­al, they’re enjoy­ing doing the studies.

Will:
Thank you very much.

JP:
Awe­some, Will. We have a spe­cif­ic ques­tion for you. Cami­la was ask­ing, how long did it take for you to recruit your sam­ple? And along with that, maybe you can talk about the meth­ods you’re using to retain par­tic­i­pa­tion with these patients.

Will:
Sure.

Will:
Cami­la, first is, let’s say, now we are now one year and a half after I start­ed at my post­doc, after I start­ed this project. And now we have about 500 par­tic­i­pants. After 10 months, we had 420 par­tic­i­pants, 150 Parkin­son’s dis­ease, 150 SCA and 120 con­trol. So, this is about the time­line of how much time it takes to recruit the par­tic­i­pants using this approach.

Will:
And JP, your sec­ond ques­tion is… What is the sec­ond ques­tion? How to main­tain the par­tic­i­pants in our database?

JP:
Yeah. How do you retain par­tic­i­pa­tion with these patients or main­tain a rela­tion­ship with them?

Will:
Yeah.

Will:
First, we are doing the online neu­ropsy­cho­log­i­cal assess­ments. So, I have a team of RAs, of research assis­tants, that are doing online video ses­sions with the par­tic­i­pants. And I think the per­son­al rela­tions here are super impor­tant because once peo­ple feel more con­nect­ed to the study and more con­nect­ed to the lab, then they have more moti­va­tion to par­tic­i­pate and do the stud­ies. This is one way.

Will:
Anoth­er way, we are send­ing them out of time their results, if they’re inter­est­ed, their indi­vid­ual results. And this way we’re con­nect­ing them over time. There is a more con­tin­u­ous rela­tion­ship between us and the participants.

Will:
And I think in gen­er­al, the patients are super moti­vat­ed because they want to under­stand the dis­ease. They want to learn about it. So, they want to con­tribute to stud­ies that help to under­stand the under­ly­ing mech­a­nisms of Parkin­son or ataxia.

JP:
Great. And anoth­er ques­tion for you. Will, so there’s in-per­son assess­ments that require peo­ple to move so you can assess their move­ments and even maybe their reflex­es. How were you able to adapt these clin­i­cal tests online?

Will:
This is a rel­a­tive­ly big oper­a­tion. It’s a very com­pli­cat­ed operation.

Will:
To take, for exam­ple, we have done a med­ical ques­tion­naire, which is rel­a­tive­ly sim­ple. But we also have the [MOCA 00:15:06] test, which assess­es gen­er­al and mild cog­ni­tive impair­ment. And we have also a motor assess­ment for Parkin­son’s dis­ease [inaudi­ble 00:15:14] and atax­ia are doing a [SAGA 00:15:18] test.

Will:
In this test, we need­ed to go item by item and see if we can do it online using the video cam­era. One of the tests for exam­ple, is the fin­ger to nose test in which par­tic­i­pants to put their fin­ger and touch their nose. So, we need­ed to think how to do it. And in the end, for exam­ple, in the SAGA, we have done actu­al­ly all the items online. And the results are exact­ly the same as the results that peo­ple have done in per­son, the aver­age at least.

JP:
Awe­some.

 

Get on the Waitlist

BeOnline is the conference to learn all about online behavioral research. It's the ideal place to discover the challenges and benefits of online research and to learn from pioneers. If that sounds interesting to you, then click the button below and sign up for our newsletter. You will be the first to know when we release new content and open applications for BeOnline 2022.

With thanks to our sponsors!

PONT: A Pro­to­col for Online Neu­ropsy­cho­log­i­cal Testing