PONT: A Pro­to­col for Online Neu­ropsy­cho­log­i­cal Testing

Will:
Let me move this. Okay.

Will:
My name is Will. I’m a post­doc at Berke­ley, and I will talk today about PONT, a pro­to­col for online neu­ropsy­cho­log­i­cal testing.

Will:
A lit­tle bit of an intro­duc­tion. The lit­er­a­ture is focused main­ly on the cor­tex when we look con­di­tion domains of stud­ies from atten­tion to arith­metic to motor sequence learn­ing. While the con­tri­bu­tion of sub-cor­ti­cal brain mech­a­nisms to [inaudi­ble 00:00:29] remain unclear, we see from Stan­ford the “cor­tic­o­cen­tric” neu­ro­science bias. Sub­cor­ti­cal struc­tures such as the basal gan­glia, the cere­bel­lum, the thal­a­mus and more might also be involved in cognition.

Will:
And one solu­tion to this prob­lem is to do more stud­ies involv­ing patients with sub­cor­ti­cal pathology.

Will:
Cur­rent­ly, I’m focus­ing on two sub­cor­ti­cal regions. One is the cere­bel­lum, using patients with spin­ocere­bel­lar atax­ia, SCA, and the oth­er sub-cor­ti­cal region is the basal gan­glia, BG, using patients with Parkin­son’s disease.

Will:
But we have anoth­er prob­lem with this solu­tion. We are deal­ing with a very lim­it­ed resource. We are deal­ing with the patients themselves.

Will:
First, it’s very dif­fi­cult to iden­ti­fy and recruit these indi­vid­u­als for projects. But let’s say you suc­ceed to recruit them into your exper­i­ment. Their abil­i­ty to par­tic­i­pate in a research project is very restrict­ed by their med­ical con­di­tions, for exam­ple, their motor impair­ments. And as a result, if you will do a lit­er­a­ture search, you will find papers with very small sam­ple sizes, such as 6, 7, 8 par­tic­i­pants. But not only that, it takes quite a bit of time to com­plete a sin­gle study, let alone a pack­age of stud­ies that might allow a more com­pre­hen­sive sto­ry with the appro­pri­ate con­trol participants.

Will:
Just the indi­ca­tion to show you how much the prob­lem is seri­ous. If you will do a PubMed search result and you will enter two key terms in PubMed, one is neu­ropsy­cho­log­i­cal study, the oth­er is fMRI stud­ies, this is what you will get. So, Y axis is the count, the num­ber of results. The X axis, the num­ber of the year.

Will:
And you can see how many more results we get for fMRI com­pared to neu­ropsy­cho­log­i­cal study. And I think the trend is also very clear.

Will:
To address is prob­lems, in my first year in my post-doc, I’ve devel­oped this pro­to­col, a pro­to­col for online neu­ropsy­cho­log­i­cal test­ing. I like to call it PONT. Pont is a bridge in France, by the way.

Will:
And as a first step, I recruit­ed a team of research assis­tants that helped me to build protocol.

Will:
And the PONT pro­to­col involves five main steps.

Will:
Step num­ber one, we reach out to sup­port groups from all across the Unit­ed States, from LA to New York. And we wrote an email to the group man­agers ask­ing to trans­fer the infor­ma­tion to the group mem­bers about the project. And step num­ber two, each one of the group mem­bers reach out to us say­ing that they want to par­tic­i­pate in our project. In step num­ber three, each indi­vid­ual is doing an online neu­ropsy­cho­log­i­cal assess­ment in which we assess med­ical con­di­tions assess­ment, we assess motor abil­i­ties, cog­ni­tive abil­i­ties. It’s about a one hour ses­sion. And step num­ber four, each indi­vid­ual is get­ting an email with a spe­cif­ic link and a spe­cif­ic ID. And they can do the exper­i­ment when­ev­er they want. When they com­plete the exper­i­ment, we asked for some feed­back and we paid them for the participation.

Will:
These are the five main steps of PONT.

Will:
In the first 10 months of the project, we were able to recruit about 420 par­tic­i­pants for our data­base, 150 with Parkin­son’s dis­ease (PD), 150 SCA, spin­ocere­bel­lar atax­ia, and 120 con­trol participants.

Will:
And the goal of the project was to do dif­fer­ent exper­i­ments to assess both cog­ni­tive abil­i­ties and motor abil­i­ties. In using PONT, the idea is to assess dif­fer­ent neur­al mech­a­nisms in a wide range of abilities.

Will:
And now we’ll focus on the motor project.

Will:
So, this is a con­crete exam­ple of a DSP task, a dis­crete seg­ments pro­duc­tion test, in which par­tic­i­pants are asked to put their fin­gers on the key­board, such that each fin­ger is on the appro­pri­ate but­ton as fol­low­ing. And then they need to exe­cute the sequence as fast as possible.

Will:
A lit­tle bit of back­ground about sequence learn­ing. Most research has focused on the effect of mem­o­ry on sequence exe­cu­tion, basi­cal­ly by com­par­ing between repeat­ing sequences to ran­dom sequences. And there is a lot of stud­ies. Large­ly, they touch on the involve­ment of basal gan­glia (BG) and cere­bel­lum on sequence exe­cu­tion, though there is one caveat here as two stud­ies by [Wat­ten­berg 00:05:15] recent­ly have sug­gest­ed that the activ­i­ty of the Parkin­son’s dis­ease lit­er­a­ture is equiv­o­cal, mean­ing peo­ple saw mixed results. Some­times peo­ple with Parkin­son’s were unim­paired and some­times not.

Will:
And as I see it, there is five main cycles involved in sequence exe­cu­tion. One is per­cep­tion. You need visu­al abil­i­ty to have visu­al pat­tern recog­ni­tion of the dig­its. Then you need to cre­ate S‑R map­ping, stim­u­lus response. So, you need to cre­ate the form, the asso­ci­a­tion between the dig­it and your motor response. Then you have mem­o­ry abil­i­ties. You can retrieve pre­vi­ous instances of the sequence. Then you have oth­er more gen­er­al algo­rithm abil­i­ties, such as plan­ning of the sequence. And, in the end, you need to have motor exe­cu­tion to exe­cute the sequence.

Will:
And we asked two ques­tions, two main questions.

Will:
One: is it pos­si­ble to con­duct rel­a­tive­ly com­plex motor sequence learn­ing tests online on patients with neu­ro­log­i­cal dis­or­ders? It was not clear because this is the first project as far as we know that has tried to do this kind of com­plex texts online on patients. And sec­ond was a more the­o­ret­i­cal ques­tion: what is the con­tri­bu­tion of the basal gan­glia and cere­bel­lum to mem­o­ry and algo­rithm process­es involved with sequence execution?

Will:
And the method is the following.

Will:
They viewed the dis­crete sequence pro­duc­tion test, the DSP test in whose par­tic­i­pants need to exe­cute four key but­tons in a spe­cif­ic order using two hands, so eight fingers.

Will:
And we had two con­di­tions. One is the rep­e­ti­tion con­di­tion in which par­tic­i­pants saw eight dif­fer­ent sequences that repeat 24 times. In this con­di­tion, we’ll assess mem­o­ry-based learn­ing. In the oth­er, the sec­ond con­di­tion is the no rep­e­ti­tion con­di­tion in which par­tic­i­pants saw 192 dif­fer­ent sequences. And this con­di­tion was just more gen­er­al algo­rithm based learn­ing abil­i­ties. And the main depen­dent vari­able is the exe­cu­tion time, or what peo­ple call ET. And this is the time from the first key press to the release of the last key press.

Will:
This is a slide from the real test, so you can see the instruc­tion. This is the way it looks for the par­tic­i­pants. So, they see the sequence they need to exe­cute and they got feed­back if it’s cor­rect or incor­rect exe­cu­tion of the sequence.

Will:
And in about one month from the moment we start­ed this exper­i­ment, we were able to col­lect data from 17 indi­vid­ual with spin­ocere­bel­lar atax­ia (SCA), 21 Parkin­son’s dis­ease and 20 con­trol participants.

Will:
This is the data. The y‑axis is the ET, exe­cu­tion time of the sequence. The x‑axis the cycle of learning.

Will:
And I think you can see four main phenomena.

Will:
One, you can see both patients with the Parkin­son dis­ease and the SCA in gen­er­al is slow­er than the con­trol par­tic­i­pants. And this is makes sense, right? Because they have motor impairments.

Will:
Anoth­er phe­nom­e­na, you can see a nice learn­ing curve. So, con­trol par­tic­i­pants are able to learn the sequences.

Will:
Third phe­nom­e­na, you can see more ben­e­fits from learn­ing in the repetition/blue con­di­tion com­pared to the no repetition/green con­di­tion. And this would make sense because in the rep­e­ti­tions, they have mem­o­ry benefits.

Will:
And you can also see already in this fig­ure that the atax­ia, the SCA group, have seri­ous learn­ing impair­ments. They are not real­ly able to learn this task.

Will:
I am men­tion­ing the obvi­ous here, because I think when some­one is doing a new approach in doing patient stud­ies online, it’s impor­tant to show a good san­i­ty test. So the data makes sense. And I think this is quite convincing.

Will:
After­wards, we want­ed to assess the learn­ing ben­e­fits. So, we cal­cu­lat­ed for each par­tic­i­pant the slope for each indi­vid­ual, for each con­di­tion. The y‑axis is the slope, x‑axis their groups. You can see first that the SCA group in green has less learn­ing ben­e­fits com­pared to the con­trol in blue. And the Parkin­son’s group in pink have less ben­e­fits only in the no rep­e­ti­tion con­di­tion selectively.

Will:
We also want­ed to exclude poten­tial con­founds. So, you can see that there is no exe­cu­tion time, ET, and RT, response time, for the first key press trade off. And there is no ET and accu­ra­cy trade off. But I think a nice fea­ture of this fig­ure is to see how much patient data we can col­lect using PONT.

Will:
So, just to sum­ma­rize these results, the lit­er­a­ture sug­gests strong evi­dence for impair­ments in the SCA group and the PD, the data show mixed results. And our added val­ue the­o­ret­i­cal­ly is that indeed, SCA showed the clas­sic impair­ment but also when we do the exper­i­ments online. And the PD impair­ment is selec­tive and depends on the algo­rithm need­ed, maybe some kind of prepa­ra­tion processes.

Will:
I start­ed with two main questions.

Will:
One, is it pos­si­ble to con­duct com­plex motor sequence learn­ing tests on patients online. And the answer is def­i­nite­ly yes. Sec­ond, I asked, what is the con­tri­bu­tion of the basal gan­glia (BG) and cere­bel­lum to mem­o­ry and algo­rithm process­es involved in sequence exe­cu­tion? And the answer is, PG is prob­a­bly is involved in motor algo­rithm, maybe plan­ning of sequences and cere­bel­lum is involved in motor mem­o­ry and algo­rithm process­es of sequence learning.

Will:
I would like to look to end, to fin­ish the pre­sen­ta­tion, with the advan­tages of PONT, this pro­to­col. And I think the key term here is acces­si­bil­i­ty. Using PONT, we have much more access to patient pop­u­la­tions. And I think it’s super impor­tant in the long run because it will allow first big­ger sam­ple size on patient pop­u­la­tion. Big­ger sam­ple size allows bet­ter rep­re­sen­ta­tion of these pop­u­la­tions. A lot of times, there’s bias in the literature.

Will:
Of course, it will allow also more stud­ies, as in all online stud­ies, which will allow bet­ter under­stand­ing of dis­eases such as Parkin­son’s, but also bet­ter under­stand­ing of the brain, such as the cerebellum.

Will:
And I think a nice fea­ture, at least from the feed­back that we are get­ting from the par­tic­i­pants, it’s quite engag­ing. It’s much more engag­ing for the par­tic­i­pants from in-per­son stud­ies. And in gen­er­al, they’re enjoy­ing doing the studies.

Will:
Thank you very much.

JP:
Awe­some, Will. We have a spe­cif­ic ques­tion for you. Cami­la was ask­ing, how long did it take for you to recruit your sam­ple? And along with that, maybe you can talk about the meth­ods you’re using to retain par­tic­i­pa­tion with these patients.

Will:
Sure.

Will:
Cami­la, first is, let’s say, now we are now one year and a half after I start­ed at my post­doc, after I start­ed this project. And now we have about 500 par­tic­i­pants. After 10 months, we had 420 par­tic­i­pants, 150 Parkin­son’s dis­ease, 150 SCA and 120 con­trol. So, this is about the time­line of how much time it takes to recruit the par­tic­i­pants using this approach.

Will:
And JP, your sec­ond ques­tion is… What is the sec­ond ques­tion? How to main­tain the par­tic­i­pants in our database?

JP:
Yeah. How do you retain par­tic­i­pa­tion with these patients or main­tain a rela­tion­ship with them?

Will:
Yeah.

Will:
First, we are doing the online neu­ropsy­cho­log­i­cal assess­ments. So, I have a team of RAs, of research assis­tants, that are doing online video ses­sions with the par­tic­i­pants. And I think the per­son­al rela­tions here are super impor­tant because once peo­ple feel more con­nect­ed to the study and more con­nect­ed to the lab, then they have more moti­va­tion to par­tic­i­pate and do the stud­ies. This is one way.

Will:
Anoth­er way, we are send­ing them out of time their results, if they’re inter­est­ed, their indi­vid­ual results. And this way we’re con­nect­ing them over time. There is a more con­tin­u­ous rela­tion­ship between us and the participants.

Will:
And I think in gen­er­al, the patients are super moti­vat­ed because they want to under­stand the dis­ease. They want to learn about it. So, they want to con­tribute to stud­ies that help to under­stand the under­ly­ing mech­a­nisms of Parkin­son or ataxia.

JP:
Great. And anoth­er ques­tion for you. Will, so there’s in-per­son assess­ments that require peo­ple to move so you can assess their move­ments and even maybe their reflex­es. How were you able to adapt these clin­i­cal tests online?

Will:
This is a rel­a­tive­ly big oper­a­tion. It’s a very com­pli­cat­ed operation.

Will:
To take, for exam­ple, we have done a med­ical ques­tion­naire, which is rel­a­tive­ly sim­ple. But we also have the [MOCA 00:15:06] test, which assess­es gen­er­al and mild cog­ni­tive impair­ment. And we have also a motor assess­ment for Parkin­son’s dis­ease [inaudi­ble 00:15:14] and atax­ia are doing a [SAGA 00:15:18] test.

Will:
In this test, we need­ed to go item by item and see if we can do it online using the video cam­era. One of the tests for exam­ple, is the fin­ger to nose test in which par­tic­i­pants to put their fin­ger and touch their nose. So, we need­ed to think how to do it. And in the end, for exam­ple, in the SAGA, we have done actu­al­ly all the items online. And the results are exact­ly the same as the results that peo­ple have done in per­son, the aver­age at least.

JP:
Awe­some.