Dr Saloni Krishnan — Royal Holloway, University of London
We constantly learn words from context, even in the absence of explicit rewards or feedback. In adults, intrinsic reward experienced during word learning is linked to a dopaminergic circuit in the brain. The experience of reward is also linked to enhancements in memory for those words. In this registered report, we examine if adolescents aged 10–18 report enhanced reward or enjoyment, and ensuing memory benefits, when they successfully learn words from sentence context. We have tested 300+ children so far. In addition to talking about developmental change in reward experienced during this period, I will focus on some of the advantages and challenges of conducting this research online.
Over to you Saloni
Hello, sorry, I just switched off my phone, which is why you probably saw me disappearing. So hopefully it’s not too noisy. And you can hear me nicely and clearly. So I have a terrible honking cold. It is, luckily not COVID. But I don’t recommend any summer cold. So if I kind of doze off in the middle, you’ll know why.
But I wanted to tell you about some work that I’ve been carrying out in my lab over the last two or three years. What are the I guess I would define myself as a developmental cognitive neuroscientist. And so in general, I tell people that I like to put kids in scanners and find out what’s different about their brains. Of course, over the last two years, because of a certain virus, it’s been incredibly difficult to do anything like that. So a lot of my research has kind of pivoted to be online. And I’ve been trying to use, particularly gorilla to kind of deliver and ask interesting new questions that didn’t necessarily involve putting children in scans.
So today, we’re talking about the role of reward in adolescent word learning. So I guess I want to start off by making a pitch given that this is a fairly diverse audience in terms of why vocabulary is important. What one reason is that we constantly encounter and learn new words through our lives. Brexit, you probably didn’t really know about Brexit, pre 2015, COVID definitely didn’t know about that pre 2020 kroner, we kind of encountered these new words all the time. And we also know that children with reading and language disorders can struggle with word learning, particularly acquiring youth from logical forms, that we don’t think like vocabulary knowledge are a predictor of school success, and eventually also life achievement.
So it seems that follow this process around kind of acquiring new words is really, really important. And lots and lots of researchers focus on kind of different aspects of this problem. So most theories focus on how words on them. So like, how do you assign a specific word to a specific object? But one of the things that I really care about and started to care about a lot in the last few years is why do we learn new words? Why do we inherently find this interesting, exciting? Why do we seek this out?
So I guess the question I’m going to be posing today is word learning intrinsically rewarding, because it’s not that someone comes and gives you five pounds, every time you encounter a new word, you learn a new word. So is there something that is an intrinsic driver from us, and if so, is word learning intrinsically rewarding for all groups. So I’m going to be talking about one particular paradigm today, which is learning words in context. This is not the only paradigm that I’ve used. But in the interest of time, I thought it might be better to kind of keep things kind of limited to one paradigm that I could explain well, so this was a really, really classic psycholinguistic paradigm. It’s been around since the 70s. And it’s been used loads and loads of time. And the basic premise would be that we can learn the meaning of new words by the context that we occur in.
So if I give you the sentence view, countries are now ruled by a side note site is not a real line of made up this mode. But the question is like, you might be able to kind of guess maybe what the meaning of this word is. And if you see the next sentence in the palace, the lips, the king and the stage, then you might have a read like a much better guess. And if you’re thinking cytosine something like a queen, then you will be right now lots of examples of how we do this sort of in the wild.
And again, lots of people have been really interested in sort of the diversity of the sentence contexts that use a number of encounters that you need, and so on. Well, one of the really nice twists in this kind of paradigm, I think, came from one of my collaborators at NYU, Pablo Ripolles, who’s now an assistant professor at NYU. What he showed is in those kinds of paradigms, when you learn what meaning from context, this is associated with intrinsic reward, and he demonstrated this in multiple ways.
So he showed this that in terms of people’s behavioural training, they tended to share say that learning these kinds of words was more pleasurable relative to when they couldn’t extract meaning of words. Brain activity and reward processing regions like the striatum was higher when when you tended to successfully learn the meanings of these words, unlike me, if you didn’t. Importantly, also, like you can also think about the effects of medication. So a couple of its really nice work looking at levodopa, which had kind of increased reward responses and Risperidone which is going to inhibit risk reward responses. And he was able to show that when you give people love dopa can boosted their learning when you give people Risperidone it,
kind of decrease or footstone Vitoria can decrease that learning Um, but the other thing that’s really, really important about this kind of experience of reward is that actually experiencing that reward seems to fuel word learning. So you get memory benefits from sort of encountering, and enjoying learning a new word. So you’re much more likely to remember that Obama had done a lot of this work in German. So all of his studies have been in German, and all of them had been with kind of neurotypical adults.
But in my work, I was really interested in trying to kind of open this up and potentially look at people, children, children with dyslexia, etc. And obviously, being in the UK, my studies are in English. So we did a sort of a couple of different studies to try and get a handle on coding and building on some of Butler’s work. So the first study I want to kind of give you a preamble onto is just to sort of say, what is the influence of modality on the sort of effects. And so he’ll probably done all of his work looking at people reading these kinds of sentences.
We also know from the literature that actually when people listen to the same kind of sentences, they also kind of extract words in the same way extract meaning from the same, right? So we hypothesise that learning new words would be intrinsically rewarding in both creating conditions as well as listening conditions. And we sort of said that, yes, intrinsic reward will also be associated with memory benefits.
Now, this was actually a project that undergrad students are going Holloway carry out in 2020 2021. So this is obviously something they have to do online. And it was pretty amazing for us that gorilla existed, that we could just go off and get this data because not a problem at all.
So I want to tell you a little bit about the paradigm. So as it as I’d hoped, demonstrate to you that you have these kinds of templates, sentences, so you sort of have sentences where you encounter another word at the end, and they’re kind of paired up so you can extract the meaningful word. Generally, after people encountered these words, they would have to type in a guess about what they thought the word was. And they would also have to give us a number of readings. So like, How confident were they? How tired were they can how much they enjoyed that particular sentence.
Um, interestingly, we also had a really good control, which is m minus sentences. So in these sentences, these are basically jumbled up sentences from another set of n plus sentences. But here, you cannot extract the meaning of a new word in this kind of really good experimental control device, you’d have things like John needed a battery for his bamboo, the teacher wrote the date on the bamboo. So it’s really kind of hard to extract the meaning for the word sample.
We presented these paradigms in in a Korean condition or listening condition, we also have a reading and listening condition, which is sort of akin to kind of thinking about subtitles. And this was a longitudinal study in a really, really simple way we’re learning study, because 24 hours later, people have to complete a recognition task on these words. It’s also not as easy as it looks. So with that, because what we actually did was we didn’t present these words sequentially. So you would tend to get a block of sentences. So you would encounter four different words. And then you would kind of encounter the pair for those words a little bit later. So it wasn’t as trivial as it looks like in my kind of schematic.
But yeah, the main thing is we tested native English speaking adults. And I can say more about this, but just to keep it brief. For the two of them this reading paradigm 36 Did the listening paradigm. And 34 did the reading and listening paradigm. And I think our goal was just to get to above 30 In all of these conditions. So we were fairly happy with these numbers. So what would it be fine?
Well, so I want you to look at the kind of M P sentences which are shown in blue over here, enjoyment is on the x axis and learning is on the y axis. And what you can very clearly see is that when people report greater enjoyment, they’re also showing Korean learning in the NP case. And this isn’t as much the case in the MM case, which is those kinds of control sentences that I talked about.
So in general, when you can successfully learn a new word, but not in kind of these experimental control controls, you do read things as being more pleasurable. I just wanted to overlay those same kind of curves, one on top of the other. So this is only for the MP condition where you can extract the meaning of the word. And I just wanted to show you how similar this was across the three conditions. So listening, reading, reading and listening.
One of the other things I wanted to show you is that on that recognition task again, so this is memory rather than learning on day one, and again, the more you enjoyed learning that word and deed one, the better you are at remembering it. And this was recorded to the conditions of listening, reading or reading or listening. My very simple kind of takeaway from that work is just to say that actually, intrinsic reward is associated with memory benefits. So we kind of replicated what Pablo had already shown but kind of expanded it into these new modality conditions. So this we felt set us up really well.
And in experiment two, we decided to test children. And we decided specifically that we were going to investigate change with development, the age group that we wanted to focus on with later lessons, and this is because of some work from Lisa Knoll, which had suggested that when you train older adolescents on kind of relational reasoning and numerosity discrimination, older adolescents show the kind of greatest jump in performance, and adults and younger adolescents to children learning, but they weren’t as pronounced as the game.
So Lisa Knoll kind of suggested late adolescence might offer a window of opportunity for educational interventions. There’s also some work from the kind of neuroscience domain, which is kind of suggesting that straight line activity is predictive of later learning performance. And that’s the general idea that increased reward sensitivity would lead to increase motivation and salience. And then this recruits systems for learning. So actually, this would be a really important window earlier, our lessons might offer a really important window for wording as well, and so this is what we decided to test in this particular registered report, where we test on 10 to 18 year olds with the idea of basically sorry, and this is sorry, this I should say, this is work conducted with RAs in my lab at the time and with our Amrita Bains, Analise Barber and Tam Nell. And the hypothesis we were looking at was that learning new words were reading will be intrinsically rewarding and early developmental stage.
When we did this work, this hypothesis hadn’t been tested in children and adolescents. But the kind of premise of the fact that word learning would be recording, we would very much expect to see this early in development if truly reward is having meaningful interaction to language.
We also hypothesised that intrinsic reward would be related to memory benefits and potential new and appears experiment. But if only we had a developmental hypothesis that the kind of reward you experience and the kind of learning from reward would increase in age and all that sort of peak in late adolescence, which we thought would be somewhere between 14 to 16, or 16, to 18 years. So this is a registered report.
So as you know, we kind of pre registered the plans, and we got reviewers, and this has been accepted in developmental science. And for this particular project and reporting data from 345, native English speaking children, aged 10 to 18. All of this data was collected during the pandemic. And basically, our goal to get above 84 Children are based on our pre registration and all of these brackets that we are ready for 10 to 12 year olds, 14 year olds, 1416 and 16 to 18.
I want to show you a couple of kind of emerging results, I still finished writing this up and submit the stage to submission. So this sort of work in progress, but some very, very clear messages right so far from the data.
The first is like we saw with the behaviours that I just presented to you learning new words as reporting. So we here we only did the reading condition. But just like before, you can see that particularly in the kind of m plus situation when you kind of enjoy when you report greater enjoyment, you tend to get greater learning. And you see some indication of this even an M minus condition in children aged 10 to 18.
What was the most surprising though, and what people weren’t expecting to find is that there were no, there was no effect of age. And so in some ways, hypothesis one, which is the fact that we would replicate this data in children is kind of surprising, because if you had any experience of registered reports, what tends to happen is that you try to replicate a finding, and it doesn’t.
Actually, we did replicate the first finding, but we didn’t find any evidence about kind of second hypothesis. So it wasn’t that 14 to 16 year olds or 16 to 18 year olds showed greater reward greater sort of learning due to reward. Or kind of the registered hypothesis, also, we don’t see really good evidence for so we don’t really observe memory benefits. And I think this is partially for two reasons.
So basically, what we would have expected to find is that enjoyment was predictive of kind of accuracy on the second day of learning. But one of the problems may have been that we made the task really easy. So for children, what we did is we didn’t have the complicated blocking structure that I showed you in the previous experiment, we did give them sequential sentences, because we thought it would be too much for children. And in doing so what we may have done is actually the learning demands of the task too easy and not being able to observe these kinds of memory benefits. That’s one idea potentially.
So I want to kind of summarise what I’ve told you so far, I’m in terms of the science so children and adults do find word learning intrinsically. And we want to suggest that when tasks are challenging enough that we find that reward is associated with green and we’re doing some future work on this. So we’re now looking at the neural basis of reward for word learning in your typical children and children with dyslexia. And we’re also starting to understanded the processes of motivation going beyond the experience of reward alone. And this is work being carried out on wholly by my postdoc Desi.
But as you alluded to, I wanted to talk a little bit about what is being online done for us. I wanted to mention some pros and some cons and perhaps give you some very quick tips and tricks. So in terms of the pros, the reach of experiment, like I mentioned that we tested something like 345 data, children, for I experiment, there is no way I would have been able to do that, based on kind of strict lab settings, it just could not have happened, I believe, to the fact that this was possible in a pandemic multiple times, there was again, no way that we could have gone into schools and got data in any other way.
Designing these tests was really, really easy. So that ease of design and kind of getting undergraduate project students in more than doing this and so on was really important. sharing and collaboration was easier. My kind of collaborator was at NYU, but it was really easy to kind of just reach out to him and say, here’s the power, and you can look at it, you can play with it, give me your comments. And of course, gorillas can open materials makes open science and sharing an optically recollection also much easier.
I think there are some cons, we while conducting this experiment online, we realised that we were at some point getting lots of bots. And that was problematic, or you know, people who were just kind of picking up our ads online and doing stuff, we didn’t have a check for age in any way. So obviously, you’re testing a child in real life, you can see that they’re a child, one of the things that worried us when maybe people are kind of participating, and there were adults, but they were just pretending to be children to get vouchers from US government. Obviously, one of the things that you get when you are watching someone do your task is a lot of insight into participant behaviour, we didn’t quite have that.
And that we did have a tech problem and one of the experiments and one of the particular tasks that we were using needed a keyboard response. But apparently lots of children nowadays use tablets to do everything. So they would come to this task and they would fail. And then basically, we needed to set up multiple versions, including a whole new day to experiment just in case they had failed this one particular childcare cutouts.
And of course, there is a bit more cost to running these things online then. So for example, if you run things in schools, you can just give people stickers. If it’s online, you need to tempt people with vouchers.
Tips and tricks, we suggest lots and lots of experimental controls we had in our experiment, accuracy checks, attention checks, we kind of as I alluded to run a new version of just the day 2 experiment. And I think the main thing that I would say is be responsive to participants and ideally, run studies in small batches so that you can touch up email with your participants and talk to them. That’s all Sorry for going over time, Jo. That’s me done thank you.
That was fabulous. Lonely.
Sorry. I was trying to unmute myself and failing to click the button. That was absolutely brilliant. I am so impressed with the work that you’ve done. I’m just going to have to change my view here because I can’t bear looking at myself when I’m speaking.
Gosh, that was a lot of kids that you’ve recruited. Those are big samples. How did you get them?
Oh, well, that was actually recruited even more than I presented here. So I think overall, we probably tested something like 540 kids, and then actually because of the kind of experimental controls that you have you drop people out. So lots and lots of kids, I guess, schools like we send a lot of emails to schools to be like, Please, can you put this up? And then lots schools are very happy to be like, you know, asking us to devote any time. People can just go do it online on this particular link. That was great. What else did we do? Lots and lots of emails, lots and lots of social media kind of pressure and people. I mean, I think part of it is like when you do a registered report you’ve committed to doing the numbers just
have to get out and get them. Yes, we’re getting the last five harder than getting the first five.
Actually. So we realise at some point that like the, for some reason, we got lots of 16 to 18 euros, we got lots of 10 euros, but the kind of in between group that 12 to 14 euros, we had this little graph where we wanted to see the numbers going up and for some reason we just never got 12 to 14 year olds. Getting those last few 12 to 14 euros was the most challenging bit. But yeah, it works out really beautifully. And
so I think there are some questions coming in the q&a. And if you’re in the audience, and you’ve got a question, please do dump it in the q&a. Now, one more question from me. What? What types of checks? Did you put into, like sanity checks on your data did you put into to to make sure you’re getting sensible data quality?
Yeah. So with Luckily, this was a registered report. So like, a review is also recommended, and your sanity checks. There are kind of one sanity check was part of our registered report. And we basically said that if we’re really seeing children, and this probably also answers, some of Matt Davis’s questions in the q&a, is that we would expect the kind of extracting meaning would be increasing with age. And we find a very, very strong relationship that so basically, accuracy in the m plus condition very strongly goes up by age. So for when you’re 10, you’re a little bit worse at answering those questions. When you’re 18, you’re much better answering those questions. And that is definitely what we were expected. So that that’s kind of helpful.
And at some point, we hit this vein of people on Facebook, who were like, emailing us. And you know, seeing those emails is really reassuring, because there’s a certain way a parent communicates. And you can definitely go yes, that definitely feels like an interaction with a real client. There are some where we were just not sure. And then we have kind of a lot of them, luckily, fair kind of retention checks. So we had stuff like we needed to have kids complete with a certain level of accuracy. We also have embedded attention checks in there. We’ve also got under control. So we’ve asked people about best sleep. And we’ve asked people about kind of we did this task Jason Yeatman group called roar. So we have raw data. And again, on the raw, we can see, which is a lexical decision task for those of you that means you can again, see the accuracy is increasing with age and reaction times kind of decreasing, really, so kind of a good set of checks in that.
Very, very cool. Yes, that sounds like a comprehensive set of sanity checks. do you what do you want to give a more comprehensive estimate? Or have you answered that sufficiently? The question was, do you think it’s possible that the null effects of age can be explained by dishonest age reports? I think you’ve just about covered that. So I think, yeah, I
don’t think it would be dishonest. I mean, I’m sure that like every single person in the data we can’t necessarily verify. And one thing we should have really done is kind of possibly enforce a mechanism that people could only do the task ones because I was just like, oh, person with the same email, who’s done my task can be another little bit of kind of sanity check kind of weeding out stuff, but I’m sure if like someone was smarter and use like a second email, that’s a bit harder to claim
this thing. Did you consider trying to get like, and use the video recording zone just to get a screenshot of the person? Or would you not have got that past ethics?
I think ethics would have been challenging. I think a few people have been talking about like getting audio recordings during consent, because obviously, your voice is a really good cue. If your agent if you say something like I can send an obviously, you know that, you know, it’s a native English speaker who gives you honestly, like, you know, I mean, I know you don’t necessarily face some of these issues, but I would be inherently extremely surprised. And I’m sure we have a couple of these sort of things in the main data set. But I’d be very surprised if that was the entire explanation.
Brilliant. So only could you just go back to your side of top tips. And while certainly he’s doing that, Neil, could you get your slides ready for sharing in a minute? And what I just wanted to ask the audience that we’ve got here today, which of the tips and tricks was most useful for you today? What What was the lesson you most needed to hear? Was it about learning lots of experimental controls and separating accuracy checks and attention sets? Or was it about needing to be responsive when things go wrong? And maybe creating a different version of experiment? That was something you might not consider? Or was it the being responsive to participants, or in these the advice on recruiting children? Brilliant, thank you so much. It’s just wonderful. It’s really great if you are willing to be generous to Saloni and give the feedback that’s been most helpful to you, because it helps Saloni know, she’s done a good job. And it’s difficult giving a talk to a room of completely blank screens. And that interaction from you guys is just really rewarding to know that we are helping you learn, learn messages that are helpful that you’re going to take into your research and will make your lives better. Now with that Saloni thank you so much. You were wonderful. Thank you for starting us and setting the note the bar so high.