Christina Y. Tzeng, San José State University
By loading the video, you agree to YouTube’s privacy policy.
Learn more
Full Transcript:
Christina Y. Tzeng:
All right. Thank you, Rachel, for the introduction at the beginning of the session and to both you and Joshua for gathering us in this space. I also want to say thank you upfront for everyone who is still here at the last talk of the day. I am excited to share about my experience conducting speech perception experiments online, highlighting the power for these online experiments to democratize science and teaching. I’ll aim to achieve two objectives in my talk today. The first is to share some findings that add to what we now know is a growing piece of evidence that online speech perception experiments are highly efficient and do yield robust data. The second objective is to share some thoughts on how online experiments, more broadly, can make science more accessible for both researchers and participants.
Christina Y. Tzeng:
In my work, I study how we, as listeners, overcome the enormous amount of variation that we encounter when we listen to different voices and utterances. Our experiments typically require participants to listen to auditory stimuli and make subsequent responses on a computer to each one. For in-person or in-lab experiments, this would typically require what’s pictured on the left: a sound-attenuated, distraction-free booth, high-quality headphones, and specialized software, as well as hardware.
Christina Y. Tzeng:
As a disclaimer, I have to state that my first real dive into the world of online experiments was in late 2019, which makes me a relatively novel user of these online experimental methods, but this is when I started to wonder, “Is such a highly controlled listening environment really necessary?”
Christina Y. Tzeng:
In the interest of achieving this first objective, I’d like to share what are now published findings from my first foray into the online experiment world. This is work done in collaboration with my colleagues, Dr. Lynne Nygaard and Dr. Rachel Theodore, where we examined the time course of a phenomenon called lexically guided perceptual learning.
Christina Y. Tzeng:
We know that listeners use a whole host of cues to map the acoustics of the speech signal onto linguistic units. One of these cues is lexical knowledge. Imagine hearing a fricative sound that’s between an S and an SH sound. If that ambiguous sound is embedded into this word on the left, the listener hears that sound as an S as in dinosaur. But if that same ambiguous sound is instead embedded in the word on the right, the listener hears that sound instead as an SH as in efficient. But if listeners are exposed to these ambiguous sounds in stable lexical contexts, that bias them to hear either S or SH sound.
Christina Y. Tzeng:
What we then see are changes in the listener’s representations of their S and SH category. These changes in sound category representation are what we call lexically guided perceptual learning. In both the online and in-person versions of this task, the lexically guided perceptual learning paradigm takes about 20 minutes to complete. So here, listeners complete an exposure phase followed by a test phase. And in the exposure phase, they complete a lexical decision task where they hear an ambiguous sound such as a fricative between S and SH. One group hears this ambiguous sound that’s embedded in words, biasing them to hear it as an S, whereas another group is biased to hear that same sound as an SH. So after exposure, the listeners complete a phonetic categorization task where they identify ambiguous sounds on a non-word continuum here, either as asi or ashi.
Christina Y. Tzeng:
We drew our samples from Prolific and executed the experiments in Gorilla. We completed a total of six experiments in this publication, but in the interest of time, I’ll share the findings from one. What will appear here are the results of the phonetic categorization task at test, whereupon hearing ambiguous sound on the asi/ashi continuum, we measured the likelihood that participants heard those sounds as asi. Here, we see robust evidence for lexically guided perceptual learning. As listeners, we’re more likely to hear the ambiguous sounds as asi when they were biased to hear S during exposure indicated by the red line, then when they were biased to hear the sounds as SH during exposure shown here by the green line.
Christina Y. Tzeng:
To showcase the high level of data quality that we see at the individual level, here are separate plots for each of the 70 participants at test where we can see the expected psychometric curves for every single participant. We only excluded 5% of our participants across the six experiments due to failure to perform the task. We did have to exclude 16% of the total number of participants due to failure to pass the woods at all, headphone check that Dr. Theodore described at the beginning of the session. But this was a small price to pay, given the speed of data collection. So for example, we collected data from the 70 participants presented in Experiment 1 in under a single hour.
Christina Y. Tzeng:
I hope what I’ve shared has supported the idea that online speech perception experiments are highly efficient and yields robust findings even with auditory tasks that require fine-grained phonetic discriminations like the one I presented.
Christina Y. Tzeng:
I now want to turn to the idea that online experiments can provide us with two things in particular: access to a larger and more diverse pool of participants and also more user-friendly experiment building interfaces for our students and research mentees.
Christina Y. Tzeng:
This is the figure I showed earlier. We replicated the finding with another end of 70 participants using a second stimulus set shown here on the right, meaning we ran a total of 150 participants within the span of about an hour and a half, which using in-person methods would have taken us weeks or even months.
Christina Y. Tzeng:
For his master’s thesis, one of my student collaborators, Ulises Quintero, is interested in recruiting participants who speak English and a second language. So in Prolific, if we use our standard inclusion criteria, including this criterion of speaking English plus another language, we automatically have access to over 3,000 participants, which is magnitudes greater than what we would have access to using in-person methods. For his undergraduate honors thesis, Justin Au built a talker ID task in Gorilla on his own using primarily the tutorial support that is on Gorilla’s website as a guide.
Christina Y. Tzeng:
And by addressing the two questions about auditory research more broadly that Rachel shared at the beginning of the session, the first is, “What do you think is the biggest challenge for auditory research online, and how do you overcome it?” As Jason mentioned, due to the pandemic, we have all been forced to some extent to embrace online methods more readily, but I think we are still very much in the process of establishing both the validity and the reliability of these methods. And one way for us to do this is to run online and in-person experiments in parallel so that we, not just as individual researchers but as a field, can be reassured that our tasks can be successfully transferred across these different platforms.
Christina Y. Tzeng:
And the second question, “What can auditory research gain most from online methods?” My take on this is that, with how quickly, we can collect data from a whole number of different populations. We’ve essentially eliminated the data collection bottleneck. Adapting in-person experiments to the online world takes a lot of trial and error, and I’m still very much in that learning phase, but I think that the reduction of this bottleneck drastically changes the pace of auditory research and science more broadly.
Christina Y. Tzeng:
With that, I’d like to extend my gratitude to my recent collaborators as well as to all of you for your attention. I look forward to your questions and comments.
Rachel Theodore:
Excellent, Christina. Thank you so much for those really careful thoughts. Questions. Yeah, here’s one for you, Christina. “I was wondering if, in your work, you’ve observed the use of different exposure phase methods besides lexical decision in an online world. How’s the story listening closed sentences and if you’ve noticed any differences at test as a function of those exposure phase methods?”
Christina Y. Tzeng:
Thanks for that question. So again, coming back to this disclaimer that I’m a relatively novel user of online methodology in general for auditory research, we’ve only done some pilot work using other kinds of exposure methodology. We’re in the process of piloting a talker identification task, where during the exposure phase, listeners well hear utterances spoken by specific talkers and have to indicate which talker they think they’re hearing with the ultimate goal being able to identify the different voices in the task. And so far, we haven’t seen any kind of noticeable difference in performance for in-person/in-lab versions of that and online versions. What we do notice is that, sometimes, participants will take self-inflicted breaks. And so one lesson we’ve learned is that in addition to keeping the task relatively short, we will build in some breaks so that they’re not leaving the computer for an extended period of time. But the short response to that question is at least with talker identification tasks and this lexically guided perceptual learning task, we haven’t seen reasons to not transfer these into the online world.
