It was also possible for participants on whom the decoder had been trained to thwart the system, for example by thinking of animals or quietly imagining another story. The decoder was personalised and when the model was tested on another person the readout was unintelligible. “It doesn’t know if it’s first-person or third-person, male or female,” said Huth. Sometimes the decoder got the wrong end of the stick and it struggled with certain aspects of language, including pronouns. “For a non-invasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences,” Huth said. The participants were also asked to watch four short, silent videos while in the scanner, and the decoder was able to use their brain activity to accurately describe some of the content, the paper in Nature Neuroscience reported. Instead, I said: ‘Leave me alone!’” were decoded as “Started to scream and cry, and then she just said: ‘I told you to leave me alone.’” In another case, the words “I didn’t know whether to scream, cry or run away. “This is the reason why what we get out is not the exact words, it’s the gist.”įor instance, when a participant was played the words “I don’t have my driver’s licence yet”, the decoder translated them as “She has not even started to learn to drive yet”. “Our system works at the level of ideas, semantics, meaning,” said Huth. About half the time, the text closely – and sometimes precisely – matched the intended meanings of the original words. Later, the same participants were scanned listening to a new story or imagining telling a story and the decoder was used to generate text from brain activity alone. The decoder was trained to match brain activity to meaning using a large language model, GPT-1, a precursor to ChatGPT. The learning process was intensive: three volunteers were required to lie in a scanner for 16 hours each, listening to podcasts. These models are able to represent, in numbers, the semantic meaning of speech, allowing the scientists to look at which patterns of neuronal activity corresponded to strings of words with a particular meaning rather than attempting to read out activity word by word. However, the advent of large language models – the kind of AI underpinning OpenAI’s ChatGPT – provided a new way in. This hard limit has hampered the ability to interpret brain activity in response to natural speech because it gives a “mishmash of information” spread over a few seconds. “It’s this noisy, sluggish proxy for neural activity,” said Huth. The lag exists because fMRI scans measure the blood flow response to brain activity, which peaks and returns to baseline over about 10 seconds, meaning even the most powerful scanner cannot improve on this. The achievement overcomes a fundamental limitation of fMRI which is that while the technique can map brain activity to a specific location with incredibly high resolution, there is an inherent time lag, which makes tracking activity in real-time impossible. I’ve been working on this for 15 years … so it was shocking and exciting when it finally did work.” Dr Alexander Huth, a neuroscientist who led the work at the University of Texas at Austin, said: “We were kind of shocked that it works as well as it does.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |