A computer model has been developed that can predict what word you are thinking of. (41) Researchers led by Tom Mitchell of Carnegie Mellon University in Pittsburgh, Pennsylvania, "trained" a computer model to recognize the patterns of brain activity associated with 60 images, each of which represented a different noun, such as "celery" or "aeroplane".
(42) . Words such as "hammer", for example, axe known to cause movement-related areas of the brain to light up; on the other hand, the word "castle" triggers activity in regions that process spatial information. Mitchell and his colleagues also knew that different nouns are associated more often with some verbs than with others--the verb "eat", for example, is more likely to be found in conjunction with "celery" than with "aeroplane". The researchers designed the model to try and use these semantic links to work out how the brain would react to particular nouns. They fed 25 such verbs into the model.
(43) . The researchers then fed the model 58 of the 60 nouns to train it. For each noun, the model sorted through a trillion-word body of text to find how it was related to the 25 verbs, and how that related to the activation pattern. After training, the models were put to the test. Their task was to predict the pattern of activity for the two missing words from the group of 60, and then to deduce which word was which. On average, the models came up with the right answer more than three-quarters of the time.
The team then went one step further, this time training the models on 59 of the 60 test words, and then showing them a new brain activity pattern and offering them a choice of 1 001 words to match it. The models performed well above chance when they were made to rank the 1001 words according to how well they matched the pattern. The idea is similar to another "brain-reading" technique. (44) . It shouldn’t be too difficult to get the model to choose accurately between a larger number of words, says John-Dylan Haynes.
An average English speaker knows 50 000 words, Mitchell says, so the model could in theory be used to select any word a subject chooses to think of. Even whole sentences might not be too distant a prospect for the model, saysMitchell. "Now that we can see individual words, it gives the scaffolding for starting to see what the brain does with multiple words as it assembles them," he says. (45)
Models such as this one could also be useful in diagnosing disorders of language or helping students pick up a foreign language. In semantic dementia, for example, people lose the ability to remember the meanings of things--shown a picture of a chihuahua, they can only recall "dog", for example--but little is known about what exactly goes wrong in the brain. "We could look at what the neural encoding is for this," says Mitchell.
[A] The team then used functional magnetic resonance imaging (FMRI) to scan the brains of 9 volunteers as they looked at images of the nouns
[B] The study can predict what picture a person is seeing from a selection of more than 100, reported by Nature earlier this year
[C] The model may help to resolve questions about how the brain processes words and language, and might even lead to techniques for decoding people’s thoughts
[D] This gives researchers the chance to understand the "mental chemistry" that the brain does when it processes such phrases, Mitchell suggests
[E] This research may be useful for a human computer interface but does not capture the complex network that allows a real brain to learn and use words in a creative way
[F] The team started with the assumption that the brain processes words in terms of how they relate to movement and sensory information
[G] The new model is different in that it has to look at the meanings of the words, rather than just lower-level visual features of a picture