Software that Swaps Out Words Can Now Fool the AI Behind Alexa and Siri

Software called TextFooler can trick natural-language processing (NLP) systems into misunderstanding text just by replacing certain words in a sentence with synonyms. From a report: In tests, it was able to drop the accuracy of three state-of-the-art NLP systems dramatically. For example, Google’s powerful BERT neural net was worse by a factor of five to seven at identifying whether reviews on Yelp were positive or negative. The software, developed by a team at MIT, looks for the words in a sentence that are most important to an NLP classifier and replaces them with a synonym that a human would find natural. For example, changing the sentence “The characters, cast in impossibly contrived situations, are totally estranged from reality” to “The characters, cast in impossibly engineered circumstances, are fully estranged from reality” makes no real difference to how we read it. But the tweaks made an AI interpret the sentences completely differently.