Sorry, I did not quite get that
Mon, March 2, 2020
The holy grail of AI is to fully understand human language in all its nuances. To do that, it should be able to assess, extract and evaluate information from textual data. Were are we now in 2020?
Robots have come a long way since one of the first chatterbots, MIT’s Eliza, born in the 1960s was able to recognizing clue words or phrases and was programmed on a set of carefully crafted rules. Siri and Alexa are gradually getting more AI, but when was the last time you had a satisfactory conversation with them or any of existing virtual assistants? Google claims its new chatbot, called Meena, can converse with people better than any other bot out there.
It is based on conversational neural network with 2.6 billion parameters and was end-to-end trained with 40 billion words using the seq2seq model.
The current state of the art in language processing is "Transformers" - a type of neural network architecture; "Attention" a technique for paying attention to specific words and new ways of word representation - like "Embeddings from Language Models" ELMo. Seq2seq that defined Meena is a variation of Google’s Transformer — a neural network that compares words in a paragraph to each other to understand the relationship between them. Meena is a milestone in the history of Natural Language Understanding. It goes a step further than another Transformer-based Microsoft's Turng-NLG that took Massive deep learning language models (such as BERT and GPT-2) to the next level. A metric called "Sensibleness and Specificity Average (SSA)", which measures Interlocutor's ability to make sense and be specific assigns Meena 79%, 23% higher than Mitsuku and Cleverbot.
Here are examples of Meena talking:
Robots have come a long way since one of the first chatterbots, MIT’s Eliza, born in the 1960s was able to recognizing clue words or phrases and was programmed on a set of carefully crafted rules. Siri and Alexa are gradually getting more AI, but when was the last time you had a satisfactory conversation with them or any of existing virtual assistants? Google claims its new chatbot, called Meena, can converse with people better than any other bot out there.
The current state of the art in language processing is "Transformers" - a type of neural network architecture; "Attention" a technique for paying attention to specific words and new ways of word representation - like "Embeddings from Language Models" ELMo. Seq2seq that defined Meena is a variation of Google’s Transformer — a neural network that compares words in a paragraph to each other to understand the relationship between them. Meena is a milestone in the history of Natural Language Understanding. It goes a step further than another Transformer-based Microsoft's Turng-NLG that took Massive deep learning language models (such as BERT and GPT-2) to the next level. A metric called "Sensibleness and Specificity Average (SSA)", which measures Interlocutor's ability to make sense and be specific assigns Meena 79%, 23% higher than Mitsuku and Cleverbot.
Here are examples of Meena talking:
- Human: Hi!
- Meena: Hey there! What's up?
- Human: Same old, Monday blues...what are you up to?
- Meena: I'm procrastinating on a paper I'm supposed to be writing.
- Human: are you thinking of pulling an all nighter?
- Meena: Possibly. What's your paper on?
Not quite like an “I’m sorry, I didn’t quite get that” response from Siri. Meena moves us closer to conversational AI but we are not quite yet.
| . |
Adiwardana D, Luong MT, So DR, Hall J, Fiedel N, Thoppilan R, Yang Z, Kulshreshtha A, Nemade G, Lu Y, Le QV. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977. 2020 Jan 27.
Song L, Zhang Y, Wang Z, Gildea D. N-ary relation extraction using graph state lstm. arXiv preprint arXiv:1808.09101. 2018 Aug 28.
Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. arXiv preprint arXiv:1802.10569. 2018 Feb 28.
Staudemeyer RC, Morris ER. Understanding LSTM--a tutorial into Long Short-Term Memory Recurrent Neural Networks. arXiv preprint arXiv:1909.09586. 2019 Sep 12.
Song L, Zhang Y, Wang Z, Gildea D. N-ary relation extraction using graph state lstm. arXiv preprint arXiv:1808.09101. 2018 Aug 28.
Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. arXiv preprint arXiv:1802.10569. 2018 Feb 28.
Staudemeyer RC, Morris ER. Understanding LSTM--a tutorial into Long Short-Term Memory Recurrent Neural Networks. arXiv preprint arXiv:1909.09586. 2019 Sep 12.