Kore.ai’s multi-pronged approach to Natural Language Processing (NLP) detects intent of the user and entities with very high accuracy by processing the user input against three different engines.
NLP will be essential in understanding the true voice of the user, and facilitating more seamless interaction.
of CAGR would be achieved by companies by 2020 due to untapped market space of adoption in NLP – Markets and Markets
is expected to be the market worth of Natural Language Processing by 2021. – Markets and Markets
Connected things by 2020 and the fact that there may be 85% of customer interactions without humans using NLP technology.
Conversational AI-first will supersede “cloud-first, mobile-first” as the most important, high-level imperative for the next ten years.
NLP driven conversational interfaces are the future
Natural Language Processing (NLP) is the ability of computers to understand and process human language. In the realm of chatbots, NLP is used to determine a user’s intention and to extract information from an utterance and to carry on a conversation with the user in order to execute and complete a task.
Most intents are simple, discrete tasks like “Find Product”, “Transfer Funds”, “Book Flight”, and are typically described with a verb and noun combination. These type of intents will initiate a dialog with the user to capture more information, to fetch and update data from remote systems and to inform the user of progress.
The goal of intent recognition is to match a user utterance with its correctly intended task or question. We do that through several different training models that define the combinations of words that typically indicate an intent. Training can be as simple as a few synonyms, for example, “find”, “locate”, “search”, to sample sentences, e.g. “I want to find some shoes”, “Can we do a shoe search?”, to complex patterns that cover a large number of possibilities in a single phrase, e.g. “~findverbs ~producttypes”.
A multi-pronged approach to NLP design
Accelerate intent recognition
Kore.ai employs three distinct NLP engines, each with a different training model, to quickly and robustly build chatbot solutions. The three engines complement each other with different perspectives and overcome the weaknesses of any one individual model. Their results are correlated and resolved to accurately identify intents. This method is unique to Kore.ai, while most other solutions depend solely on one.
The fundamental meaning engine is a deterministic process that uses semantic rules and language context to find an intent match.
The machine learning engine uses statistical modeling and neural networks to train an intent prediction model from a set of example sentences for each intent.
The knowledge graph model is based around an ontology representing the hierarchical structure that is inherent in a set of questions.
Ranking & resolver
Compares the results from the primary intent recognition engines to determine the most appropriate intent given the user’s utterance.
Fundamental Meaning: Train your bots using synonyms, patterns
Machine Learning model
Create Bot Ontologies using Knowledge Graph
Ranking and Resolver: Score Winning Intent
Advantages of using Kore.ai’s multi-pronged NLP engine
The individual engines have their own specialized capabilities that cover weaknesses of any one individual model. Different use cases can be trained with the mechanisms that are appropriate. Indeed bot testing can proceed without comprehensive training and as the design solidifies, more training can be added.
The goal of entity extraction is to identify elements needed to complete the task. These elements can be simple items like numbers and dates to complex items like addresses and airport names to user defined domains such as product categories. Out of the box the platform supports identification and extraction of 20+ system entities.
Kore.ai Entity extraction capabilities
Trainable named entity extraction (NER)
Ability to train the bot using NER (Named Entity Recognition), a neural network based model, to identify entities. The NER approach best suits detecting an entity where information is provided as unformatted data. For entities like Date and Time, the platform has been trained with a large data set.
Kore.ai NLP engine recognizes system defined entities automatically that do not require pre-configuration or training by developers. Either a single value or multiple values can be extracted.
Alternate intent recognition
At any entity prompt a user may respond with the appropriate value along with another task to perform. The platform will detect the user intent and add it to the context for developer to conditionally initiate, either immediately or at the conclusion of the current task, or at some appropriate point in between.
A composite entity is a collection of individual entities that typically occur together and should be treated as one unit. For example, “black dress shoes”, “2 lb potatoes”. The same custom entity can be used to extract different combinations of sub-entities, “I want to buy 2 lb potatoes, 500 ml of milk and 4 bananas”.
Entities values that follow a specific pattern like account numbers and SKUs can be identified by regular expressions.
Enterprise PII masking
Identification and redaction of confidential information to protect personally identifiable information.
Extracts terms from a developer defined domain, which may be dynamically created based on the current state of the dialog, or from one of the 2000+ concepts delivered with the platform. The bot developer can augment those standard concepts with their own additions.
Conversational intelligence through NLP
Powerful outcomes from implementing a well architectured NLP design
Benefits of building Chatbots using Kore.ai
Kore.ai takes a unique hybrid approach to understand user intent by using machine learning model based engine, semantic rules driven model and domain taxonomy and ontology based model, our bots can not only understand the user’s input with higher accuracy, but also intelligently handle complex human conversation.
Bots interpret customer utterances accurately with fewer false positives
Bots communicate with users comprehensively
Developers resolve development gaps faster
Bots require less training data to be NL capable
Developers can repurpose training data
Faster resolution of false positives
Uses statistical modeling for conflict resolution and user input failures