10/28/2016 Webmaster
A Poor Man QnA Maker
First, let me point out that this is an experiment that mostly failed, and this is a bit of a code dump, but there are parts that may still prove useful in the future.
Basically, you can simulate a service such as the Microsoft QnA Maker using the Microsoft Cognitive Services - Text Analytics API.
The Microsoft QnA Maker was covered in the article: Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot.
In the sample application for this article, you first download it (from the Downloads page on this site) and then open it in Visual Studio.
Next, you obtain a free Text Analytics subscription key from: https://www.microsoft.com/cognitive-services/en-US/subscriptions and put it in the web.config file.
Open the database in the App_Data folder.
Open the QuestionAnswers table.
Enter your items for Question and Answer (and set Processed to False).
Run the application…
Click Update Database to run the questions and answers through the Microsoft Cognitive Services - Text Analytics API.
This will update the KeyPhrases table with the results from the Microsoft Cognitive Services - Text Analytics API.
Now you can enter a question, that will also be analyzed by the Microsoft Cognitive Services - Text Analytics API, and after comparing those results with what is in the database, it will return the best answer… sometimes…
Methodology
The application uses the following procedure to try to find the best response:
- Pass the end-users Question through the Microsoft Cognitive Services - Text Analytics API and receive a collection of Key Phrases of that question.
- Find all Questions in the database that have any Key Phrases that match any part of the Key Phrase that matched the Question entered by the end-user.
- Find the Question in the database that has the most matches.
I need to point out that when step #3 finds the best answer it is great, but there are many times when this does not produce the best result.
The Code
As I pointed out before, while this experiment may not be considered successful, the code created may prove useful in the future.
The markup for the screen is as follows:
<div class="form-horizontal"> @Html.ValidationSummary(true, "", new { @class = "text-danger" }) @using (Html.BeginForm()) { <div class="form-group"> <div class="col-md-10"> <input name="submit" type="submit" value="Update Database" /> </div> </div> <div class="form-group"> <div class="col-md-10"> @Html.TextBox("Question", "", new { style = "width: 500px;" }) <input name="submit" type="submit" value="Ask Question" /> </div> </div> } <div class="form-group"> <div class="col-md-10"> @Html.DisplayFor(model => model.Answer, new { htmlAttributes = new { @class = "form-control" } }) </div> </div> </div>
When the Update Database button is pressed, the following method in the HomeController.cs file is triggered:
#region The "Update Database" button was pressed if (Request.Form["submit"] == "Update Database") { // Update the database by calling Cognitive Services for any // question that has its Processed flag set to false int intQuestionsUpdated = await UpdateDatabase(); // Return a response indicating how many questions were updated QnAResponse.Answer = $"There are {intQuestionsUpdated} records that have been updated."; } #endregion
This calls calls the UpdateDatabase() method:
private async Task<int> UpdateDatabase() { //Get all records in the database that were not updated var colQuestions = (from QuestionsAnswers in db.QuestionsAnswers where QuestionsAnswers.Processed == false select QuestionsAnswers).ToList(); // Loop through each Question var documents = new List<DocumentInput>(); foreach (var objQuestion in colQuestions) { // Clear any related KeyPhrases for that Question var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords where KeyPhraseRecord.QuestionsAnswer.Id == objQuestion.Id select KeyPhraseRecord); foreach (var objKeyPhraseRecord in colKeyPhraseRecords) { db.KeyPhraseRecords.Remove(objKeyPhraseRecord); } // Add the text of the current Question (and Answer) to the documents list documents.Add(new DocumentInput { language = "en", id = objQuestion.Id, text = objQuestion.Question + " " + objQuestion.Answer, }); } // Save any changes to the database db.SaveChanges(); // Submit the Batch var TextAnalyticsJsonResponse = await SubmitBatch(documents); // Insert results to the database InsertIntoDatabase(TextAnalyticsJsonResponse); // Return the number of Questions that were updated return colQuestions.Count; }
This method calls two other methods:
- SubmitBatch – Submits a query to the Microsoft Cognitive Text Analytics API. This method will also be called when analyzing the question inputted by the end-user.
- InsertIntoDatabase – Inserts the results from the Microsoft Cognitive Text Analytics API into the database.
The SubmitBatch() method is shown below:
private async Task<BatchResult> SubmitBatch(List<DocumentInput> documents) { // Send the batch to Congnitive Services var client = new HttpClient { DefaultRequestHeaders = { { "Ocp-Apim-Subscription-Key", apiKey}, { "Accept", "application/json"} } }; // Pass all the documents as a single batch var TextAnalyticsInput = new BatchInput { documents = documents }; // Submit batch var json = JsonConvert.SerializeObject(TextAnalyticsInput); var TextAnalyticsPost = await client.PostAsync(queryUri, new StringContent(json, Encoding.UTF8, "application/json")); var TextAnalyticsRawResponse = await TextAnalyticsPost.Content.ReadAsStringAsync(); var TextAnalyticsJsonResponse = JsonConvert.DeserializeObject<BatchResult>(TextAnalyticsRawResponse); // Return the response from Cognitive Services return TextAnalyticsJsonResponse; }
This is the code for the InsertIntoDatabase() method:
private void InsertIntoDatabase(BatchResult TextAnalyticsJsonResponse) { // Ensure we have records to process if (TextAnalyticsJsonResponse.documents != null) { // Loop through each response foreach (var ResponseDocument in TextAnalyticsJsonResponse.documents) { // Get QuestionsAnswer int intQuestionAnswerId = Convert.ToInt32(Convert.ToDouble(ResponseDocument.id)); var objQuestionsAnswer = (from QuestionsAnswers in db.QuestionsAnswers where QuestionsAnswers.Id == intQuestionAnswerId select QuestionsAnswers).FirstOrDefault(); // Set this QuestionAnswer as Processed objQuestionsAnswer.Processed = true; // Loop through each KeyPhrase returned for the document foreach (var keyPhrase in ResponseDocument.keyPhrases) { // Add each KeyPhrase to the Question KeyPhraseRecord objKeyPhraseRecord = new KeyPhraseRecord(); objKeyPhraseRecord.QuestionsAnswer = objQuestionsAnswer; objKeyPhraseRecord.KeyPhrase = keyPhrase; db.KeyPhraseRecords.Add(objKeyPhraseRecord); db.SaveChanges(); } } } }
When a question is asked by the end-user the following code is triggered:
#region "Ask Question" button was pressed if (Request.Form["submit"] == "Ask Question") { // Ensure that a question was passed if (Question != null) { // Return a response QnAResponse.Answer = "Got a Question"; // Create a Document collection var documents = new List<DocumentInput>(); // Add the text of the current Question to the documents list documents.Add(new DocumentInput { language = "en", id = 1, text = Question, }); // Submit the Batch -- (only one question) var TextAnalyticsJsonResponse = await SubmitBatch(documents); if (TextAnalyticsJsonResponse.documents[0].keyPhrases[0] != "") { var TextDatabaseResponse = GetDatabaseResponse(TextAnalyticsJsonResponse.documents[0].keyPhrases); // Return a response QnAResponse.Answer = $"If your question is: {TextDatabaseResponse.Question}. The answer is: {TextDatabaseResponse.Answer}."; } else { // Return a response QnAResponse.Answer = "Sorry there is no match to your question. Perhaps ask a longer question?"; } } } #endregion
This method calls the SubmitBatch() method covered earlier.
It then takes those results, and analyzes them against data in the database, to find the best matching question, using the following code:
private QuestionsAnswer GetDatabaseResponse(List<string> paramKeyPhrases) { QuestionsAnswer objQuestionsAnswer = new QuestionsAnswer(); // Create a list of KeyPhraseRecords List<KeyPhraseRecord> ColCompleteKeyPhraseRecords = new List<KeyPhraseRecord>(); // Loop through each subject passed foreach (var Phrase in paramKeyPhrases) { // Find records with at least one match var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords where Phrase.Contains(KeyPhraseRecord.KeyPhrase) || KeyPhraseRecord.KeyPhrase.Contains(Phrase) select KeyPhraseRecord).ToList(); // Add matching KeyPhraseRecords to the complete list ColCompleteKeyPhraseRecords.AddRange(colKeyPhraseRecords); } // Group the Questions var groupedQuestionsAnswerList = ColCompleteKeyPhraseRecords .GroupBy(x => x.QuestionsAnswer.Id) .Select(grp => grp.ToList().FirstOrDefault().QuestionsAnswer) .ToList(); // Create a list of questions and the count of matches Dictionary<QuestionsAnswer, int> colQuestionsAndMatches = new Dictionary<QuestionsAnswer, int>(); // Loop through all QuestionAnswers foreach (var objQuestionAnswer in groupedQuestionsAnswerList) { int intMatchCount = 0; // Lopp through each KeyPhrase of each QuestionAnswer foreach (var objKeyPhrase in objQuestionAnswer.KeyPhrases) { // Lopp through each keyPhrase passed foreach (var Phrase in paramKeyPhrases) { // See if there is a match if ( (objKeyPhrase.KeyPhrase.ToLower().Contains(Phrase.ToLower())) || (Phrase.ToLower().Contains(objKeyPhrase.KeyPhrase.ToLower())) ) { // Update MatchCount intMatchCount++; } } } // Add Question and its count of matches colQuestionsAndMatches.Add(objQuestionAnswer, intMatchCount); } if (colQuestionsAndMatches.Count > 0) { // Get the record with the highest matches var objHighestMatchedRecord = (from Question in colQuestionsAndMatches orderby Question.Value descending select Question).FirstOrDefault(); objQuestionsAnswer = objHighestMatchedRecord.Key; } return objQuestionsAnswer; }
Compared Against The Original QnA Maker
In the original article, I imported questions and answers from my wife’s website.
You can play with the Bot I made at this link.
I did the same with my Poor Man’s QnA Maker…
While it performed well on some questions…
On others it didn’t do so well…
This is the response the QnA Maker website gave to the same question (from above):
Conclusion
Basically you can’t underestimate the power of machine learning that the original QnA Maker provides.
To get the best answer, you would want to employ machine learning that Language Understanding Intelligent Service (LUIS) employs. The LUIS service doesn’t just count the number of times a Key Phrase matches. It evaluates the context of the matching.
However, the LUIS service has a limitation on the number of intents you can create in a single application.
However, the LUIS service does have an API that may allow a way to chain multiple applications together and that may be explored in a future article.
Links
Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot
Implementing Language Understanding Intelligent Service (LUIS) In Microsoft Bot FrameworkQnA Maker Application Website
Create your first QnA bot using botframework’s QnA Maker
Download
You can download the code from the Download page