First, let me point out that this is an experiment that mostly failed, and this is a bit of a code dump, but there are parts that may still prove useful in the future.

Basically, you can simulate a service such as the Microsoft QnA Maker using the Microsoft Cognitive Services - Text Analytics API.

The Microsoft QnA Maker was covered in the article: Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot.

In the sample application for this article, you first download it (from the Downloads page on this site) and then open it in Visual Studio .

Next, you obtain a free Text Analytics subscription key from: https://www.microsoft.com/cognitive-services/en-US/subscriptions and put it in the web.config file.

Open the database in the App_Data folder.

Open the QuestionAnswers table.

Enter your items for Question and Answer (and set Processed to False).

Run the application…

Click Update Database to run the questions and answers through the Microsoft Cognitive Services - Text Analytics API.

This will update the KeyPhrases table with the results from the Microsoft Cognitive Services - Text Analytics API.

Now you can enter a question, that will also be analyzed by the Microsoft Cognitive Services - Text Analytics API, and after comparing those results with what is in the database, it will return the best answer… sometimes…

Methodology

The application uses the following procedure to try to find the best response:

1. Pass the end-users Question through the Microsoft Cognitive Services - Text Analytics API and receive a collection of Key Phrases of that question.
2. Find all Questions in the database that have any Key Phrases that match any part of the Key Phrase that matched the Question entered by the end-user.
3. Find the Question in the database that has the most matches.

I need to point out that when step #3 finds the best answer it is great, but there are many times when this does not produce the best result.

The Code

As I pointed out before, while this experiment may not be considered successful, the code created may prove useful in the future.

The markup for the screen is as follows:

<div class="form-horizontal">     @Html.ValidationSummary(true, "", new { @class = "text-danger" })     @using (Html.BeginForm())     {         <div class="form-group">             <div class="col-md-10">                 <input name="submit" type="submit" value="Update Database" />             </div>         </div>         <div class="form-group">             <div class="col-md-10">                 @Html.TextBox("Question", "", new { style = "width: 500px;" })                 <input name="submit" type="submit" value="Ask Question" />             </div>         </div>     }     <div class="form-group">         <div class="col-md-10">             @Html.DisplayFor(model => model.Answer,            new { htmlAttributes = new { @class = "form-control" } })         </div>     </div> </div>
							

When the Update Database button is pressed, the following method in the HomeController.cs file is triggered:

								#region The "Update Database" button was pressed     if (Request.Form["submit"] == "Update Database")     {         // Update the database by calling Cognitive Services for any          // question that has its Processed flag set to false         int intQuestionsUpdated = await UpdateDatabase();         // Return a response indicating how many questions were updated         QnAResponse.Answer =             $"There are {intQuestionsUpdated} records that have been updated.";     }     #endregion
							

This calls calls the UpdateDatabase() method:

								private async Task<int> UpdateDatabase()         {             //Get all records in the database that were not updated             var colQuestions = (from QuestionsAnswers in db.QuestionsAnswers                                 where QuestionsAnswers.Processed == false                                 select QuestionsAnswers).ToList();             // Loop through each Question             var documents = new List<DocumentInput>();             foreach (var objQuestion in colQuestions)             {                 // Clear any related KeyPhrases for that Question                 var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords                                            where KeyPhraseRecord.QuestionsAnswer.Id == objQuestion.Id                                            select KeyPhraseRecord);                 foreach (var objKeyPhraseRecord in colKeyPhraseRecords)                 {                     db.KeyPhraseRecords.Remove(objKeyPhraseRecord);                 }                 // Add the text of the current Question (and Answer) to the documents list                 documents.Add(new DocumentInput                 {                     language = "en",                     id = objQuestion.Id,                     text = objQuestion.Question                     + " "                     + objQuestion.Answer,                 });             }             // Save any changes to the database             db.SaveChanges();             // Submit the Batch             var TextAnalyticsJsonResponse = await SubmitBatch(documents);             // Insert results to the database             InsertIntoDatabase(TextAnalyticsJsonResponse);             // Return the number of Questions that were updated             return colQuestions.Count;         }
							

This method calls two other methods:

• SubmitBatch – Submits a query to the Microsoft Cognitive Text Analytics API. This method will also be called when analyzing the question inputted by the end-user.
• InsertIntoDatabase – Inserts the results from the Microsoft Cognitive Text Analytics API into the database.

The SubmitBatch() method is shown below:

								private async Task<BatchResult> SubmitBatch(List<DocumentInput> documents)         {             // Send the batch to Congnitive Services             var client = new HttpClient             {                 DefaultRequestHeaders = {                     { "Ocp-Apim-Subscription-Key", apiKey},                     { "Accept", "application/json"}                 }             };             // Pass all the documents as a single batch             var TextAnalyticsInput = new BatchInput             {                 documents = documents             };             // Submit batch             var json = JsonConvert.SerializeObject(TextAnalyticsInput);             var TextAnalyticsPost =                  await client.PostAsync(queryUri,                  new StringContent(json, Encoding.UTF8, "application/json"));             var TextAnalyticsRawResponse =                  await TextAnalyticsPost.Content.ReadAsStringAsync();             var TextAnalyticsJsonResponse =                  JsonConvert.DeserializeObject<BatchResult>(TextAnalyticsRawResponse);             // Return the response from Cognitive Services              return TextAnalyticsJsonResponse;         }
							

This is the code for the InsertIntoDatabase() method:

								private void InsertIntoDatabase(BatchResult TextAnalyticsJsonResponse)     {         // Ensure we have records to process         if (TextAnalyticsJsonResponse.documents != null)         {             // Loop through each response             foreach (var ResponseDocument in TextAnalyticsJsonResponse.documents)             {                 // Get QuestionsAnswer                 int intQuestionAnswerId = Convert.ToInt32(Convert.ToDouble(ResponseDocument.id));                 var objQuestionsAnswer = (from QuestionsAnswers in db.QuestionsAnswers                                             where QuestionsAnswers.Id == intQuestionAnswerId                                             select QuestionsAnswers).FirstOrDefault();                 // Set this QuestionAnswer as Processed                 objQuestionsAnswer.Processed = true;                 // Loop through each KeyPhrase returned for the document                 foreach (var keyPhrase in ResponseDocument.keyPhrases)                 {                     // Add each KeyPhrase to the Question                     KeyPhraseRecord objKeyPhraseRecord = new KeyPhraseRecord();                     objKeyPhraseRecord.QuestionsAnswer = objQuestionsAnswer;                     objKeyPhraseRecord.KeyPhrase = keyPhrase;                     db.KeyPhraseRecords.Add(objKeyPhraseRecord);                     db.SaveChanges();                 }             }         }     }
							

When a question is asked by the end-user the following code is triggered:

								#region "Ask Question" button was pressed     if (Request.Form["submit"] == "Ask Question")     {         // Ensure that a question was passed         if (Question != null)         {             // Return a response             QnAResponse.Answer = "Got a Question";             // Create a Document collection             var documents = new List<DocumentInput>();             // Add the text of the current Question to the documents list             documents.Add(new DocumentInput             {                 language = "en",                 id = 1,                 text = Question,             });             // Submit the Batch -- (only one question)             var TextAnalyticsJsonResponse = await SubmitBatch(documents);             if (TextAnalyticsJsonResponse.documents[0].keyPhrases[0] != "")             {                 var TextDatabaseResponse = GetDatabaseResponse(TextAnalyticsJsonResponse.documents[0].keyPhrases);                 // Return a response                  QnAResponse.Answer =                      $"If your question is: {TextDatabaseResponse.Question}. The answer is: {TextDatabaseResponse.Answer}.";             }             else             {                 // Return a response                  QnAResponse.Answer = "Sorry there is no match to your question. Perhaps ask a longer question?";             }         }     }     #endregion
							

This method calls the SubmitBatch() method covered earlier.

It then takes those results, and analyzes them against data in the database, to find the best matching question, using the following code:

								private QuestionsAnswer GetDatabaseResponse(List<string> paramKeyPhrases)         {             QuestionsAnswer objQuestionsAnswer = new QuestionsAnswer();             // Create a list of KeyPhraseRecords             List<KeyPhraseRecord> ColCompleteKeyPhraseRecords = new List<KeyPhraseRecord>();             // Loop through each subject passed             foreach (var Phrase in paramKeyPhrases)             {                 // Find records with at least one match                 var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords                                            where Phrase.Contains(KeyPhraseRecord.KeyPhrase)                                            ||                                            KeyPhraseRecord.KeyPhrase.Contains(Phrase)                                            select KeyPhraseRecord).ToList();                 // Add matching KeyPhraseRecords to the complete list                 ColCompleteKeyPhraseRecords.AddRange(colKeyPhraseRecords);             }             // Group the Questions             var groupedQuestionsAnswerList = ColCompleteKeyPhraseRecords                 .GroupBy(x => x.QuestionsAnswer.Id)                 .Select(grp => grp.ToList().FirstOrDefault().QuestionsAnswer)                 .ToList();             // Create a list of questions and the count of matches             Dictionary<QuestionsAnswer, int> colQuestionsAndMatches = new Dictionary<QuestionsAnswer, int>();             // Loop through all QuestionAnswers             foreach (var objQuestionAnswer in groupedQuestionsAnswerList)             {                 int intMatchCount = 0;                 // Lopp through each KeyPhrase of each QuestionAnswer                 foreach (var objKeyPhrase in objQuestionAnswer.KeyPhrases)                 {                     // Lopp through each keyPhrase passed                     foreach (var Phrase in paramKeyPhrases)                     {                         // See if there is a match                         if (                             (objKeyPhrase.KeyPhrase.ToLower().Contains(Phrase.ToLower()))                             ||                             (Phrase.ToLower().Contains(objKeyPhrase.KeyPhrase.ToLower()))                             )                         {                             // Update MatchCount                             intMatchCount++;                         }                     }                 }                 // Add Question and its count of matches                 colQuestionsAndMatches.Add(objQuestionAnswer, intMatchCount);             }             if (colQuestionsAndMatches.Count > 0)             {                 // Get the record with the highest matches                 var objHighestMatchedRecord = (from Question in colQuestionsAndMatches                                                orderby Question.Value descending                                                select Question).FirstOrDefault();                 objQuestionsAnswer = objHighestMatchedRecord.Key;             }             return objQuestionsAnswer;         }
							

Compared Against The Original QnA Maker

In the original article, I imported questions and answers from my wife’s website.

You can play with the Bot I made at this link.

I did the same with my Poor Man’s QnA Maker…

While it performed well on some questions…

On others it didn’t do so well…

This is the response the QnA Maker website gave to the same question (from above):

Conclusion

Basically you can’t underestimate the power of machine learning that the original QnA Maker provides.

To get the best answer, you would want to employ machine learning that Language Understanding Intelligent Service (LUIS) employs. The LUIS service doesn’t just count the number of times a Key Phrase matches. It evaluates the context of the matching.

However, the LUIS service has a limitation on the number of intents you can create in a single application.

However, the LUIS service does have an API that may allow a way to chain multiple applications together and that may be explored in a future article.

Links

Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot

Implementing Language Understanding Intelligent Service (LUIS) In Microsoft Bot Framework QnA Maker Application Website

Create your first QnA bot using botframework’s QnA Maker

Download

You can download the code from the Download page