A Poor Man QnA Maker

Oct 27

Written by:
10/27/2016 8:44 PM  RssIcon

image

First, let me point out that this is an experiment that mostly failed, and this is a bit of a code dump, but there are parts that may still prove useful in the future.

Basically, you can simulate a service such as the Microsoft QnA Maker using the Microsoft Cognitive Services - Text Analytics API.

image

The Microsoft QnA Maker was covered in the article: Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot.

image

In the sample application for this article, you first download it (from the Downloads page on this site) and then open it in Visual Studio.

Next, you obtain a free Text Analytics subscription key from: https://www.microsoft.com/cognitive-services/en-US/subscriptions and put it in the web.config file.

image

Open the database in the App_Data folder.

image

Open the QuestionAnswers table.

image

Enter your items for Question and Answer (and set Processed to False).

image

Run the application…

image

Click Update Database to run the questions and answers through the Microsoft Cognitive Services - Text Analytics API.

image

This will update the KeyPhrases table with the results from the Microsoft Cognitive Services - Text Analytics API.

image

Now you can enter a question, that will also be analyzed by the Microsoft Cognitive Services - Text Analytics API, and after comparing those results with what is in the database, it will return the best answer… sometimes…

Methodology

The application uses the following procedure to try to find the best response:

  1. Pass the end-users Question through the Microsoft Cognitive Services - Text Analytics API and receive a collection of Key Phrases of that question.
  2. Find all Questions in the database that have any Key Phrases that match any part of the Key Phrase that matched the Question entered by the end-user.
  3. Find the Question in the database that has the most matches.

I need to point out that when step #3 finds the best answer it is great, but there are many times when this does not produce the best result.

The Code

As I pointed out before, while this experiment may not be considered successful, the code created may prove useful in the future.

 

image

 

The markup for the screen is as follows:

 

<div class="form-horizontal">
    @Html.ValidationSummary(true, "", new { @class = "text-danger" })
    @using (Html.BeginForm())
    {
        <div class="form-group">
            <div class="col-md-10">
                <input name="submit" type="submit" value="Update Database" />
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-10">
                @Html.TextBox("Question", "", new { style = "width: 500px;" })
                <input name="submit" type="submit" value="Ask Question" />
            </div>
        </div>
    }
    <div class="form-group">
        <div class="col-md-10">
            @Html.DisplayFor(model => model.Answer,
           new { htmlAttributes = new { @class = "form-control" } })
        </div>
    </div>
</div>

 

When the Update Database button is pressed, the following method in the HomeController.cs file is triggered:

 

    #region The "Update Database" button was pressed
    if (Request.Form["submit"] == "Update Database")
    {
        // Update the database by calling Cognitive Services for any 
        // question that has its Processed flag set to false
        int intQuestionsUpdated = await UpdateDatabase();
        // Return a response indicating how many questions were updated
        QnAResponse.Answer =
            $"There are {intQuestionsUpdated} records that have been updated.";
    }
    #endregion

 

This calls calls the UpdateDatabase() method:

 

        private async Task<int> UpdateDatabase()
        {
            //Get all records in the database that were not updated
            var colQuestions = (from QuestionsAnswers in db.QuestionsAnswers
                                where QuestionsAnswers.Processed == false
                                select QuestionsAnswers).ToList();
            // Loop through each Question
            var documents = new List<DocumentInput>();
            foreach (var objQuestion in colQuestions)
            {
                // Clear any related KeyPhrases for that Question
                var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords
                                           where KeyPhraseRecord.QuestionsAnswer.Id == objQuestion.Id
                                           select KeyPhraseRecord);
                foreach (var objKeyPhraseRecord in colKeyPhraseRecords)
                {
                    db.KeyPhraseRecords.Remove(objKeyPhraseRecord);
                }
                // Add the text of the current Question (and Answer) to the documents list
                documents.Add(new DocumentInput
                {
                    language = "en",
                    id = objQuestion.Id,
                    text = objQuestion.Question
                    + " "
                    + objQuestion.Answer,
                });
            }
            // Save any changes to the database
            db.SaveChanges();
            // Submit the Batch
            var TextAnalyticsJsonResponse = await SubmitBatch(documents);
            // Insert results to the database
            InsertIntoDatabase(TextAnalyticsJsonResponse);
            // Return the number of Questions that were updated
            return colQuestions.Count;
        }

 

This method calls two other methods:

  • SubmitBatch – Submits a query to the Microsoft Cognitive Text Analytics API. This method will also be called when analyzing the question inputted by the end-user.
  • InsertIntoDatabase – Inserts the results from the Microsoft Cognitive Text Analytics API into the database.

 

The SubmitBatch() method is shown below:

 

        private async Task<BatchResult> SubmitBatch(List<DocumentInput> documents)
        {
            // Send the batch to Congnitive Services
            var client = new HttpClient
            {
                DefaultRequestHeaders = {
                    { "Ocp-Apim-Subscription-Key", apiKey},
                    { "Accept", "application/json"}
                }
            };
            // Pass all the documents as a single batch
            var TextAnalyticsInput = new BatchInput
            {
                documents = documents
            };
            // Submit batch
            var json = JsonConvert.SerializeObject(TextAnalyticsInput);
            var TextAnalyticsPost = 
                await client.PostAsync(queryUri, 
                new StringContent(json, Encoding.UTF8, "application/json"));
            var TextAnalyticsRawResponse = 
                await TextAnalyticsPost.Content.ReadAsStringAsync();
            var TextAnalyticsJsonResponse = 
                JsonConvert.DeserializeObject<BatchResult>(TextAnalyticsRawResponse);
            // Return the response from Cognitive Services 
            return TextAnalyticsJsonResponse;
        }

 

 

This is the code for the InsertIntoDatabase() method:

 

    private void InsertIntoDatabase(BatchResult TextAnalyticsJsonResponse)
    {
        // Ensure we have records to process
        if (TextAnalyticsJsonResponse.documents != null)
        {
            // Loop through each response
            foreach (var ResponseDocument in TextAnalyticsJsonResponse.documents)
            {
                // Get QuestionsAnswer
                int intQuestionAnswerId = Convert.ToInt32(Convert.ToDouble(ResponseDocument.id));
                var objQuestionsAnswer = (from QuestionsAnswers in db.QuestionsAnswers
                                            where QuestionsAnswers.Id == intQuestionAnswerId
                                            select QuestionsAnswers).FirstOrDefault();
                // Set this QuestionAnswer as Processed
                objQuestionsAnswer.Processed = true;
                // Loop through each KeyPhrase returned for the document
                foreach (var keyPhrase in ResponseDocument.keyPhrases)
                {
                    // Add each KeyPhrase to the Question
                    KeyPhraseRecord objKeyPhraseRecord = new KeyPhraseRecord();
                    objKeyPhraseRecord.QuestionsAnswer = objQuestionsAnswer;
                    objKeyPhraseRecord.KeyPhrase = keyPhrase;
                    db.KeyPhraseRecords.Add(objKeyPhraseRecord);
                    db.SaveChanges();
                }
            }
        }
    }

 

When a question is asked by the end-user the following code is triggered:

 

   #region "Ask Question" button was pressed
    if (Request.Form["submit"] == "Ask Question")
    {
        // Ensure that a question was passed
        if (Question != null)
        {
            // Return a response
            QnAResponse.Answer = "Got a Question";
            // Create a Document collection
            var documents = new List<DocumentInput>();
            // Add the text of the current Question to the documents list
            documents.Add(new DocumentInput
            {
                language = "en",
                id = 1,
                text = Question,
            });
            // Submit the Batch -- (only one question)
            var TextAnalyticsJsonResponse = await SubmitBatch(documents);
            if (TextAnalyticsJsonResponse.documents[0].keyPhrases[0] != "")
            {
                var TextDatabaseResponse = GetDatabaseResponse(TextAnalyticsJsonResponse.documents[0].keyPhrases);
                // Return a response 
                QnAResponse.Answer = 
                    $"If your question is: {TextDatabaseResponse.Question}. The answer is: {TextDatabaseResponse.Answer}.";
            }
            else
            {
                // Return a response 
                QnAResponse.Answer = "Sorry there is no match to your question. Perhaps ask a longer question?";
            }
        }
    }
    #endregion

 

This method calls the SubmitBatch() method covered earlier.

It then takes those results, and analyzes them against data in the database, to find the best matching question, using the following code:

 

        private QuestionsAnswer GetDatabaseResponse(List<string> paramKeyPhrases)
        {
            QuestionsAnswer objQuestionsAnswer = new QuestionsAnswer();
            // Create a list of KeyPhraseRecords
            List<KeyPhraseRecord> ColCompleteKeyPhraseRecords = new List<KeyPhraseRecord>();
            // Loop through each subject passed
            foreach (var Phrase in paramKeyPhrases)
            {
                // Find records with at least one match
                var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords
                                           where Phrase.Contains(KeyPhraseRecord.KeyPhrase)
                                           ||
                                           KeyPhraseRecord.KeyPhrase.Contains(Phrase)
                                           select KeyPhraseRecord).ToList();
                // Add matching KeyPhraseRecords to the complete list
                ColCompleteKeyPhraseRecords.AddRange(colKeyPhraseRecords);
            }
            // Group the Questions
            var groupedQuestionsAnswerList = ColCompleteKeyPhraseRecords
                .GroupBy(x => x.QuestionsAnswer.Id)
                .Select(grp => grp.ToList().FirstOrDefault().QuestionsAnswer)
                .ToList();
            // Create a list of questions and the count of matches
            Dictionary<QuestionsAnswer, int> colQuestionsAndMatches = new Dictionary<QuestionsAnswer, int>();
            // Loop through all QuestionAnswers
            foreach (var objQuestionAnswer in groupedQuestionsAnswerList)
            {
                int intMatchCount = 0;
                // Lopp through each KeyPhrase of each QuestionAnswer
                foreach (var objKeyPhrase in objQuestionAnswer.KeyPhrases)
                {
                    // Lopp through each keyPhrase passed
                    foreach (var Phrase in paramKeyPhrases)
                    {
                        // See if there is a match
                        if (
                            (objKeyPhrase.KeyPhrase.ToLower().Contains(Phrase.ToLower()))
                            ||
                            (Phrase.ToLower().Contains(objKeyPhrase.KeyPhrase.ToLower()))
                            )
                        {
                            // Update MatchCount
                            intMatchCount++;
                        }
                    }
                }
                // Add Question and its count of matches
                colQuestionsAndMatches.Add(objQuestionAnswer, intMatchCount);
            }
            if (colQuestionsAndMatches.Count > 0)
            {
                // Get the record with the highest matches
                var objHighestMatchedRecord = (from Question in colQuestionsAndMatches
                                               orderby Question.Value descending
                                               select Question).FirstOrDefault();
                objQuestionsAnswer = objHighestMatchedRecord.Key;
            }
            return objQuestionsAnswer;
        }

 

 

Compared Against The Original QnA Maker

image

In the original article, I imported questions and answers from my wife’s website.

You can play with the Bot I made at this link.

image

I did the same with my Poor Man’s QnA Maker

image

While it performed well on some questions…

image

On others it didn’t do so well…

This is the response the QnA Maker website gave to the same question (from above):

image

Conclusion

image

Basically you can’t underestimate the power of machine learning that the original QnA Maker provides.

To get the best answer, you would want to employ machine learning that Language Understanding Intelligent Service (LUIS) employs. The LUIS service doesn’t just count the number of times a Key Phrase matches. It evaluates the context of the matching.

However, the LUIS service has a limitation on the number of intents you can create in a single application.

However, the LUIS service does have an API that may allow a way to chain multiple applications together and that may be explored in a future article.

Links

Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot

Implementing Language Understanding Intelligent Service (LUIS) In Microsoft Bot Framework

QnA Maker Application Website

Create your first QnA bot using botframework’s QnA Maker

Download

You can download the code from the Download page