10/28/2016 Webmaster
A Poor Man QnA Maker
First, let me point out that this is an experiment that mostly failed, and this is a bit of a code dump, but there are parts that may still prove useful in the future.
Basically, you can simulate a service such as the Microsoft QnA Maker using the Microsoft Cognitive Services - Text Analytics API.
The Microsoft QnA Maker was covered in the article: Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot.
In the sample application for this article, you first download it (from the Downloads page on this site) and then open it in Visual Studio.
Next, you obtain a free Text Analytics subscription key from: https://www.microsoft.com/cognitive-services/en-US/subscriptions and put it in the web.config file.
Open the database in the App_Data folder.
Open the QuestionAnswers table.
Enter your items for Question and Answer (and set Processed to False).
Run the application…
Click Update Database to run the questions and answers through the Microsoft Cognitive Services - Text Analytics API.
This will update the KeyPhrases table with the results from the Microsoft Cognitive Services - Text Analytics API.
Now you can enter a question, that will also be analyzed by the Microsoft Cognitive Services - Text Analytics API, and after comparing those results with what is in the database, it will return the best answer… sometimes…
Methodology
The application uses the following procedure to try to find the best response:
- Pass the end-users Question through the Microsoft Cognitive Services - Text Analytics API and receive a collection of Key Phrases of that question.
- Find all Questions in the database that have any Key Phrases that match any part of the Key Phrase that matched the Question entered by the end-user.
- Find the Question in the database that has the most matches.
I need to point out that when step #3 finds the best answer it is great, but there are many times when this does not produce the best result.
The Code
As I pointed out before, while this experiment may not be considered successful, the code created may prove useful in the future.
The markup for the screen is as follows:
<div class="form-horizontal">
@Html.ValidationSummary(true, "", new { @class = "text-danger" })
@using (Html.BeginForm())
{
<div class="form-group">
<div class="col-md-10">
<input name="submit" type="submit" value="Update Database" />
</div>
</div>
<div class="form-group">
<div class="col-md-10">
@Html.TextBox("Question", "", new { style = "width: 500px;" })
<input name="submit" type="submit" value="Ask Question" />
</div>
</div>
}
<div class="form-group">
<div class="col-md-10">
@Html.DisplayFor(model => model.Answer,
new { htmlAttributes = new { @class = "form-control" } })
</div>
</div>
</div>
When the Update Database button is pressed, the following method in the HomeController.cs file is triggered:
#region The "Update Database" button was pressed
if (Request.Form["submit"] == "Update Database")
{
// Update the database by calling Cognitive Services for any
// question that has its Processed flag set to false
int intQuestionsUpdated = await UpdateDatabase();
// Return a response indicating how many questions were updated
QnAResponse.Answer =
$"There are {intQuestionsUpdated} records that have been updated.";
}
#endregion
This calls calls the UpdateDatabase() method:
private async Task<int> UpdateDatabase()
{
//Get all records in the database that were not updated
var colQuestions = (from QuestionsAnswers in db.QuestionsAnswers
where QuestionsAnswers.Processed == false
select QuestionsAnswers).ToList();
// Loop through each Question
var documents = new List<DocumentInput>();
foreach (var objQuestion in colQuestions)
{
// Clear any related KeyPhrases for that Question
var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords
where KeyPhraseRecord.QuestionsAnswer.Id == objQuestion.Id
select KeyPhraseRecord);
foreach (var objKeyPhraseRecord in colKeyPhraseRecords)
{
db.KeyPhraseRecords.Remove(objKeyPhraseRecord);
}
// Add the text of the current Question (and Answer) to the documents list
documents.Add(new DocumentInput
{
language = "en",
id = objQuestion.Id,
text = objQuestion.Question
+ " "
+ objQuestion.Answer,
});
}
// Save any changes to the database
db.SaveChanges();
// Submit the Batch
var TextAnalyticsJsonResponse = await SubmitBatch(documents);
// Insert results to the database
InsertIntoDatabase(TextAnalyticsJsonResponse);
// Return the number of Questions that were updated
return colQuestions.Count;
}
This method calls two other methods:
- SubmitBatch – Submits a query to the Microsoft Cognitive Text Analytics API. This method will also be called when analyzing the question inputted by the end-user.
- InsertIntoDatabase – Inserts the results from the Microsoft Cognitive Text Analytics API into the database.
The SubmitBatch() method is shown below:
private async Task<BatchResult> SubmitBatch(List<DocumentInput> documents)
{
// Send the batch to Congnitive Services
var client = new HttpClient
{
DefaultRequestHeaders = {
{ "Ocp-Apim-Subscription-Key", apiKey},
{ "Accept", "application/json"}
}
};
// Pass all the documents as a single batch
var TextAnalyticsInput = new BatchInput
{
documents = documents
};
// Submit batch
var json = JsonConvert.SerializeObject(TextAnalyticsInput);
var TextAnalyticsPost =
await client.PostAsync(queryUri,
new StringContent(json, Encoding.UTF8, "application/json"));
var TextAnalyticsRawResponse =
await TextAnalyticsPost.Content.ReadAsStringAsync();
var TextAnalyticsJsonResponse =
JsonConvert.DeserializeObject<BatchResult>(TextAnalyticsRawResponse);
// Return the response from Cognitive Services
return TextAnalyticsJsonResponse;
}
This is the code for the InsertIntoDatabase() method:
private void InsertIntoDatabase(BatchResult TextAnalyticsJsonResponse)
{
// Ensure we have records to process
if (TextAnalyticsJsonResponse.documents != null)
{
// Loop through each response
foreach (var ResponseDocument in TextAnalyticsJsonResponse.documents)
{
// Get QuestionsAnswer
int intQuestionAnswerId = Convert.ToInt32(Convert.ToDouble(ResponseDocument.id));
var objQuestionsAnswer = (from QuestionsAnswers in db.QuestionsAnswers
where QuestionsAnswers.Id == intQuestionAnswerId
select QuestionsAnswers).FirstOrDefault();
// Set this QuestionAnswer as Processed
objQuestionsAnswer.Processed = true;
// Loop through each KeyPhrase returned for the document
foreach (var keyPhrase in ResponseDocument.keyPhrases)
{
// Add each KeyPhrase to the Question
KeyPhraseRecord objKeyPhraseRecord = new KeyPhraseRecord();
objKeyPhraseRecord.QuestionsAnswer = objQuestionsAnswer;
objKeyPhraseRecord.KeyPhrase = keyPhrase;
db.KeyPhraseRecords.Add(objKeyPhraseRecord);
db.SaveChanges();
}
}
}
}
When a question is asked by the end-user the following code is triggered:
#region "Ask Question" button was pressed
if (Request.Form["submit"] == "Ask Question")
{
// Ensure that a question was passed
if (Question != null)
{
// Return a response
QnAResponse.Answer = "Got a Question";
// Create a Document collection
var documents = new List<DocumentInput>();
// Add the text of the current Question to the documents list
documents.Add(new DocumentInput
{
language = "en",
id = 1,
text = Question,
});
// Submit the Batch -- (only one question)
var TextAnalyticsJsonResponse = await SubmitBatch(documents);
if (TextAnalyticsJsonResponse.documents[0].keyPhrases[0] != "")
{
var TextDatabaseResponse = GetDatabaseResponse(TextAnalyticsJsonResponse.documents[0].keyPhrases);
// Return a response
QnAResponse.Answer =
$"If your question is: {TextDatabaseResponse.Question}. The answer is: {TextDatabaseResponse.Answer}.";
}
else
{
// Return a response
QnAResponse.Answer = "Sorry there is no match to your question. Perhaps ask a longer question?";
}
}
}
#endregion
This method calls the SubmitBatch() method covered earlier.
It then takes those results, and analyzes them against data in the database, to find the best matching question, using the following code:
private QuestionsAnswer GetDatabaseResponse(List<string> paramKeyPhrases)
{
QuestionsAnswer objQuestionsAnswer = new QuestionsAnswer();
// Create a list of KeyPhraseRecords
List<KeyPhraseRecord> ColCompleteKeyPhraseRecords = new List<KeyPhraseRecord>();
// Loop through each subject passed
foreach (var Phrase in paramKeyPhrases)
{
// Find records with at least one match
var colKeyPhraseRecords = (from KeyPhraseRecord in db.KeyPhraseRecords
where Phrase.Contains(KeyPhraseRecord.KeyPhrase)
||
KeyPhraseRecord.KeyPhrase.Contains(Phrase)
select KeyPhraseRecord).ToList();
// Add matching KeyPhraseRecords to the complete list
ColCompleteKeyPhraseRecords.AddRange(colKeyPhraseRecords);
}
// Group the Questions
var groupedQuestionsAnswerList = ColCompleteKeyPhraseRecords
.GroupBy(x => x.QuestionsAnswer.Id)
.Select(grp => grp.ToList().FirstOrDefault().QuestionsAnswer)
.ToList();
// Create a list of questions and the count of matches
Dictionary<QuestionsAnswer, int> colQuestionsAndMatches = new Dictionary<QuestionsAnswer, int>();
// Loop through all QuestionAnswers
foreach (var objQuestionAnswer in groupedQuestionsAnswerList)
{
int intMatchCount = 0;
// Lopp through each KeyPhrase of each QuestionAnswer
foreach (var objKeyPhrase in objQuestionAnswer.KeyPhrases)
{
// Lopp through each keyPhrase passed
foreach (var Phrase in paramKeyPhrases)
{
// See if there is a match
if (
(objKeyPhrase.KeyPhrase.ToLower().Contains(Phrase.ToLower()))
||
(Phrase.ToLower().Contains(objKeyPhrase.KeyPhrase.ToLower()))
)
{
// Update MatchCount
intMatchCount++;
}
}
}
// Add Question and its count of matches
colQuestionsAndMatches.Add(objQuestionAnswer, intMatchCount);
}
if (colQuestionsAndMatches.Count > 0)
{
// Get the record with the highest matches
var objHighestMatchedRecord = (from Question in colQuestionsAndMatches
orderby Question.Value descending
select Question).FirstOrDefault();
objQuestionsAnswer = objHighestMatchedRecord.Key;
}
return objQuestionsAnswer;
}
Compared Against The Original QnA Maker
In the original article, I imported questions and answers from my wife’s website.
You can play with the Bot I made at this link.
I did the same with my Poor Man’s QnA Maker…
While it performed well on some questions…
On others it didn’t do so well…
This is the response the QnA Maker website gave to the same question (from above):
Conclusion
Basically you can’t underestimate the power of machine learning that the original QnA Maker provides.
To get the best answer, you would want to employ machine learning that Language Understanding Intelligent Service (LUIS) employs. The LUIS service doesn’t just count the number of times a Key Phrase matches. It evaluates the context of the matching.
However, the LUIS service has a limitation on the number of intents you can create in a single application.
However, the LUIS service does have an API that may allow a way to chain multiple applications together and that may be explored in a future article.
Links
Using Microsoft Bot Framework QnA Maker To Quickly Create A Chat Bot
Implementing Language Understanding Intelligent Service (LUIS) In Microsoft Bot FrameworkQnA Maker Application Website
Create your first QnA bot using botframework’s QnA Maker
Download
You can download the code from the Download page