3/26/2017 Webmaster

Retraining an Azure Machine Learning Application

You can programmatically retrain an Azure Machine Learning Web Service thereby improving the predictions of your predictive applications.

This article is part of a series where we create an Azure Machine Learning experiment that predicts the price of a vehicle given parameters such as make, horsepower, and body style. It does that by creating a Model based on prices of previous vehicles. We then operationalize the model by creating a web service. Finally, we consume that web service using an Angular 2+ application.

In this installment, we will take the output from our Angular 2+ application (that consumes the web service), and update its underlying predictive Model by retraining it using new data produced by the Angular 2+ application (the Angular 2+ application allows the user to record actual prices of vehicles based on the options selected). We will then use this updated web service for future predictions.

The series is as follows:

An End-To-End Azure Machine Learning Studio Application – Create an Azure Machine Learning experiment and operationalize it by creating a web service and consuming it using Microsoft Excel.
An Angular 2+ Application Consuming an Azure Machine Learning Model – Create an Angular 2+ application that consumes a web service created from an Azure Machine Learning experiment.
(This article) Retraining an Azure Machine Learning Application – Covers the steps needed to update the Azure Machine Learning Model with new data to improve its predictions.

The Retraining Process

To implement the retraining, we will perform the following functions:

Prepare our retraining data - We will export recorded automobile price sales from the Angular 2+ application, and combine that data with the data we used to previously train our Model (each time we retrain our Model, we have to use all the available data).
Create a new iLearner file using the Training experiment – We will create a new web service, using the original Auto Price Experiment (not the Auto Price Experiment [Predictive Exp.]). This will allow us to pass the new training data to it and retrieve a new iLearner file that we will use for future improved predictions.
Add a new endpoint to the Predictive Web Service and Patch the iLearner file to it - To consume the new iLearner file, we will add a new web service endpoint to the Auto Price Experiment [Predictive Exp.] web service and perform a Patch to have it use the new iLearner Model. We will then point the Angular 2+ application to this new endpoint for improved predictions.

Prepare The Training Data

Currently our Angular 2+ application calls the endpoint of the Auto Price Experiment [Predictive Exp.] web service:

This allows us to pass parameters to the predictive Model and receive price predictions.

The Angular 2+ application also allows us to record prices.

These prices can be exported to a .csv file.

We will record some prices and export them to a .csv file and save it.

We will log into the Azure Machine Learning Studio and select the Auto Price Experiment we created earlier.

We will ensure we have selected the Training Experiment and we will download the Automobile price data (Raw) file.

We will cut and paste the data from the Angular 2+ application into the file and save it.

We now have our new training data.

Set-up An Azure Storage Account

We will need an Azure Storage account to perform the retraining process, so if you do not have one, head over to https://portal.azure.com/ and create one.

Next, download and install the Azure Storage Explorer.

Run the Azure Storage Explorer.

Select Add an Azure Account and Sign in.

Navigate to the Azure storage account you created and expand the tree to show all the nodes.

Right-click on the Blob Containers node and select Create Blob Container.

Create a container called experimentoutput.

We will use this container to create and store our iLearner Model.

If you then click on the storage account and then look in the Properties, you can retrieve account name and key.

Us the image above to locate and copy down the following values:

StorageAccountName
StorageAccountKey
StorageContainerName

We will need these values for the Batch Retraining program we will create later.

We will now create a Retraining Web Service.

Log into the Azure Machine Learning Studio and select the Auto Price Experiment we created earlier.

Ensure that you are on the Training experiment tab.

Select and drag a Web Service Input to the design surface.

Connect it to the first input on the Execute R Script module.

Select and drag two Web Service output modules to the design surface.

Connect one to the output of the Train Model module (this will be used to create the iLearner file in the Azure Storage container).

Connect one to the output of the Evaluate Model module (this will be used to retrieve a .csv file that will indicate how well the Model currently performs).

Select Save As.

Name the experiment Auto Price Experiment (Training).

Note: At this time retraining works best in workspaces in the central us Azure region.

You will now be in the new experiment.

Click the Run button (to validate the experiment).

Next, click the Set Up Web Service button and select Deploy Web Service (Classic).

If we switch to the Web Services node we can see the newly created Auto Price Experiment (Training) web service.

Click on it to select it.

This will take you to the Dashboard for the Web Service.

Copy the API key and save it. You will need this for the Batch Retraining program we will create later.

When we submit data to retrain the web service, it must be done as a Batch Execution.

Click the Batch Execution link.

This will open the configuration page for the web service.

Click the Sample Code button.

This will take you to the sample code to call the web service.

Click on the C# tab and copy all the code.

Create The Batch Retraining Program

Open Visual Studio 2017 (note: Visual Studio 2017 Community is free).

Select File then New then Project.

Create a new Console App.

Open the Program.cs file and replace all the code with the code you copied from the Azure Machine Learning Studio web site.

Follow the directions to install the required assemblies:

Install-Package Microsoft.AspNet.WebApi.Client
Install-Package WindowsAzure.Storage

Update the code with the values saved from the earlier steps.

Set the value for UploadFileToBlob to the location on your computer where you saved the .csv file we downloaded earlier (and updated with the data from the Angular 2+ application).

Select Properties in the Solution Explorer, then Application, then Startup object, and select the CallBatchExecutionService.Program method.

Build The application

Note: If you get the error:

Argument 2: cannot convert from 'System.IO.FileMode' to 'Microsoft.WindowsAzure.Storage.AccessCondition'

Simply remove the FileMode.Open value.

Run the application.

The job will run and inform you when it is finished.

If we open the container in the Azure Storage Explorer, we will see the input file (that was uploaded by the program we created), and the output files.

For example, we can open the output .csv file to see the output from the Evaluate Model module.

Note: See Evaluate Model for instructions on how to properly interpret results. For example, with Coefficient of Determination, “…caution should be used in interpreting R2 values, as low values can be entirely normal and high values can be suspect.”

Get Required Values

The output will contain following values that you will need for the final step:

BaseLocation
RelativeLocation
SasBlobToken

You will want to gather the values for the output that creates the iLearner file (not the output that creates the .csv file).

This is how you would get the values for a normal automated process.

Note: These values represent a lease for a Shared Access Signature to the iLearner file that will expire in 1.5 hours. If you don’t complete the final step (creating a new endpoint on the predictive web service and updating it with the new iLearner model) the lease will expire. The file will remain but the access key to get to it will no longer work. You can obtain a new key by running the previous process again, or by using the Azure Storage Explorer.

If you do need to get a new access key, open the Azure Storage Explorer, locate the iLearner file, right-click on it, and select Get Shared Access Signature.

Click Create.

You can then retrieve the needed values.

We will now add a new web service endpoint to the original web service (Auto Price Experiment [Predictive Exp.] not Auto Price Experiment (Training)) and perform a Patch on that new endpoint to have it use the new iLearner Model.

We will then point the Angular 2+ application to this new endpoint.

Add A New Endpoint And Patch It

Log into the Azure Machine Learning Studio and select the Auto Price Experiment [Predictive Exp.].

Click the Manage endpoints link.

Click the New button to create a new endpoint.

Fill in the form and click the Save button.

The new endpoint will display.

Click on it to select it.

Click the Consume link, then copy and save the Primary Key (you will need it for the API Key in the Patch service created in the next steps).

Also copy and save the value in the Request-Response section (you will need it for the BaseAddress in the Angular 2+ application that we will update in the final step).

Scroll down to the Patch section and click the API Help link.

This will open the Patch configuration page.

Click the Sample Code link.

Click the C# tab and copy the sample code.

Use Visual Studio to create a new console program.

Follow the directions (that you will see in the code comments) to install the required assemblies.

Update the code with the values saved from the earlier steps.

(Remember that apiKey is the Primary Key that we saved on the Consume page earlier)

Run the application.

If a window opens for a few seconds then closes without showing any errors, this means the Patch worked.

Consume The New Endpoint

Open the Angular 2+ application we created in the tutorial: An Angular 2+ Application Consuming an Azure Machine Learning Model in Visual Studio.

Open the appsettings.json file and update the PrimaryKey with the PrimaryKey you copied earlier and the BaseAddress with the Request-Response value you copied earlier.

However, with BaseAddress, change the ending of the value from: