Retraining an Azure Machine Learning Application

Mar 26

Written by:
3/26/2017 9:29 AM  RssIcon

 

image

You can  programmatically retrain an Azure Machine Learning Web Service thereby improving the predictions of your predictive applications.

This article is part of a series where we create an Azure Machine Learning experiment that predicts the price of a vehicle given parameters such as make, horsepower, and body style. It does that by creating a Model based on prices of previous vehicles. We then operationalize the model by creating a web service. Finally, we consume that web service using an Angular 2+ application.

In this installment, we we will take the output from our Angular 2+ application (that consumes the web service), and update its underlying predictive Model by retraining it using new data produced by the Angular 2+ application (the Angular 2+ application allows the user to record actual prices of vehicles based on the options selected). We will then use this updated web service for future predictions.

The series is as follows:

The Retraining Process

To implement the retraining, we will perform the following functions:

  • Prepare our retraining data - We will export recorded automobile price sales from the Angular 2+ application, and combine that data with the data we used to previously train our Model (each time we retrain our Model, we have to use all the available data).
  • Create a new iLearner file using the Training experiment – We will create a new web service, using the original Auto Price Experiment (not the Auto Price Experiment [Predictive Exp.]). This will allow us to pass the new training data to it and retrieve a new iLearner file that we will use for future improved predictions.
  • Add a new endpoint to the Predictive Web Service and Patch the iLearner file to it -  To consume the new iLearner file, we will add a new web service endpoint to the Auto Price Experiment [Predictive Exp.] web service and perform a Patch to have it use the new iLearner Model. We will then point the Angular 2+ application to this new endpoint for improved predictions.

Prepare The Training Data

Currently our Angular 2+ application calls the endpoint of the Auto Price Experiment [Predictive Exp.] web service:

image

This allows us to pass parameters to the predictive Model and receive price predictions.

image

The Angular 2+ application also allows us to record prices.

image

These prices can be exported to a .csv file.

We will record some prices and export them to a .csv file and save it.

image

We will log into the Azure Machine Learning Studio and select the Auto Price Experiment we created earlier.

image

We will ensure we have selected the Training Experiment and we will download the Automobile price data (Raw) file.

image

We will cut and paste the data from the Angular 2+ application into the file and save it.

We now have our new training data.

Set-up An Azure Storage Account

image

We will need an Azure Storage account to perform the retraining process, so if you do not have one, head over to https://portal.azure.com/ and create one.

image

Next, download and install the Azure Storage Explorer.

image

Run the Azure Storage Explorer.

image

Select Add an Azure Account and Sign in.

image

Navigate to the Azure storage account you created and expand the tree to show all the nodes.

Right-click on the Blob Containers node and select Create Blob Container.

 image

Create a container called experimentoutput.

We will use this container to create and store our iLearner Model.

image

If you then click on the storage account and then look in the Properties, you can retrieve account name and key.

Us the image above to locate and copy down the following values:

  • StorageAccountName
  • StorageAccountKey
  • StorageContainerName

We will need these values for the Batch Retraining program we will create later.

 

 image

We will now create a Retraining Web Service.

image

Log into the Azure Machine Learning Studio and select the Auto Price Experiment we created earlier.

image

Ensure that you are on the Training experiment tab.

Select and drag a Web Service Input to the design surface.

Connect it to the first input on the Execute R Script module.

image

Select and drag two Web Service output modules to the design surface.

Connect one to the output of the Train Model module (this will be used to create the iLearner file in the Azure Storage container).

Connect one to the output of the Evaluate Model module (this will be used to retrieve a .csv file that will indicate how well the Model currently performs).

image

Select Save As.

image

Name the experiment Auto Price Experiment (Training).

Note: At this time retraining works best in workspaces in the central us Azure region.

image

You will now be in the new experiment.

Click the Run button (to validate the experiment).

image

Next, click the Set Up Web Service button and select Deploy Web Service (Classic).

image

If we switch to the Web Services node we can see the newly created Auto Price Experiment (Training) web service.

Click on it to select it.

image

This will take you to the Dashboard for the Web Service.

Copy the API key and save it. You will need this for the Batch Retraining program we will create later.

When we submit data to retrain the web service, it must be done as a Batch Execution.

Click the Batch Execution link.

image

This will open the configuration page for the web service.

Click the Sample Code button.

image

This will take you to the sample code to call the web service.

Click on the C# tab and copy all the code.

Create The Batch Retraining Program

image

Open Visual Studio 2017 (note: Visual Studio 2017 Community is free).

image

Select File then New then Project.

image

Create a new Console App.

image

Open the Program.cs file and replace all the code with the code you copied from the Azure Machine Learning Studio web site.

image

Follow the directions to install the required assemblies:

  • Install-Package Microsoft.AspNet.WebApi.Client
  • Install-Package WindowsAzure.Storage

 

image

Update the code with the values saved from the earlier steps.

Set the value for UploadFileToBlob to the location on your computer where you saved the .csv file we downloaded earlier (and updated with the data from the Angular 2+ application).

image

Select Properties in the Solution Explorer, then Application, then Startup object, and select the CallBatchExecutionService.Program method.

image

Build The application

image

Note: If you get the error:

Argument 2: cannot convert from 'System.IO.FileMode' to 'Microsoft.WindowsAzure.Storage.AccessCondition'

Simply remove the FileMode.Open value.

image

Run the application.

image

The job will run and inform you when it is finished.

image

If we open the container in the Azure Storage Explorer, we will see the input file (that was uploaded by the program we created), and the output files.

image

For example, we can open the output .csv file to see the output from the Evaluate Model module.

Note: See Evaluate Model for instructions on how to properly interpret results. For example, with Coefficient of Determination, “…caution should be used in interpreting R2 values, as low values can be entirely normal and high values can be suspect.”

Get Required Values

image

The output will contain following values that you will need for the final step:

  • BaseLocation
  • RelativeLocation
  • SasBlobToken

You will want to gather the values for the output that creates the iLearner file (not the output that creates the .csv file).

This is how you would get the values for a normal automated process.

Note: These values represent a lease for a  Shared Access Signature to the iLearner file that will expire in 1.5 hours. If you don’t complete the final step (creating a new endpoint on the predictive web service and updating it with the new iLearner model) the lease will expire. The file will remain but the access key to get to it will no longer work. You can obtain a new key by running the previous process again, or by using the Azure Storage Explorer.

image

If you do need to get a new access key, open the Azure Storage Explorer, locate the iLearner file, right-click on it, and select Get Shared Access Signature.

image

Click Create.

image

You can then retrieve the needed values.

 

image

We will now add a new web service endpoint to the original web service (Auto Price Experiment [Predictive Exp.] not Auto Price Experiment (Training)) and perform a Patch on that new endpoint to have it use the new iLearner Model.

We will then point the Angular 2+ application to this new endpoint.

Add A New Endpoint And Patch It

image

Log into the Azure Machine Learning Studio and select the Auto Price Experiment [Predictive Exp.].

image

Click the Manage endpoints link.

image

Click the New button to create a new endpoint.

image

Fill in the form and click the Save button.

image

The new endpoint will display.

Click on it to select it.

image

Click the Consume link, then copy and save the Primary Key (you will need it for the API Key in the Patch service created in the next steps).

image

Also copy and save the value in the Request-Response section (you will need it for the BaseAddress in the Angular 2+ application that we will update in the final step).

image

Scroll down to the Patch section and click the API Help link.

image

This will open the Patch configuration page.

Click the Sample Code link.

image

Click the C# tab and copy the sample code.

image

Use Visual Studio to create a new console program.

image

Follow the directions (that you will see in the code comments) to install the required assemblies.

Update the code with the values saved from the earlier steps.

(Remember that apiKey is the Primary Key that we saved on the Consume page earlier)

image

Run the application.

If a window opens for a few seconds then closes without showing any errors, this means the Patch worked.

Consume The New Endpoint

image

Open the Angular 2+ application we created in the tutorial: An Angular 2+ Application Consuming an Azure Machine Learning Model in Visual Studio.

Open the appsettings.json file and update the PrimaryKey with the PrimaryKey you copied earlier and the BaseAddress with the Request-Response value you copied earlier.

However, with BaseAddress, change the ending of the value from:

image

To:

image

image

Run the application.

image

You will now be able to retrieve improved predictions.

 

Azure Machine Learning Retraining Links

Machine Learning Links

Links