You can easily create a .Net Core Angular 4 application that programmatically trains your Custom Vision Service predictive model.

Microsoft’s Custom Vision Service allows you to upload sample images, classify them, train a model, and then use that model to evaluate and automatically classify your images.

It also allows you to add additional images, correct, and re-train the model. All features can be performed and invoked using the REST based API so you can incorporate all functionality into your custom programs.

This article is the third installment in a series of tutorials:

• Custom Vision Application – Create a Custom Vision project that classifies images and allows you to upload images to be classified.
• Angular 4 Application (Part One) – Creating an application that allows you to upload an image and have it classified.
• (this article) Angular 4 Application (Part Two) – Upload new training images, tag them, and re-train the model.

 

Set-up The Application

To run the sample code (available on the Downloads page of this site), we first need to obtain the Prediction URL and keys for the application (created in the first tutorial of the series: Microsoft Cognitive Custom Vision Service (Tutorial Series)).

Next, we open the .Net Core Angular 4 application (available on the Downloads page of this site), in Visual Studio 2017 (or higher), and update and save the values in the appsettings.json file.

Run The Application

We run the application.

Click the Training tab.

Select one of your Projects from the Projects dropdown, then one of the Tags for the Project in the Tags dropdown.

Any images uploaded and associated with that Tag will display in the pageable list.

(Note: You create Projects and Tags for those Projects in the Custom Vision Portal)

To add new images to be associated with the selected Tag, click the Choose button, select the files to upload, then click the Upload New Files button.

The files will be added to the list of images.

If you log into the Custom Vision Portal and select the Project, the Training Images tab, then the Tag, you will see the images.

Returning to the Angular project, we can now click the Train Project button to create the predictive model.

We can now log into the Custom Vision Portal and select the Project, and the Performance tab, to see the Iteration that is created and the performance.

We can follow the directions in the article Angular 4 Application (Part One) to update the PredictionKey and PredictionURL in the Angular application to point to this project and allow us to upload images and make predictions (on the Home tab of the Angular application).

The API

The REST based API called by the Angular application is documented at the following link: Custom Vision Service API Reference.

Get Projects

The first code to run is the code to determine what projects you have and to populate the dropdown.

The following code is used for the markup for the dropdown:

 

        <label for="Projects">Projects</label>
        <br />
        <p-dropdown id="Projects"
                    appendTo="body"
                    [options]="projectsDropdown"
                    [(ngModel)]="selectedProject"
                    [style]="{'width':'250px'}"
                    (onChange)="onProjectChange()"></p-dropdown>

 

This dropdown is filled by the following code in the associated code file:

 

    public getProjects() {
        this.errorMessage = "";
        // Clear Projects Dropdown
        this.projectsDropdown = [];
        // Call the service -- to get Projects
        this._CustomVisionTrainingService.getProjects()
            .subscribe((projects) => {
                // Temp value to hold Projects because they cannot be 
                // assigned to the public property so deep inside the 
                // nested iteration
                var tempProjectsDropdown: SelectItem[] = [];
                // Loop through the returned Projects
                for (let project of projects) {
                    // Create a new SelectedItem
                    let newSelectedItem: SelectItem = {
                        label: project.name + ' - ' + project.description,
                        value: project
                    }
                    // Add Selected Item to the DropDown
                    tempProjectsDropdown.push(newSelectedItem);
                }
                // Add items to the DropDown
                this.projectsDropdown = tempProjectsDropdown;
                if (this.projectsDropdown[0].value !== 'undefined') {
                    // Set the selected option to the first option
                    this.selectedProject = this.projectsDropdown[0].value;
                    // Get the Tags for the Project
                    this.getTags();
                }
            },
            error => this.errorMessage = <any>error);
    }

 

(note: this method then calls the getTags method, to fill the Tags dropdown. This will be covered in the next section)

The getProjects method calls the following getProjects service method:

 

    // ** Get Projects **
    getProjects(): Observable<ICustomVisionProjectResponse[]> {
        var _Url = 'api/training/GetProjects';
        // Call the client side code
        return this._http.get(_Url)
            .map((response: Response) =>
                <ICustomVisionProjectResponse[]>response.json())
            .catch(this.handleError);
    }

 

This calls the following server side .Net Core method that calls the Microsoft Custom Vision Service API:

 

        // api/training/GetProjects
        [HttpGet("[action]")]
        #region public IActionResult GetProjects()
        public IActionResult GetProjects()
        {
            List<CustomVisionProjectResponse> colCustomVisionTrainingResponse =
                new List<CustomVisionProjectResponse>();
            // Create a HttpClient to make the request
            using (HttpClient client = new HttpClient())
            {
                // Set Training Key in the request headers 
                client.DefaultRequestHeaders.Add("Training-key", _TrainingKey);
                // Build the request to the Custom Vision API
                StringBuilder uri = new StringBuilder();
                uri.Append("https://southcentralus.api.cognitive.microsoft.com");
                uri.Append("/customvision/v2.0/Training");
                uri.Append("/projects?");
                // Make the request to the Custom Vision API
                HttpResponseMessage response = client.GetAsync(uri.ToString()).Result;
                // Get the response
                string ResponseContent = response.Content.ReadAsStringAsync().Result;
                // Convert the response to the CustomVisionTrainingResponse object
                colCustomVisionTrainingResponse =
                    JsonConvert.DeserializeObject<List<CustomVisionProjectResponse>>(ResponseContent);
            }
            // Return the Response to the Angular application
            return Ok(colCustomVisionTrainingResponse);
        }
        #endregion

 

Get Tags

The code that gets the Tags is similar to the code that gets the Projects. The difference is that the currently selected Project is used as a parameter because a set of Tags is always associated with a specific Project.

Markup:

 

        <label for="Tags">Tags</label>
        <br />
        <p-dropdown id="Tags"
                    appendTo="body"
                    [options]="tagsDropdown"
                    [(ngModel)]="selectedTag"
                    [style]="{'width':'250px'}"
                    (onChange)="onTagChange()"></p-dropdown>

 

Code file:

 

    public getTags() {
        this.errorMessage = "";
        // Clear TotalRecords
        this.totalRecords = 0;
        // Clear Tags Dropdown
        this.tagsDropdown = [];
        // Call the service -- to get Tags
        this._CustomVisionTrainingService.getTags(this.selectedProject)
            .subscribe((tags) => {
                // Temp value to hold Tags because they cannot be 
                // assigned to the public property so deep inside the 
                // nested iteration
                var tempTagsDropdown: SelectItem[] = [];
                // Loop through the returned Tags
                for (let tag of tags.tags) {
                    // Create a new SelectedItem
                    let newSelectedItem: SelectItem = {
                        label: tag.name,
                        value: tag
                    }
                    // Add Selected Item to the DropDown
                    tempTagsDropdown.push(newSelectedItem);
                }
                // Add items to the DropDown
                this.tagsDropdown = tempTagsDropdown;
                if (this.tagsDropdown[0].value !== 'undefined') {
                    // Set the selected option to the first option
                    this.selectedTag = this.tagsDropdown[0].value;
                    // Set defaults
                    this.currentPage = 1;
                    
                    // Get the Images for the Tag
                    this.getImages();
                }
            },
            error => this.errorMessage = <any>error);
    }

 

Service:

 

    // ** Get Tags **
    getTags(paramProject: ICustomVisionProjectResponse):
        Observable<ICustomVisionTagResponse> {
        var _Url = 'api/training/GetTags?projectId=' + paramProject.id;
        // Call the client side code
        return this._http.get(_Url)
            .map((response: Response) =>
                <ICustomVisionTagResponse>response.json())
            .catch(this.handleError);
    }

 

Server side code:

 

        // api/training/GetTags
        [HttpGet("[action]")]
        #region public IActionResult GetTags(string projectId)
        public IActionResult GetTags(string projectId)
        {
            List<CustomVisionTagResponse> objCustomVisionTagResponse =
                new List<CustomVisionTagResponse>();
            // Create a HttpClient to make the request
            using (HttpClient client = new HttpClient())
            {
                // Set Training Key in the request headers 
                client.DefaultRequestHeaders.Add("Training-key", _TrainingKey);
                // Build the request to the Custom Vision API
                StringBuilder uri = new StringBuilder();
                uri.Append("https://southcentralus.api.cognitive.microsoft.com");
                uri.Append("/customvision/v2.0/Training");
                uri.Append("/projects/" + projectId);
                uri.Append("/tags?");
                // Make the request to the Custom Vision API
                HttpResponseMessage response = client.GetAsync(uri.ToString()).Result;
                // Get the response
                string ResponseContent = response.Content.ReadAsStringAsync().Result;
                // Convert the response to the CustomVisionTagResponse object
                objCustomVisionTagResponse =
                    JsonConvert.DeserializeObject<List<CustomVisionTagResponse>>(ResponseContent);
            }
            // Return the Response to the Angular application
            return Ok(objCustomVisionTagResponse);
        }
        #endregion

 

Images

When a Project and a Tag are selected, the code to display the images associated with the selected Tag runs.

The following is the markup:

 

<p-dataGrid [value]="images"
            [paginator]="true"
            [rows]="8"
            [lazy]="true"
            [totalRecords]="totalRecords"
            (onLazyLoad)="loadData($event)">
    <p-header>
        List of Images
    </p-header>
    <ng-template let-image pTemplate="item">
        <div style="padding:3px" class="ui-g-12 ui-md-3">
            <p-panel [header]="image.id" [style]="{'text-align':'center'}">
                <a href="{{image.imageUri}}"
                   target="_blank">
                <img src="{{image.thumbnailUri}}" 
                     width="100" height="100" 
                     style="border:2px solid blue"></a>
            </p-panel>
        </div>
    </ng-template>
</p-dataGrid>

 

Note, that the PrimeNG dataGrid control uses Lazy Loading, meaning data is requested one page at a time.

The following shows the method, in the associated code file, that is called when the user requests each page of data:

 

    loadData(event) {
        // Set Current Page
        this.currentPage = ((event.first + 8) / 8);
        // Get the Images for the Tag
        this.getImages();
    }

 

The code above calls the getImages method, that gets the image data:

 

    getImages() {
        this.errorMessage = "";
        // Clear Images
        this.images = [];
        // Call the service -- to get Images
        this._CustomVisionTrainingService.getImages(
            this.selectedProject,
            this.selectedTag.id,
            this.currentPage.toString()).subscribe((images) => {
                // Temp value to hold Images because they cannot be 
                // assigned to the public property so deep inside the 
                // nested iteration
                var tempImags: ICustomVisionImageResponse[] = [];
                // Loop through the returned Images
                for (let image of images) {
                    // Add Selected Item to the list of images
                    tempImags.push(image);
                }
                // Add items to the Image list
                this.images = tempImags;
                // Set total records
                this.totalRecords = this.selectedTag.imageCount;
            },
            error => this.errorMessage = <any>error);
    }

 

This calls the following service method:

 

    // ** Get Images **
    getImages(
        paramProject: ICustomVisionProjectResponse,
        paramTag: string,
        paramPageNumber: string): Observable<ICustomVisionImageResponse[]> {
        var _Url = 'api/training/GetImages?'
            + 'projectId=' + paramProject.id
            + '&tagId=' + paramTag
            + '&pageNumber=' + paramPageNumber;
        // Call the client side code
        return this._http.get(_Url)
            .map((response: Response) =>
                <ICustomVisionImageResponse[]>response.json())
            .catch(this.handleError);
    }

 

This calls the following server side method:

 

        // api/training/GetImages
        [HttpGet("[action]")]
        #region public IActionResult GetImages(string projectId, string tagId, string pageNumber)
        public IActionResult GetImages(string projectId, string tagId, string pageNumber)
        {
            List<CustomVisionImageResponse> colCustomVisionImageResponse =
                new List<CustomVisionImageResponse>();
            if ((tagId != null) && (projectId != null))
            {
                // Create a HttpClient to make the request
                using (HttpClient client = new HttpClient())
                {
                    // Set Training Key in the request headers 
                    client.DefaultRequestHeaders.Add("Training-key", _TrainingKey);
                    // Build the request
                    string SkipNumber = ((Convert.ToInt32(pageNumber) - 1) * 8).ToString();
                    StringBuilder uri = new StringBuilder();
                    uri.Append("https://southcentralus.api.cognitive.microsoft.com");
                    uri.Append("/customvision/v2.0/Training");
                    uri.Append("/projects/" + projectId);
                    uri.Append("/images/tagged?");
                    uri.Append("tagIds=" + tagId);
                    uri.Append("&take=" + "8");
                    uri.Append("&skip=" + SkipNumber);
                    // Make the request to the Custom Vision API
                    HttpResponseMessage response = client.GetAsync(uri.ToString()).Result;
                    // Get the response
                    string ResponseContent = response.Content.ReadAsStringAsync().Result;
                    // Convert the response to the CustomVisionImageResponse object
                    colCustomVisionImageResponse =
                        JsonConvert.DeserializeObject<List<CustomVisionImageResponse>>(ResponseContent);
                }
            }
            // Return the Response to the Angular application
            return Ok(colCustomVisionImageResponse);
        }
        #endregion

 

Uploading

When a Project and a Tag are selected, you can upload images that will then be associated with the Tag.

The following is the markup for the PrimeNG upload control:

 

<p-fileUpload name="myfile[]"
              url="api/training"
              multiple="true"
              accept="image/*"
              maxFileSize="10000000"
              uploadLabel="Upload New Files"
              (onBeforeUpload)="onBeforeUploadHandler($event)"
              (onUpload)="onUpload($event)">
</p-fileUpload>

 

This has two associated methods in the code file, onBeforeUploadHandler to pass the selected Project and Tag to the server side code, and onUpload, that is called after the upload, to refresh the images in the dataGrid:

 

    // Upload
    public onBeforeUploadHandler(event) {
        // called before the file(s) are uploaded
        // Send the currently selected Project and Tag in the Header
        event.formData.append("selectedProject", this.selectedProject.id);
        event.formData.append("selectedTag", this.selectedTag.id);
    }
    public onUpload(event) {
        // This is called after the upload
        this.getImages();
    }

 

The following server side method handles the upload:

 

        // api/training
        [HttpPost]
        #region public IActionResult Index()
        public IActionResult Index()
        {
            if (!Request.HasFormContentType)
            {
                return BadRequest();
            }
            // Retrieve data from Form
            var form = Request.Form;
            // Retrieve Project and Tag
            string selectedProject = form["selectedProject"].First();
            string selectedTag = form["selectedTag"].First();
            if ((selectedProject != null) && (selectedTag != null))
            {
                // Process all Files
                foreach (var file in form.Files)
                {
                    // Process file
                    using (var readStream = file.OpenReadStream())
                    {
                        // Create a HttpClient to make the request
                        using (HttpClient client = new HttpClient())
                        {
                            // Set Training Key in the request headers 
                            client.DefaultRequestHeaders.Add("Training-key", _TrainingKey);
                            // Serialize Request
                            MultipartFormDataContent _multiPartContent =
                                new MultipartFormDataContent();
                            StreamContent _imageData =
                                new StreamContent(readStream);
                            _imageData.Headers.ContentType =
                                new MediaTypeHeaderValue("application/octet-stream");
                            ContentDispositionHeaderValue _contentDispositionHeaderValue =
                                new ContentDispositionHeaderValue("form-data");
                            _contentDispositionHeaderValue.Name = "imageData";
                            _contentDispositionHeaderValue.FileName = file.Name;
                            _imageData.Headers.ContentDisposition = _contentDispositionHeaderValue;
                            _multiPartContent.Add(_imageData, "imageData");
                            // Build the request to the Custom Vision API
                            StringBuilder uri = new StringBuilder();
                            uri.Append("https://southcentralus.api.cognitive.microsoft.com");
                            uri.Append("/customvision/v2.0/Training");
                            uri.Append("/projects/" + selectedProject);
                            uri.Append("/images?");
                            uri.Append("tagIds=" + selectedTag);
                            // Make the request to the Custom Vision Service API
                            HttpResponseMessage response =
                                client.PostAsync(uri.ToString(), _multiPartContent).Result;
                            // Get the response -- (Do nothing with it for now)
                            string ResponseContent = response.Content.ReadAsStringAsync().Result;
                        }
                    }
                }
            }
            return Ok();
        }
        #endregion

 

Train Project

After uploading new images, you need to Train the project to create a new predictive model.

The following is the markup for the button:

 

<button pButton type="button"
        (click)="TrainProject()"
        label="Train Project"
        icon="fa-plus-square"
        class="ui-button-success"></button>

 

This calls the following method:

 

    TrainProject() {
        // Call the service -- to train the project
        this._CustomVisionTrainingService.trainProject(this.selectedProject)
            .subscribe((response: ICustomVisionTrainResponse) => {
                alert(response.status);
            });
    }

 

This calls the following service method:

 

    // ** Train Project **
    trainProject(paramProject: ICustomVisionProjectResponse):
        Observable<ICustomVisionTrainResponse> {
        var _Url = 'api/training/TrainProject?projectId=' + paramProject.id;
        // Call the client side code
        return this._http.get(_Url)
            .map((response: Response) =>
                <ICustomVisionTrainResponse>response.json())
            .catch(this.handleError);
    }

 

This calls the following server side code:

 

        // api/training/TrainProject
        [HttpGet("[action]")]
        #region public IActionResult TrainProject(string projectId)
        public IActionResult TrainProject(string projectId)
        {
            CustomVisionTrainResponse objCustomVisionTrainResponse = new CustomVisionTrainResponse();
            // Create a HttpClient to make the request
            using (HttpClient client = new HttpClient())
            {
                HttpResponseMessage response;
                // Set Training Key in the request headers 
                client.DefaultRequestHeaders.Add("Training-key", _TrainingKey);
                // Build the request to the Custom Vision API
                StringBuilder uri = new StringBuilder();
                uri.Append("https://southcentralus.api.cognitive.microsoft.com");
                uri.Append("/customvision/v2.0/Training");
                uri.Append("/projects/" + projectId);
                uri.Append("/train?");
                // Request body
                byte[] byteData = Encoding.UTF8.GetBytes("{body}");
                // Make the request to the Custom Vision Service API
                using (var content = new ByteArrayContent(byteData))
                {
                    content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
                    response = client.PostAsync(uri.ToString(), content).Result;
                }
                if (response.StatusCode == System.Net.HttpStatusCode.BadRequest)
                {
                    objCustomVisionTrainResponse.Status = "No Training needed";
                }
                else
                {
                    // Get the response
                    string ResponseContent = response.Content.ReadAsStringAsync().Result;
                    // Convert the response to the CustomVisionTrainResponse object
                    objCustomVisionTrainResponse =
                        JsonConvert.DeserializeObject<CustomVisionTrainResponse>(ResponseContent);
                }
            }
            // Return a Response to the Angular application
            return Ok(objCustomVisionTrainResponse);
        }
        #endregion

 

Links

https://www.customvision.ai/

Custom Vision Documentation

Custom Vision Service API Reference

PrimeNG dataGrid

PrimeNG upload

Download

You can download the code from the Download page