Using Images, Cards, Carousels, and Buttons In The Microsoft Bot Framework

Sep 5

Written by:
9/5/2016 3:55 PM  RssIcon

image

Note: You can test out the sample Bot from this article in Skype using this link: https://join.skype.com/bot/d6a7af21-b56f-411f-b7cf-868d0fdbb1ba.

Implementing images in your Microsoft Bot Framework application adds a visual element to the interaction with your users.

You can easily pass a media attachment (image/audio/video/file) to a message using code such as this:

        activity.Attachments.Add(new Attachment()
        {
            ContentUrl = "http://aihelpwebsite.com/portals/0/Images/AIHelpWebsiteLogo_Large.png",
            ContentType = "image/png",
            Name = "AIHelpWebsiteLogo_Large.png"
        });

Depending on the channel used and the media type, a image will be displayed or a link will be displayed for the user to download the media element.

However, real power is provided when you use rich card attachments.

There are four card types:

Card Type Description Single or Carousel/List (multiple cards)

Hero Card

A big image with text

Single or Carousel/List

Thumbnail Card 

A small image with text

Single or Carousel/List

Receipt Card 

An invoice or receipt

Single

Sign-In Card 

A user sign-in form

Single

 

Hero Card

image

A Hero Card and a Thumbnail Card differ only in the size of their image and card.

Each consists of the following properties:

Property Description

Title

Title of the card

Text

Text to display on the card

Subtitle

A link for the title

Images

(actually only a single image)

Buttons

One or more buttons (in Skype only 5 buttons will display on a card. If you have more buttons, it will create two cards)

Tap

An action that is triggered when a user taps on the card (does not work in some Skype clients, so for Skype, use a hyperlink for the Subtitle property)

image

To demonstrate implementing a Hero Card, we will start with the code from the article: Introduction To Using Dialogs With The Microsoft Bot Framework (note: you will have to use Manage NuGet Packages in Visual Studio and update Microsoft.Bot.Builder to the latest stable version).

Also, we will make some small changes, such as making the number to guess from 1 to 5 rather than 1 to 10 so that it shows up well in Skype (because Skype will only display 5 buttons on a single card).

Open the project in Visual Studio.

image

Add a new folder and call it Images.

Add the NumberGuesserOpeningCard.png file, (that you can get from the code for this article on the Downloads page), to the Images folder.

Open the MessagesController.cs file (in the Controllers directory), and add the following using statement to the top of the file:

 

using System.Collections.Generic;

 

image

Next, locate the else if (message.Type == ActivityTypes.ContactRelationUpdate) section in the HandleSystemMessage method.

This method fires when the Bot is added to the Contact list of a user.

Replace all the code in the section with the following code:

 

        // Construct a base URL for Image
        // To allow it to be found wherever the application is deployed
        string strCurrentURL =
            this.Url.Request.RequestUri.AbsoluteUri.Replace(@"api/messages", "");
        // Create a reply message
        Activity replyToConversation = message.CreateReply();
        replyToConversation.Recipient = message.From;
        replyToConversation.Type = "message";
        replyToConversation.Attachments = new List<Attachment>();
        // AttachmentLayout options are list or carousel
        replyToConversation.AttachmentLayout = "carousel";
        #region Card One
        // Full URL to the image
        string strNumberGuesserOpeningCard =
            String.Format(@"{0}/{1}",
            strCurrentURL,
            "Images/NumberGuesserOpeningCard.png");
        // Create a CardImage and add our image
        List<CardImage> cardImages1 = new List<CardImage>();
        cardImages1.Add(new CardImage(url: strNumberGuesserOpeningCard));
        // Create a CardAction to make the HeroCard clickable
        // Note this does not work in some Skype clients
        CardAction btnAiHelpWebsite = new CardAction()
        {
            Type = "openUrl",
            Title = "AiHelpWebsite.com",
            Value = "http://AiHelpWebsite.com"
        };
        // Finally create the Hero Card
        // adding the image and the CardAction
        HeroCard plCard1 = new HeroCard()
        {
            Title = "Ai Help Website - Number Guesser",
            Subtitle = "Hi Welcome! - Guess a number between 1 and 5",
            Images = cardImages1,
            Tap = btnAiHelpWebsite
        };
        // Create an Attachment by calling the
        // ToAttachment() method of the Hero Card
        Attachment plAttachment1 = plCard1.ToAttachment();
        // Add the Attachment to the reply message
        replyToConversation.Attachments.Add(plAttachment1);
        #endregion
        // Create a ConnectorClient and use it to send the reply message
        var connector = 
            new ConnectorClient(new Uri(message.ServiceUrl));
        var reply = 
            connector.Conversations.SendToConversationAsync(replyToConversation);

 

 

image

Save the file and hit F5 to run the application.

image

Connect to the Bot using the Bot Framework Emulator (Windows).

(for information on using the Emulator, see Creating a Hello World! Bot Using The Microsoft Bot Framework)

image

In the Emulator, invoke the Contact Relation Update-add event.

image

The Hero Card will display.

Clicking on the card will take you to the AiHelpWebsite.com website.

 

Carousel

image

A carousel displays multiple cards (either Hero Cards or Thumbnail Cards) horizontally.

A List (this is the default layout for multiple cards), displays the cards vertically.

The image above shows what a carousel of two Hero Cards looks like in the Skype client.

image

To implement this, first, add the AiLogo_smallSquare.png file, (that you can get from the code for this article on the Downloads page), to the Images folder.

Next, add the following code to the HandleSystemMessage method (above the var connector = new ConnectorClient(new Uri(message.ServiceUrl)); line):

 

        #region Card Two
        string strAiLogo_smallSquare =
            String.Format(@"{0}/{1}",
            strCurrentURL,
            "Images/AiLogo_smallSquare.png");
        List<CardImage> cardImages2 = new List<CardImage>();
        cardImages2.Add(new CardImage(url: strAiLogo_smallSquare));
        // CardAction to make the HeroCard clickable
        CardAction btnTutorial = new CardAction()
        {
            Type = "openUrl",
            Title = "http://bit.ly/2bRyJMj",
            Value = "http://bit.ly/2bRyJMj"
        };
        HeroCard plCard2 = new HeroCard()
        {
            Title = "Based on the Using Dialogs Tutorial",
            Subtitle = "http://bit.ly/2bRyJMj",
            Images = cardImages2,
            Tap = btnTutorial
        };
        Attachment plAttachment2 = plCard2.ToAttachment();
        replyToConversation.Attachments.Add(plAttachment2);
        #endregion

 

Save the file, run the application, and connect to it in the Emulator.

In the Emulator, invoke the Contact Relation Update-add event.

image

The first card will show, but you will now see a slider.

Slide it to the right.

image

You will see the second Hero Card.

In Skype, the Subtitle property will display as a hyperlink and be clickable.

 

Rich Card Attachments In Dialogs

There are additional challenges when implementing rich card attachments in Dialogs.

Primarily, the biggest challenge Is that, inside the Dialog class, you do not have access to the base URL of the application (to determine where your images are).

We can address this by using the Bot State Service.

Open the the MessagesController.cs file:

image 

Add the following code to the top of the Post method:

 

        #region Set CurrentBaseURL
        // Get the base URL that this service is running at
        // This is used to show images
        string CurrentBaseURL =
                this.Url.Request.RequestUri.AbsoluteUri.Replace(@"api/messages", "");
        // Create an instance of BotData to store data
        BotData objBotData = new BotData();
        // Instantiate a StateClient to save BotData            
        StateClient stateClient = activity.GetStateClient();
        // Use stateClient to get current userData
        BotData userData = await stateClient.BotState.GetUserDataAsync(
            activity.ChannelId, activity.From.Id);
        // Update userData by setting CurrentBaseURL and Recipient
        userData.SetProperty<string>("CurrentBaseURL", CurrentBaseURL);
        // Save changes to userData
        await stateClient.BotState.SetUserDataAsync(
            activity.ChannelId, activity.From.Id, userData);
        #endregion

 

This will store the CurrentBaseURL and the Recipient using the Bot State Service so that the values can be retrieved in the Dialog class.

image

Open the NumberGuesserDialog.cs file.

(this file contains the NumberGuesserDialog class that is instantiated in the Post method of the MessagesController class. You can read more about it here: Introduction To Using Dialogs With The Microsoft Bot Framework)

Add this above the namespace:

 

    using System.Collections.Generic;

 

Add this to the top of the class:

    string strBaseURL;


This will provide a global variable (that will be automatically serialized and persisted between calls) to store the Base URL.

Add the following code to the top of the MessageReceivedAsync method:

 

    // Set BaseURL
    context.UserData.TryGetValue<string>(
        "CurrentBaseURL", out strBaseURL);

This will retrieve the value of CurrentBaseURL that was stored in Bot State Service in the MessagesController class and store it in the strBaseURL global variable. 

Add the following Utility method to the class. This method will create 5 CardAction buttons that will be attached to the Hero Card that we will create later:

 

    #region private static List<CardAction> CreateButtons()
    private static List<CardAction> CreateButtons()
    {
        // Create 5 CardAction buttons 
        // and return to the calling method 
        List<CardAction> cardButtons = new List<CardAction>();
        for (int i = 1; i < 6; i++)
        {
            string CurrentNumber = Convert.ToString(i);
            CardAction CardButton = new CardAction()
            {
                Type = "imBack",
                Title = CurrentNumber,
                Value = CurrentNumber
            };
            cardButtons.Add(CardButton);
        }
        return cardButtons;
    }
    #endregion

 

The buttons are set to the TypeimBack” that makes the button (CardAction) simply post the value (in this case the button number) back to the Bot.

Create The Hero Card In The Dialog Class

image

We will now display a Hero Card with 5 buttons in the Dialog class.

First add the NumberGuesserCard.png file, (that you can get from the code for this article on the Downloads page), to the Images folder.

Next, find the following code in the MessageReceivedAsync method:

 

    // See if a number was passed
    if (!int.TryParse(message.Text, out intGuessedNumber))
    {
        // A number was not passed                
        await context.PostAsync("Hi Welcome! - Guess a number between 1 and 10");
        context.Wait(MessageReceivedAsync);
    }

 

Replace it with the following code:

 

        // See if a number was passed
        if (!int.TryParse(message.Text, out intGuessedNumber))
        {
            // A number was not passed  
            // Create a reply Activity
            Activity replyToConversation = (Activity)context.MakeMessage();
            replyToConversation.Recipient = replyToConversation.Recipient;
            replyToConversation.Type = "message";
            string strNumberGuesserCard =
                String.Format(@"{0}/{1}",
                strBaseURL,
                "Images/NumberGuesserCard.png");
            List<CardImage> cardImages = new List<CardImage>();
            cardImages.Add(new CardImage(url: strNumberGuesserCard));
            // Create the Buttons
            // Call the CreateButtons utility method
            List<CardAction> cardButtons = CreateButtons();
            // Create the Hero Card
            // Set the image and the buttons
            HeroCard plCard = new HeroCard()
            {
                Images = cardImages,
                Buttons = cardButtons,
            };
            // Create an Attachment by calling the
            // ToAttachment() method of the Hero Card                
            Attachment plAttachment = plCard.ToAttachment();
            // Attach the Attachment to the reply
            replyToConversation.Attachments.Add(plAttachment);
            // set the AttachmentLayout as 'list'
            replyToConversation.AttachmentLayout = "list";
            // Send the reply
            await context.PostAsync(replyToConversation);
            context.Wait(MessageReceivedAsync);
        }

 

Save the file, hit F5 to run the project, and connect to it in the Emulator.

image

The Bot will now display a image and buttons.

We can either click one of the buttons, or enter a number, to play the game.

The following image displays what it looks like in Skype:

image

 

Creating A Re-Usable Hero Card

There are a few more places we need to display the Hero Card with the buttons.

We will want to create a re-usable Hero Card.

This Hero Card will not have an image, only the buttons.

First, add the following utility method to the Dialog class:

 

    #region private static Activity ShowButtons(IDialogContext context, string strText)
    private static Activity ShowButtons(IDialogContext context, string strText)
    {
        // Create a reply Activity
        Activity replyToConversation = (Activity)context.MakeMessage();
        replyToConversation.Text = strText;
        replyToConversation.Recipient = replyToConversation.Recipient;
        replyToConversation.Type = "message";
        // Call the CreateButtons utility method 
        // that will create 5 buttons to put on the Here Card
        List<CardAction> cardButtons = CreateButtons();
        // Create a Hero Card and add the buttons 
        HeroCard plCard = new HeroCard()
        {
            Buttons = cardButtons
        };
        // Create an Attachment
        // set the AttachmentLayout as 'list'
        Attachment plAttachment = plCard.ToAttachment();
        replyToConversation.Attachments.Add(plAttachment);
        replyToConversation.AttachmentLayout = "list";
        // Return the reply to the calling method
        return replyToConversation;
    }
    #endregion

 

Note that it calls the CreateButtons utility method that we created earlier.

Now, locate the following code:

 

        // A number was passed
        // See if it was the correct number
        if (intGuessedNumber != this.intNumberToGuess)
        {
            // The number was not correct
            this.intAttempts++;
            await context.PostAsync("Not correct. Guess again.");
            context.Wait(MessageReceivedAsync);
        }

 

Replace it with:

 

        // A number was passed
        // See if it was the correct number
        if (intGuessedNumber != this.intNumberToGuess)
        {
            // The number was not correct
            this.intAttempts++;
            // Create a response
            // This time call the ** ShowButtons ** method
            Activity replyToConversation =
                ShowButtons(context, "Not correct. Guess again.");
            await context.PostAsync(replyToConversation);
            context.Wait(MessageReceivedAsync);
        }

 

The code will now call the ShowButtons method that will return the Hero Card.

Finally, find the following code:

 

    // Start a new Game
    await context.PostAsync("Hi Welcome! - Guess a number between 1 and 10");
    context.Wait(MessageReceivedAsync);

 

Replace it with:

 

        // Start a new Game
        // Create a response
        // This time call the ** ShowButtons ** method
        Activity replyToConversation =
            ShowButtons(context, "Hi Welcome! - Guess a number between 1 and 5");
        await context.PostAsync(replyToConversation);
        context.Wait(MessageReceivedAsync);

 

image

The Bot will now display the Hero Card, with only the buttons, in each situation where we called the utility method.

 

Ai Help Website Links

Creating a Hello World! Bot Using The Microsoft Bot Framework

Introduction To Using Dialogs With The Microsoft Bot Framework

Creating a Skype Bot Using The Microsoft Bot Framework

Microsoft Links

Microsoft Bot Framework

Bot Builder (GitHub)

Bot Framework Forum (stack overflow)

Download

You can download the code from the Download page