Serverless Architecture - Microsoft Cloud Workshop

Microsoft Cloud Workshop Microsoft Cloud Workshop on Mar 01, 2019

In this hand-on lab, you will be challenged to implement an end-to-end scenario using a supplied sample that is based on Microsoft Azure Functions, Azure Cosmos DB, Event Grid, and related services. The scenario will include implementing compute, storage, workflows, and monitoring, using various components of Microsoft Azure. The hands-on lab can be implemented on your own, but it is highly recommended to pair up with other members at the lab to model a real-world experience and to allow each member to share their expertise for the overall solution.
At the end of the hands-on-lab, you will have confidence in designing, developing, and monitoring a serverless solution that is resilient, scalable, and cost-effective.

Before the Hands-on Lab

Serverless architecture
Before the hands-on lab setup guide
March 2019

Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

The names of manufacturers, products, or URLs are provided for informational purposes only and Microsoft makes no representations and warranties, either expressed, implied, or statutory, regarding these manufacturers or the use of the products with any Microsoft technologies. The inclusion of a manufacturer or product does not imply endorsement of Microsoft of the manufacturer or product. Links may be provided to third party sites. Such sites are not under the control of Microsoft and Microsoft is not responsible for the contents of any linked site or any link contained in a linked site, or any changes or updates to such sites. Microsoft is not responsible for webcasting or any other form of transmission received from any linked site. Microsoft is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement of Microsoft of the site or the products contained therein.

© 2019 Microsoft Corporation. All rights reserved.

Contents

Serverless architecture before the hands-on lab setup guide

Requirements

Before the hands-on lab

Duration: 10 minutes

In this exercise, you will set up your environment you will use for the rest of the exercises. This will involve downloading the sample application and creating your Azure resource group for the lab.

Task 1: Create a new Azure Resource group

  1. Open the Azure Portal.

  2. Within the Azure Management Portal, open the Resource groups tile and select Add.

    In the menu of the Azure Portal, Resource groups is selected. In the Resource Groups blade, the Add button is selected.

  3. Specify the name of the resource group as ServerlessArchitecture, and choose the Azure region to which you want to deploy the lab. This resource group will be used throughout the rest of the lab. Select Review + Create. This will show you a summary of changes. Select Create to create the resource group.

    In the Resource group blade, the Resource group name field displays ServerlessArchtecture.

Task 2: Set up a development environment

If you do not have a machine with Visual Studio Community 2017 (or greater) and the Azure development workload, complete this task.

  1. Create a virtual machine (VM) in Azure using the Visual Studio Community 2017 on Windows Server 2016 (x64) image. A Windows 10 image will work as well. Note: Your Azure subscription must include MSDN offers to create a VM with Visual Studio pre-loaded.

    In Azure Portal, in the search field, Visual Studio Community 2017 on Windows Server 2016 (x64) is typed. Under Results, Visual Studio Community 2017 on Windows Server 2016 (x64) is selected.

    1. Click +Create a resource.

    2. Type Visual Studio.

    3. Select the Visual Studio Community 2017 (latest release) on Windows Server 2016 (x64).

    4. Click Create.

    5. Select your subscription and recently created resource group

    6. For Virtual machine name, type MainVM, or a different name that is unique.

    7. Leave availability option as No infrastructure redundancy required

    8. Ensure the image is Visual Studio Community 2017 (latest release) on Windows Server 2016 (x64)

    9. Select your VM size.

    Note: It is highly recommended to use a DS2 or D2 instance size for this VM.

    1. For username, type demouser

    2. For password, type Password.1!!

    3. Click Allow selected ports.

    4. For the inbound ports, select RDP (3389).

    5. Click Review + create.

    6. Click Create.

Task 3: Disable IE Enhanced Security

Note: Sometimes this image has IE ESC disabled. Sometimes it does not.

  1. Login to the newly created VM using RDP and the username and password you supplied earlier.

  2. After the VM loads, the Server Manager should open.

  3. Select Local Server.

    Local Server is selected from the Server Manager menu.

  4. On the side of the pane, for IE Enhanced Security Configuration, if it displays On select it

    Screenshot of IE Enhanced Security Configuration, which is set to On.

    • Change to Off for Administrators and select OK.

    In the Internet Explorer Enhanced Security Configuration dialog box, under Administrators, the Off button is selected.

Task 4: Install Google Chrome

Note: Some aspects of this lab require the use of Google Chrome. You may find yourself blocked if using Internet Explorer later in the lab.

  1. Launch Internet Explorer and download Google Chrome.

  2. Follow the setup instructions and make sure you can run Chrome to navigate to any webpage.

Note: Chrome is needed for one of the labs as Internet Explorer is not supported for some specific activities.

Task 5: Validate connectivity to Azure

  1. From within the virtual machine, launch Visual Studio and validate that you can log in with your Microsoft Account when prompted.

  2. Verify your Visual Studio version is 15.4.0 or higher.

    1. Click Help in the menu, then select About Microsoft Visual Studio.

    2. If the version is not 15.4.0, you will need to update it. Click OK, then click View in the menu. Select Notifications, you should see an entry for Visual Studio Update is available*. Select it and then click Update to update your instance.

  3. To validate connectivity to your Azure subscription, open Server Explorer from the View menu, and ensure that you can connect to your Azure subscription.

    In Server Explorer, Azure is selected, and its right-click menu displays with options to Refresh, Connect to Microsoft Azure Subscription, Manage and Filter Subscriptions, or Open the Getting Started Page.

Task 6: Download and explore the TollBooth starter solution

  1. From your LabVM, download the starter files by downloading a .zip copy of the Cosmos DB real-time advanced analytics GitHub repo.

  2. In a web browser, navigate to the MCW Serverless architecture repo.

  3. On the repo page, select Clone or download, then select Download ZIP.

    Download .zip containing the repository

  4. Unzip the contents to the folder C:\ServerlessMCW\.

  5. Navigate to C:\ServerlessMCW\MCW-Serverless-architecture-master\Hands-on lab\starter

  6. From the TollBooth folder under Hackathon, open the Visual Studio Solution file: TollBooth.sln. Notice the solution contains the following projects:

  • TollBooth
  • UploadImages

Note: The UploadImages project is used for uploading a handful of car photos for testing scalability of the serverless architecture.

  1. Switch to windows explorer, navigate back to the starter subfolder and open the license plates subfolder. It contains sample license plate photos used for testing out the solution. One of the photos is guaranteed to fail OCR processing, which is meant to show how the workload is designed to handle such failures. The copyfrom folder is used by the UploadImages project as a basis for the 1,000 photo upload option for testing scalability.

You should follow all steps provided before performing the Hands-on lab.

Hands-on Lab Guide

Serverless architecture
Hands-on lab step-by-step
March 2019

Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

The names of manufacturers, products, or URLs are provided for informational purposes only and Microsoft makes no representations and warranties, either expressed, implied, or statutory, regarding these manufacturers or the use of the products with any Microsoft technologies. The inclusion of a manufacturer or product does not imply endorsement of Microsoft of the manufacturer or product. Links may be provided to third party sites. Such sites are not under the control of Microsoft and Microsoft is not responsible for the contents of any linked site or any link contained in a linked site, or any changes or updates to such sites. Microsoft is not responsible for webcasting or any other form of transmission received from any linked site. Microsoft is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement of Microsoft of the site or the products contained therein.

© 2019 Microsoft Corporation. All rights reserved.

Microsoft and the trademarks listed at https://www.microsoft.com/legal/intellectualproperty/Trademarks/Usage/General.aspx are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners.

Contents

Serverles architecture hands-on lab step-by-step

Abstract and learning objectives

In this hand-on lab, you will be challenged to implement an end-to-end scenario using a supplied sample that is based on Microsoft Azure Functions, Azure Cosmos DB, Event Grid, and related services. The scenario will include implementing compute, storage, workflows, and monitoring, using various components of Microsoft Azure. The hands-on lab can be implemented on your own, but it is highly recommended to pair up with other members at the lab to model a real-world experience and to allow each member to share their expertise for the overall solution.

At the end of the hands-on-lab, you will have confidence in designing, developing, and monitoring a serverless solution that is resilient, scalable, and cost-effective.

Overview

Contoso Ltd. is rapidly expanding their toll booth management business to operate in a much larger area. As this is not their primary business, which is online payment services, they are struggling with scaling up to meet the upcoming demand to extract license plate information from a large number of new tollbooths, using photos of vehicles uploaded to cloud storage. Currently, they have a manual process where they send batches of photos to a 3rd-party who manually transcodes the license plates to CSV files that they send back to Contoso to upload to their online processing system. They want to automate this process in a way that is cost effective and scalable. They believe serverless is the best route for them, but do not have the expertise to build the solution.

Solution architecture

Below is a diagram of the solution architecture you will build in this lab. Please study this carefully, so you understand the whole of the solution as you are working on the various components.

The Solution diagram is described in the text following this diagram.

The solution begins with vehicle photos being uploaded to an Azure Storage blobs container, as they are captured. A blob storage trigger fires on each image upload, executing the photo processing Azure Function endpoint (on the side of the diagram), which in turn sends the photo to the Cognitive Services Computer Vision API OCR service to extract the license plate data. If processing was successful and the license plate number was returned, the function submits a new Event Grid event, along with the data, to an Event Grid topic with an event type called "savePlateData". However, if the processing was unsuccessful, the function submits an Event Grid event to the topic with an event type called "queuePlateForManualCheckup". Two separate functions are configured to trigger when new events are added to the Event Grid topic, each filtering on a specific event type, both saving the relevant data to the appropriate Azure Cosmos DB collection for the outcome, using the Cosmos DB output binding. A Logic App that runs on a 15-minute interval executes an Azure Function via its HTTP trigger, which is responsible for obtaining new license plate data from Cosmos DB and exporting it to a new CSV file saved to Blob storage. If no new license plate records are found to export, the Logic App sends an email notification to the Customer Service department via their Office 365 subscription. Application Insights is used to monitor all of the Azure Functions in real-time as data is being processed through the serverless architecture. This real-time monitoring allows you to observe dynamic scaling first-hand and configure alerts when certain events take place.

Requirements

Exercise 1: Azure data, storage, and serverless environment setup

Duration: 30 minutes

You must provision a few resources in Azure before you start developing the solution. Ensure all resources use the same resource group for easier cleanup.

In this exercise, you will provision a blob storage account using the Hot tier, and create two containers within to store uploaded photos and exported CSV files. You will then provision two Function Apps instances, one you will deploy from Visual Studio, and the other you will manage using the Azure portal. Next, you will create a new Event Grid topic. After that, you will create an Azure Cosmos DB account with two collections. Finally, you will provision a new Cognitive Services Computer Vision API service for applying object character recognition (OCR) on the license plates.

Help references

Description Links
Creating a storage account (blob hot tier) https://docs.microsoft.com/azure/storage/common/storage-create-storage-account?toc=%2fazure%2fstorage%2fblobs%2ftoc.json%23create-a-storage-account
Creating a function app https://docs.microsoft.com/azure/azure-functions/functions-create-function-app-portal
Concepts in Event Grid https://docs.microsoft.com/azure/event-grid/concepts
Creating an Azure Cosmos DB account https://docs.microsoft.com/azure/cosmos-db/manage-account

Task 1: Provision the storage account

  1. Using a new tab or instance of your browser, navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then select Storage, Storage account.

    In the menu pane of Azure Portal, Create a resource is selected. Under Azure Marketplace, Storage is selected, and under Featured, Storage account - blob, file, table, queue is selected.

  3. On the Create storage account blade, specify the following configuration options:

    1. For Resource group, select the Use existing radio button, and specify the ServerlessArchitecture resource group.

    2. Name: enter a unique value for the storage account such as tollboothstorageINIT (ensure the green check mark appears).

    3. Ensure the Location is the same region as the resource group.

    4. For performance, ensure Standard is selected.

    5. For account kind, select Blobstorage.

    6. Select the Hot access tier.

    7. For replication, select Locally-redundant storage (LRS).

    Fields in the Create storage account blade are set to the previously defined values.

  4. Select Review + create, then select Create.

  5. After the storage account has completed provisioning, open the storage account by opening the ServerlessArchitecture resource group, and then selecting the storage account name.

    In the ServerlessArchitecture blade, Overview is selected. Under Name, the tollbooth storage account is selected.

  6. On the Storage account blade, select Access Keys, under Settings in the menu. Then on the Access keys blade, select the Click to copy button for key1 connection string.

    In the Storage account blade, under Settings, Access keys is selected. Under Default keys, the copy button next to the key1 connection string is selected.

  7. Paste the value into a text editor, such as Notepad, for later reference.

  8. Select Blobs under Blob Service in the menu. Then select the + Container button to add a new container. In the Name field, enter images, select Private (no anonymous access) for the public access level, then select OK to save.

    In the Storage blade, under Settings, Containers is selected. In the Containers blade, the + (add icon) Container button is selected. Below, the Name field displays images, and the Public access level is set to Private (no anonymous access).

  9. Repeat these steps to create a container named export.

    In the Storage blade, under Settings, Containers is selected. In the Containers blade, the + (add icon) Container button is selected. Below, the Name field displays export, and the Public access level is set to Private (no anonymous access).

Task 2: Provision the Function Apps

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then enter function into the search box on top. Select Function App from the results.

    In the menu pane of the Azure Portal, the New button is selected. Function is typed in the search field, and Function App is selected from the search results.

  3. Select the Create button on the Function App overview blade.

  4. On the Create Function App blade, specify the following configuration options:

    1. Name: Unique value for the App name (ensure the green check mark appears). Provide a name similar to TollBoothFunctionApp.

    2. Specify the Resource Group ServerlessArchitecture.

    3. For hosting plan, select the Consumption Plan.

    4. Select the same location as your Resource Group.

    5. For Runtime stack, select .NET.

    6. Leave the storage option as create new.

    7. Ensure Disabled is selected for Application Insights (we'll add this later).

    Fields in the Function App blade are set to the previously defined settings.

  5. Select Create.

    Screenshot of the Create button.

  6. Repeat steps 1-3 to create a second Function App.

  7. On the Create Function App blade, specify the following configuration options:

    1. Name: Unique value for the App name (ensure the green check mark appears). Provide a name similar to TollBoothEvents.

    2. Specify the Resource Group ServerlessArchitecture.

    3. For hosting plan, select the Consumption Plan.

    4. Select the same location as your Resource Group.

    5. For Runtime stack, select Javascript.

    6. Leave the storage option as create new.

    7. Ensure Disabled is selected for Application Insights (we'll add this later).

    Fields in the Function App blade are set to the previously defined settings.

  8. Select Create.

    Screenshot of the Create button.

Task 3: Provision the Event Grid topic

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then enter event grid into the search box on top. Select Event Grid Topic from the results.

    In the menu pane of the Azure Portal, the New button is selected. Event grid is typed in the search field, and Event Grid Topic is selected from the search results.

  3. Select the Create button on the Event Grid Topic overview blade.

  4. On the Create Topic blade, specify the following configuration options:

    1. Name: Unique value for the App name such as TollboothEventGrid (ensure the green check mark appears).

    2. Specify the Resource Group ServerlessArchitecture.

    3. Select the same location as your Resource Group.

    4. Leave the schema as Event Grid Schema.

    In the Create Topic blade, the Name field is TollBoothTopic, Resource Group is using existing ServerlessArchitecture, and the location is West US 2.

  5. Select Create.

  6. After the Event Grid topic has completed provisioning, open the account by opening the ServerlessArchitecture resource group, and then selecting the Event Grid topic name.

  7. Select Overview in the menu, and then copy the Topic Endpoint value.

    In the TollBoothTopic blade, Overview is selected, and the copy button next to the Topic Endpoint is called out.

  8. Select Access Keys under Settings in the menu.

  9. Within the Access Keys blade, copy the Key 1 value.

    In the TollBoothTopic - Access keys, under Settings, Access keys is selected. The copy button next to the Key 1 access key is also selected.

  10. Paste the values into a text editor, such as Notepad, for later reference.

Task 4: Provision the Azure Cosmos DB account

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, select Databases then select Azure Cosmos DB.

    In Azure Portal, in the menu, New is selected. Under Azure marketplace, Databases is selected, and under Featured, Azure Cosmos DB is selected.

  3. On the Create new Azure Cosmos DB account blade, specify the following configuration options:

    1. Specify the Resource Group ServerlessArchitecture.

    2. For Account Name, type a unique value for the App name such as ToolboothDB (ensure the green check mark appears).

    3. Select the Core (SQL) API.

    4. Select the same location as your Resource Group if available. Otherwise, select the next closest region.

    5. Ensure Disable geo-redundancy is selected.

    6. Ensure Disable multi-region writes is selected.

    Fields in the Azure Cosmos DB blade are set to the previously defined settings.

  4. Select Review + create, then select Create

  5. After the Azure Cosmos DB account has completed provisioning, open the account by opening the ServerlessArchitecture resource group, and then selecting the Azure Cosmos DB account name.

  6. Select Browse in the blade menu, then select + Add Collection.

    In the Tollbooths blade, Overview is selected, and the Add Collection button is selected.

  7. On the Add Collection blade, specify the following configuration options:

    1. Enter LicensePlates for the Database id.

    2. Leave Provision database throughput unchecked.

    3. Enter Processed for the Collection id.

    4. Partition key: /licensePlateText

    5. Throughput: 5000

    In the Add Collection blade, fields are set to the previously defined settings.

  1. Select OK.

  2. Select New Collection to add another collection.

  3. On the Add Collection blade, specify the following configuration options:

    1. For Database id, choose Use existing and select LicensePlates.

    2. For Collection id, enter NeedsManualReview.

    3. Partition key: /fileName

    4. Throughput: 5000

    In the Add Collection blade, fields are set to the previously defined values.

  4. Select OK.

  5. Select Keys under Settings in the menu.

  6. Underneath the Read-write Keys tab within the Keys blade, copy the URI and Primary Key values.

    In the tollbooth - Keys blade, under Settings, Keys is selected. On the Read-write Keys tab, the copy buttons for the URL and Primary Key fields are selected.

  7. Paste the values into a text editor, such as Notepad, for later reference.

Task 5: Provision the Computer Vision API service

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then enter computer vision into the search box on top. Select Computer Vision from the results.

    In the Azure Portal, Computer vision is typed in the search box.

  3. Select the Create button on the Computer Vision API overview blade.

  4. On the Create Computer Vision API blade, specify the following configuration options:

    1. Name: Unique value for the App name such as tollboothvisionINIT (ensure the green check mark appears).

    2. Select the same location as your Resource Group.

    3. For pricing tier, select S1 (10 Calls per second)

    4. Specify the Resource Group ServerlessArchitecture.

    In the Create blade, fields are set to the previously defined settings.

  5. Select Create.

    Screenshot of the Create button.

  6. After the Computer Vision API has completed provisioning, open the service by opening the ServerlessArchitecture resource group, and then selecting the Computer Vision API service name.

  7. Select Overview in the menu, and then copy the Endpoint value.

    In the TollboothOCR blade, in the pane, Overview is selected. In the  pane, the copy button next to the Endpoint URL is selected.

  8. Under Resource Management, select Keys.

  9. Within the Keys blade, copy the Key 1 value.

    In the TollboothOCR - Keys blade, under Resource Management, Keys is selected. The Copy button next to the Key 1 value is selected.

  10. Paste the values into a text editor, such as Notepad, for later reference.

Exercise 2: Develop and publish the photo processing and data export functions

Duration: 45 minutes

Use Visual Studio 2017 and its integrated Azure Functions tooling to develop and debug the functions locally, and then publish them to Azure. The starter project solution, TollBooths, contains most of the code needed. You will add in the missing code before deploying to Azure.

Help references

Description Links
Code and test Azure Functions locally https://docs.microsoft.com/azure/azure-functions/functions-run-local

Task 1: Configure application settings

In this task, you will apply application settings using the Microsoft Azure Portal. You will then add the application settings to the TollBooth Starter Project.

  1. Using a new tab or instance of your browser navigate to the Azure Management portal, http://portal.azure.com.

  2. Open the ServerlessArchitecture resource group, and then select the Azure Function App you created whose name ends with FunctionApp. This is the one you created using the .NET runtime stack. If you did not use this naming convention, that's fine. Just be sure to make note of the name so you can distinguish it from the Function App you will be developing using the portal later on.

    In the ServerlessArchtecture resource group, TollBoothFunctionApp is selected.

  3. Select Application settings on the Overview pane.

    In the TollBoothFunctionApp blade, under Configured features, Application settings is selected.

  4. Scroll down to the Application settings section. Use the + Add new setting link to create the following additional Key/Value pairs (the key names must exactly match those found in the table below):

Application Key Value
computerVisionApiUrl Computer Vision API endpoint you copied earlier. Append vision/v2.0/ocr to the end. Example: https://westus.api.cognitive.microsoft.com/vision/v2.0/ocr
computerVisionApiKey Computer Vision API key
eventGridTopicEndpoint Event Grid Topic endpoint
eventGridTopicKey Event Grid Topic access key
cosmosDBEndPointUrl Cosmos DB URI
cosmosDBAuthorizationKey Cosmos DB Primary Key
cosmosDBDatabaseId Cosmos DB database id (LicensePlates)
cosmosDBCollectionId Cosmos DB processed collection id (Processed)
exportCsvContainerName Blob storage CSV export container name (export)
blobStorageConnection Blob storage connection string

In the Application Settings section, the previously mentioned key / value pairs are called out with a purple line.

  1. Select Save.

    Screenshot of the Save icon.

  2. Optional steps, only if you wish to debug the functions locally on your development machine:

    1. Update the local.settings.json file with the same values.

    2. Update the AzureWebJobsStorage and AzureWebJobsDashboard values in local.settings.json with those found under Application settings for the Function App. Save your changes.

Task 2: Finish the ProcessImage function

There are a few components within the starter project that must be completed, marked as TODO in the code. The first set of TODO items we will address are in the ProcessImage function, the FindLicensePlateText class that calls the Computer Vision API, and finally the SendToEventGrid.cs class, which is responsible for sending processing results to the Event Grid topic you created earlier.

Note: Do NOT update the version of any NuGet package. This solution is built to function with the NuGet package versions currently defined within. Updating these packages to newer versions could cause unexpected results.

  1. Navigate to the TollBooth project using the Solution Explorer of Visual Studio.

  2. From the Visual Studio View menu, select Task List.

    The Visual Studio Menu displays, with View and Task List selected.

  3. There you will see a list of TODO tasks, where each task represents one line of code that needs to be completed.

    A list of TODO tasks, including their description, project, file, and line number display.

  4. Open ProcessImage.cs. Notice that the Run method is decorated with the FunctionName attribute, which sets the name of the Azure Function to "ProcessImage". This is triggered by HTTP requests sent to it from the Event Grid service. You tell Event Grid that you want to get these notifications at your function's URL by creating an event subscription, which you will do in a later task, in which you subscribe to blob-created events. The function's trigger watches for new blobs being added to the images container of the storage account that was created in Exercise 1. The data passed to the function from the Event Grid notification includes the URL of the blob. That URL is in turn passed to the input binding to obtain the uploaded image from Blob storage.

  5. The following code represents the completed task in ProcessImage.cs:

// **TODO 1: Set the licensePlateText value by awaiting a new FindLicensePlateText.GetLicensePlate method.**

licensePlateText = await new FindLicensePlateText(log, _client).GetLicensePlate(licensePlateImage);
  1. Open FindLicensePlateText.cs. This class is responsible for contacting the Computer Vision API to find and extract the license plate text from the photo, using OCR. Notice that this class also shows how you can implement a resilience pattern using Polly, an open source .NET library that helps you handle transient errors. This is useful for ensuring that you do not overload downstream services, in this case, the Computer Vision API. This will be demonstrated later on when visualizing the Function's scalability.

  2. The following code represents the completed task in FindLicensePlateText.cs:

// TODO 2: Populate the below two variables with the correct AppSettings properties.
var uriBase = Environment.GetEnvironmentVariable("computerVisionApiUrl");
var apiKey = Environment.GetEnvironmentVariable("computerVisionApiKey");
  1. Open SendToEventGrid.cs. This class is responsible for sending an Event to the Event Grid topic, including the event type and license plate data. Event listeners will use the event type to filter and act on the events they need to process. Make note of the event types defined here (the first parameter passed into the Send method), as they will be used later on when creating new functions in the second Function App you provisioned earlier.

  2. The following code represents the completed tasks in SendToEventGrid.cs:

// TODO 3: Modify send method to include the proper eventType name value for saving plate data.
await Send("savePlateData", "TollBooth/CustomerService", data);

// TODO 4: Modify send method to include the proper eventType name value for queuing plate for manual review.
await Send("queuePlateForManualCheckup", "TollBooth/CustomerService", data);

TODOs 5, 6, and 7 will be completed in later steps of the guide.

Task 3: Publish the Function App from Visual Studio

In this task, you will publish the Function App from the starter project in Visual Studio to the existing Function App you provisioned in Azure.

  1. Navigate to the TollBooth project using the Solution Explorer of Visual Studio.

  2. Right-click the TollBooth project and select Publish from the context menu.

    In Solution Explorer, TollBooth is selected, and in its right-click menu, Publish is selected.

  3. In the Pick a Publish Target window that appears, make sure Azure Function App is selected, choose the Select Existing radio button, then select Publish.

    In the Publish window, the Azure Function App tile is selected. Under this, both the Select Existing radio button and the Publish button are selected.

Note: If you do not see the ability to publish to an Azure Function, you may need to update your Visual Studio instance.

  1. In the App Service form, select your Subscription, select Resource Group under View, then expand your ServerlessArchitecture resource group and select the Function App whose name ends with FunctionApp.

  2. Whatever you named the Function App when you provisioned it is fine. Just make sure it is the same one to which you applied the Application Settings in Task 1 of this exercise.

    In the App Service form, Resource Group displays in the View field, and in the tree-view below, the ServerlessArchitecture folder is expanded, and TollBoothFunctionApp is selected.

  3. After you select the Function App, select OK.

    Note: If prompted to update the functions version on Azure, click Yes.

  4. Watch the Output window in Visual Studio as the Function App publishes. When it is finished, you should see a message that says, "Publish Succeeded."

  5. Using a new tab or instance of your browser navigate to the Azure Management portal, http://portal.azure.com.

  6. Open the ServerlessArchitecture resource group, then select the Azure Function App to which you just published.

  7. Expand the functions underneath your Function App in the menu. You should see both functions you just published from the Visual Studio solution listed.

    In the TollBoothFunctionApp blade, in the pane, both TollBoothFunctionApp, and Functions (Read Only) are expanded. Below Functions, two functions (ExportLicensePlates and ProcessImage) are called out.

  8. Now we need to add an Event Grid subscription to the ProcessImage function so the function is triggered when new images are added to blob storage. Select the ProcessImage function, then select Add Event Grid subscription.

    The ProcessImage function and the Add Event Grid subscription items are highlighted.

  9. On the Create Event Subscription blade, specify the following configuration options:

    1. Name: Unique value for the App name similar to processimagesub (ensure the green check mark appears).

    2. Event Schema: Select Event Grid Schema.

    3. For Topic Type, select Storage Accounts.

    4. Select your subscription and ServerlessArchitecture resource group.

    5. For resource, select your recently created Event Grid.

    6. Uncheck Subscribe to all event types, then select Blob Created from the event types dropdown list.

    7. Leave Web Hook as the Endpoint Type.

  10. Leave the remaining fields at their default values and select Create.

    In the Create Event Subscription blade, fields are set to the previously defined settings.

Exercise 3: Create functions in the portal

Duration: 45 minutes

Create two new Azure Functions written in Node.js, using the Azure portal. These will be triggered by Event Grid and output to Azure Cosmos DB to save the results of license plate processing done by the ProcessImage function.

Help references

Description Links
Create your first function in the Azure portal https://docs.microsoft.com/azure/azure-functions/functions-create-first-azure-function
Store unstructured data using Azure Functions and Azure Cosmos DB https://docs.microsoft.com/azure/azure-functions/functions-integrate-store-unstructured-data-cosmosdb

Task 1: Create function to save license plate data to Azure Cosmos DB

In this task, you will create a new Node.js function triggered by Event Grid and that outputs successfully processed license plate data to Azure Cosmos DB.

  1. Using a new tab or instance of your browser navigate to the Azure Management portal, http://portal.azure.com.

  2. Open the ServerlessArchitecture resource group, then select the Azure Function App you created whose name ends with Events. If you did not use this naming convention, make sure you select the Function App that you did not deploy to in the previous exercise.

  3. Select Functions in the menu. In the Functions blade, select + New Function.

    In the TollBoothEvents2 blade, in the pane under Function Apps, TollBoothEvents2 is expanded, and Functions is selected. In the pane, the + New function button is selected.

  4. Enter event grid into the template search form, then select the Azure Event Grid trigger template.

    1. If prompted, click Install and wait for the extension to install.

    2. Click Continue

    In the Template search form, event grid is typed in the search field. Below, the Event Grid trigger function option displays.

  5. In the New Function form, fill out the following properties:

    1. For name, enter SavePlateData

    In the New Function form SavePlateData is typed in the Name field.

    1. Select Create.
  1. Replace the code in the new SavePlateData function with the following:
module.exports = function(context, eventGridEvent) {
  context.log(typeof eventGridEvent);
  context.log(eventGridEvent);

  context.bindings.outputDocument = {
    fileName: eventGridEvent.data['fileName'],
    licensePlateText: eventGridEvent.data['licensePlateText'],
    timeStamp: eventGridEvent.data['timeStamp'],
    exported: false
  };

  context.done();
};
  1. Select Save.

Task 2: Add an Event Grid subscription to the SavePlateData function

In this task, you will add an Event Grid subscription to the SavePlateData function. This will ensure that the events sent to the Event Grid topic containing the savePlateData event type are routed to this function.

  1. With the SavePlateData function open, select Add Event Grid subscription.

    In the SavePlateData blade, the Add Event Grid subscription link is selected.

  2. On the Create Event Subscription blade, specify the following configuration options:

    1. Name: Unique value for the App name similar to saveplatedatasub (ensure the green check mark appears).

    2. Event Schema: Select Event Grid Schema.

    3. For Topic Type, select Event Grid Topics.

    4. Select your subscription and ServerlessArchitecture resource group.

    5. For resource, select your recently created Event Grid.

    6. Ensure that Subscribe to all event types is checked. You will enter a custom event type later.

    7. Leave Web Hook as the Endpoint Type.

  3. Leave the remaining fields at their default values and select Create.

    In the Create Event Subscription blade, fields are set to the previously defined settings.

Task 3: Add an Azure Cosmos DB output to the SavePlateData function

In this task, you will add an Azure Cosmos DB output binding to the SavePlateData function, enabling it to save its data to the Processed collection.

  1. Expand the SavePlateData function in the menu, then select Integrate.

  2. Under Outputs, select + New Output, select Azure Cosmos DB from the list of outputs, then select Select.

    In the SavePlateData blade, in the pane under Function Apps, TollBoothEvents2, Functions, and SavePlateData are expanded, and Integrate is selected. In the pane, + New Output is selected under Outputs. In the list of outputs, the Azure Cosmos DB tile is selected.

  3. In the Azure Cosmos DB output form, select new next to the Azure Cosmos DB account connection field.

    The New button is selected next to the Azure Cosmos DB account connection field.

    Note: If you see a notice for "Extensions not installed", click Install.

  4. Select your Cosmos DB account from the list that appears.

  5. Specify the following configuration options in the Azure Cosmos DB output form:

    1. For database name, type LicensePlates.

    2. For the collection name, type Processed.

    Under Azure Cosmos DB output the following field values display: Document parameter name, outputDocument; Collection name, Processed; Database name, LicensePlates; Azure Cosmos DB account conection, tollbooths_DOCUMENTDB.

  6. Select Save.

    Note: you should wait for the template dependency to install if you were prompted earlier.

Task 4: Create function to save manual verification info to Azure Cosmos DB

In this task, you will create a new function triggered by Event Grid and outputs information about photos that need to be manually verified to Azure Cosmos DB.

  1. Select Functions in the menu. In the Functions blade, select + New Function.

    In the pane of the TollBoothEvents2 blade under Functions Apps, TollBoothEvents2 is expanded, and Functions is selected. In the pane, + New function is selected.

  2. Enter event grid into the template search form, then select the Azure Event Grid trigger template.

    1. If prompted, click Install and wait for the extension to install.

    2. Click Continue.

    Event grid displays in the choose a template search field, and in the results, Event Grid trigger displays.

  3. In the New Function form, fill out the following properties:

    1. For name, type QueuePlateForManualCheckup.

    In the New Function form, JavaScript is selected from the Language drop-down menu, and QueuePlateForManualCheckup is typed in the Name field.

  4. Select Create.

  5. Replace the code in the new QueuePlateForManualCheckup function with the following:

module.exports = async function(context, eventGridEvent) {
  context.log(typeof eventGridEvent);
  context.log(eventGridEvent);

  context.bindings.outputDocument = {
    fileName: eventGridEvent.data['fileName'],
    licensePlateText: '',
    timeStamp: eventGridEvent.data['timeStamp'],
    resolved: false
  };

  context.done();
};
  1. Select Save.

Task 5: Add an Event Grid subscription to the QueuePlateForManualCheckup function

In this task, you will add an Event Grid subscription to the QueuePlateForManualCheckup function. This will ensure that the events sent to the Event Grid topic containing the queuePlateForManualCheckup event type are routed to this function.

  1. With the QueuePlateForManualCheckup function open, select Add Event Grid subscription.

    In the QueuePlateForManualCheckup blade, the Add Event Grid subscription link is selected.

  2. On the Create Event Subscription blade, specify the following configuration options:

    1. Name: Unique value for the App name similar to queueplateFormanualcheckupsub (ensure the green check mark appears).

    2. Event Schema: Select Event Grid Schema.

    3. For Topic Type, select Event Grid Topics.

    4. Select your subscription and ServerlessArchitecture resource group.

    5. For resource, select your recently created Event Grid.

    6. Ensure that Subscribe to all event types is checked. You will enter a custom event type later.

    7. Leave Web Hook as the Endpoint Type.

  3. Leave the remaining fields at their default values and select Create.

    In the Create Event Subscription blade, fields are set to the previously defined settings.

Task 6: Add an Azure Cosmos DB output to the QueuePlateForManualCheckup function

In this task, you will add an Azure Cosmos DB output binding to the QueuePlateForManualCheckup function, enabling it to save its data to the NeedsManualReview collection.

  1. Expand the QueuePlateForManualCheckup function in the menu, the select Integrate.

  2. Under Outputs, select + New Output then select Azure Cosmos DB from the list of outputs, then select Select.

    In the blade, in the pane under Function Apps, TollBoothEvents2\ Functions\QueuePlateForManualCheckup are expanded, and Integrate is selected. In the pane, + New Output is selected under Outputs. In the list of outputs, the Azure Cosmos DB tile is selected.

  3. Specify the following configuration options in the Azure Cosmos DB output form:

    Note: If you see a notice for "Extensions not installed", click Install.

    1. For database name, enter LicensePlates.

    2. For collection name, enter NeedsManualReview.

    3. Select the Azure Cosmos DB account connection you created earlier.

    In the Azure Cosmos DB output form, the following field values display: Document parameter name, outputDocument; Collection name, NeedsManualReview; Database name, LicensePlates; Azure Cosmos DB account connection, tollbooths_DOCUMENTDB.

  4. Select Save.

Task 7: Configure custom event types for the new Event Grid subscriptions

In this task, you will configure a custom event type for each new Event Grid subscription you created for your functions in the previous steps of this exercise. Currently the event types are set to All. We want to narrow this down to only the event types specified within the SendToEventGrid class in the TollBooth solution. This will ensure that all other event types are ignored by your functions.

  1. Using a new tab or instance of your browser navigate to the Azure Management portal, http://portal.azure.com.

  2. Open the ServerlessArchitecture resource group, then select your Event Grid Topic.

  3. At the bottom of the Overview blade, you will see both Event Grid subscriptions created from the functions. Select saveplatedatasub.

    On the Event Grid Topic overview blade, select the saveplatedatasub Event Grid subscription.

  4. Select the Filters tab, then uncheck Subscribe to all event types.

  5. Select the Delete button (x inside of a circle) on the All event type to delete it.

    The Filters tab is selected and the Subscribe to all event types checkbox has been cleared. An arrow is pointing to the Delete button on the All event type.

  6. Select Add Event Type, then enter savePlateData into the event types field. If you specified a different name in the SendToEventGrid class in the TollBooth solution, use that instead.

    The savePlateData event type has been added.

  7. Select Save.

  8. Select the queueplateformanualcheckupsub Event Grid subscription at the bottom of the Overview blade.

    On the Event Grid Topic overview blade, select the queueplateformanualcheckupsub Event Grid subscription.

  9. Select the Filters tab, then uncheck Subscribe to all event types.

  10. Select the Delete button (x inside of a circle) on the All event type to delete it.

    The Filters tab is selected and the Subscribe to all event types checkbox has been cleared. An arrow is pointing to the Delete button on the All event type.

  11. Select Add Event Type, then enter queuePlateForManualCheckup into the event types field. If you specified a different name in the SendToEventGrid class in the TollBooth solution, use that instead.

    The queuePlateForManualCheckup type has been added.

  12. Select Save.

Exercise 4: Monitor your functions with Application Insights

Duration: 45 minutes

Application Insights can be integrated with Azure Function Apps to provide robust monitoring for your functions. In this exercise, you will provision a new Application Insights account and configure your Function Apps to send telemetry to it.

Help references

Description Links
Monitor Azure Functions using Application Insights https://docs.microsoft.com/azure/azure-functions/functions-monitoring
Live Metrics Stream: Monitor & Diagnose with 1-second latency https://docs.microsoft.com/azure/application-insights/app-insights-live-stream

Task 1: Provision an Application Insights instance

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then type application insights into the search box on top. Select Application Insights from the results.

    In the Azure Portal menu pane, New is selected. In the New blade, application inisght is typed in the search box, and Application insights is selected from the search results.

  3. Select the Create button on the Application Insights overview blade.

  4. On the Application Insights blade, specify the following configuration options:

    1. Name: Unique value for the App name similar to TollboothMonitor (ensure the green check mark appears).

    2. For Application Type, select ASP.NET web application.

    3. Specify the Resource Group ServerlessArchitecture.

    4. Select the same location as your Resource Group.

    Fields in the Application Insights blade are set to the previously defined settings.

  5. Select Create.

Task 2: Enable Application Insights integration in your Function Apps

Both of the Function Apps need to be updated with the Application Insights instrumentation key so they can start sending telemetry to your new instance.

  1. After the Application Insights account has completed provisioning, open the instance by opening the ServerlessArchitecture resource group, and then selecting the your recently created application insights instance.

  2. Copy the instrumentation key from the Essentials section of the Overview blade.

Note: You may need to expand the Essentials section.

In the pane of the TollBoothMonitor blade, Overview is selected. In the pane, the copy button next to the Instrumentation key is selected.

  1. Open the Azure Function App you created whose name ends with FunctionApp, or the name you specified for the Function App containing the ProcessImage function.

  2. Select on Application settings on the Overview pane.

    In the pane of the TollBoothFunctionApp blade, under Configured features, Application settings is selected.

  3. Scroll down to the Application settings section. Use the + Add new setting link and name the new setting APPINSIGHTS_INSTRUMENTATIONKEY. Paste the copied instrumentation key into its value field.

    In the TollBoothFunctionApp blade, the + Add new setting link is selected. In the list of application settings, APPINSIGHTS_INSTRUMENTATIONKEY is selected.

  4. Select Save.

    Screenshot of the Save icon.

  5. Follow the steps above to add the APPINSIGHTS_INSTRUMENTATIONKEY setting to the function app that ends in Events.

Task 3: Use the Live Metrics Stream to monitor functions in real time

Now that Application Insights has been integrated into your Function Apps, you can use the Live Metrics Stream to see the functions' telemetry in real time.

  1. Open the Azure Function App you created whose name ends with FunctionApp, or the name you specified for the Function App containing the ProcessImage function.

  2. Select Application Insights on the Overview pane.

    Note: It may take a few minutes for this link to display.

    In the TollBoothFunctionApp blade, under Configured features, the Application Insights link is selected.

  3. In Application Insights, select Live Metrics Stream underneath Investigate in the menu.

    In the TollBoothMonitor blade, in the pane under Investigate, Live Metrics Stream is selected.

  4. Leave the Live Metrics Stream open and go back to the starter app solution in Visual Studio.

  5. Navigate to the UploadImages project using the Solution Explorer of Visual Studio.

    In Solution Explorer, UploadImages is expanded, and App.config is selected.

  6. Open App.config.

  7. In App.config, update the blobStorageConnection appSetting value to the connection string for your Blob storage account.

    On the App.config tab, a connection string is circled. To the side of the string is a green bar, and a hammer icon.

  8. Save your changes to the file.

  9. Right-click the UploadImages project in the Solution Explorer, then select Debug then Start new instance.

    In Solution Explorer, UploadImages is selected. From the Debug menu, Start new instance is selected.

  10. When the console window appears, enter 1 and press ENTER. This uploads a handful of car photos to the images container of your Blob storage account.

    A Command prompt window displays, showing images being uploaded.

  11. Switch back to your browser window with the Live Metrics Stream still open within Application Insights. You should start seeing new telemetry arrive, showing the number of servers online, the incoming request rate, CPU process amount, etc. You can select some of the sample telemetry in the list to the side to view output data.

    The Live Metrics Stream window displays information for the two online servers. Displaying line and point graphs include incoming requests, outgoing requests, and overvall health. To the side is a list of Sample Telemetry information.

  12. Leave the Live Metrics Stream window open once again, and close the console window for the image upload. Debug the UploadImages project again, then enter 2 and press ENTER. This will upload 1,000 new photos.

    the Command prompt window displays with image uploading information.

  13. Switch back to the Live Metrics Stream window and observe the activity as the photos are uploaded. It is possible that the process will run so efficiently that no more than two servers will be allocated at a time. You should also notice things such as a steady cadence for the Request Rate monitor, the Request Duration hovering below ~500ms second, and the Process CPU percentage roughly matching the Request Rate.

    In the Live Metrics Stream window, two servers are online. Under Incoming Requests. the Request Rate heartbeat line graph is selected, as is the Request Duration dot graph. Under Overall Health, the Process CPU heartbeat line graph is also selected, and a bi-directional dotted line points out the similarities between this graph and the Request Rate graph under Incoming Requests.

  14. After this has run for a while, close the image uploader console window once again, but leave the Live Metrics Stream window open.

Task 4: Observe your functions dynamically scaling when resource-constrained

In this task, you will change the Computer Vision API to the Free tier. This will limit the number of requests to the OCR service to 10 per minute. Once changed, run the UploadImages console app to upload 1,000 images again. The resiliency policy programmed into the FindLicensePlateText.MakeOCRRequest method of the ProcessImage function will begin exponentially backing off requests to the Computer Vision API, allowing it to recover and lift the rate limit. This intentional delay will greatly increase the function's response time, thus causing the Consumption plan's dynamic scaling to kick in, allocating several more servers. You will watch all of this happen in real time using the Live Metrics Stream view.

  1. Open your Computer Vision API service by opening the ServerlessArchitecture resource group, and then selecting the Cognitive Services service name.

  2. Select Pricing tier under Resource Management in the menu. Select the F0 Free pricing tier, then select Select.

    Note: If you already have a FO free pricing tier instance, you will not be able to create another one.

    In the TollboothOCR blade, under Resource Management, Pricing tier is selected. In the Choose your pricing tier blade, the F0 Free option is selected.

  3. Switch to Visual Studio, debug the UploadImages project again, then enter 2 and press ENTER. This will upload 1,000 new photos.

    the Command prompt window displays image uploading information.

  4. Switch back to the Live Metrics Stream window and observe the activity as the photos are uploaded. After running for a couple of minutes, you should start to notice a few things. The Request Duration will start to increase over time. As this happens, you should notice more servers being brought online. Each time a server is brought online, you should see a message in the Sample Telemetry stating that it is "Generating 2 job function(s)", followed by a Starting Host message. You should also see messages logged by the resilience policy that the Computer Vision API server is throttling the requests. This is known by the response codes sent back from the service (429). A sample message is "Computer Vision API server is throttling our requests. Automatically delaying for 32000ms".

    Note: If you do not see data flow after a short period, consider restarting the Function App.

    In the Live Metrics Stream window, 11 servers are now online.

  5. After this has run for some time, close the UploadImages console to stop uploading photos.

  6. Navigate back to the Computer Vision API and set the pricing tier back to S1 Standard.

Exercise 5: Explore your data in Azure Cosmos DB

Duration: 15 minutes

In this exercise, you will use the Azure Cosmos DB Data Explorer in the portal to view saved license plate data.

Help references

Description Links
About Azure Cosmos DB https://docs.microsoft.com/azure/cosmos-db/introduction

Task 1: Use the Azure Cosmos DB Data Explorer

  1. Open your Azure Cosmos DB account by opening the ServerlessArchitecture resource group, and then selecting the Azure Cosmos DB account name.

  2. Select Data Explorer from the menu.

    In the Tollbooth - Data Explorer blade, Data Explorer is selected.

  3. Expand the Processed collection, then select Documents. This will list each of the JSON documents added to the collection.

  4. Select one of the documents to view its contents. The first four properties are ones that were added by your functions. The remaining properties are standard and are assigned by Cosmos DB.

    Under Collections, Processed is expanded, and Documents is selected. On the Documents tab, a document is selected, and to the side, the first four properties of the document (fileName, licencePlateText, timeStamp, and exported) are circled.

  5. Expand the NeedsManualReview collection, then select Documents.

  6. Select one of the documents to view its contents. Notice that the filename is provided, as well as a property named "resolved". While this is out of scope for this lab, those properties can be used together to provide a manual process for viewing the photo and entering the license plate.

    Under Collections, NeedsManualReview is expanded, and Documents is selected. On the Documents tab, a document is selected, and to the side, the first four properties of the document (fileName, licencePlateText, timeStamp, and resolved) are circled.

  7. Right-click on the Processed collection and select New SQL Query.

    Under Collections, LicencePlates is expanded, and Processed is selected. From its right-click menu, New SQL Query is selected.

  8. Modify the SQL query to count the number of processed documents that have not been exported:

SELECT VALUE COUNT(c.id) FROM c WHERE c.exported = false
  1. Execute the query and observe the results. In our case, we have 1,369 processed documents that need to be exported.

    On the Query 1 tab, under Execute Query, the previously defined SQL query displays. Under Results, the number 1369 is highlighted.

Exercise 6: Create the data export workflow

Duration: 30 minutes

In this exercise, you create a new Logic App for your data export workflow. This Logic App will execute periodically and call your ExportLicensePlates function, then conditionally send an email if there were no records to export.

Help references

Description Links
What are Logic Apps? https://docs.microsoft.com/azure/logic-apps/logic-apps-what-are-logic-apps
Call Azure Functions from logic apps https://docs.microsoft.com/azure/logic-apps/logic-apps-azure-functions%23call-azure-functions-from-logic-apps

Task 1: Create the Logic App

  1. Navigate to the Azure Management portal, http://portal.azure.com.

  2. Select + Create a resource, then enter logic app into the search box on top. Select Logic App from the results.

    In the Azure Portal, in the menu, New is selected. In the New blade, logic ap is typed in the search field, and Logic App is selected in the search results.

  3. Select the Create button on the Logic App overview blade.

  4. On the Create Logic App blade, specify the following configuration options:

    1. For Name, type a unique value for the App name similar to TollBoothLogic (ensure the green check mark appears).

    2. Specify the Resource Group ServerlessArchitecture.

    3. Select the same location as your Resource Group.

    4. Select Off underneath Log Analytics.

    In the Create logic app blade, fields are set to the previously defined settings.

  5. Click Create. Open the Logic App once it has been provisioned.

  6. In the Logic App Designer, scroll through the page until you locate the Start with a common trigger section. Select the Recurrence trigger.

    The Recurrence tile is selected in the Logic App Designer.

  7. Enter 15 into the Interval box, and make sure Frequency is set to Minute. This can be set to an hour or some other interval, depending on business requirements.

  8. Select + New step.

    Under Recurrence, the Interval field is set to 15, and the + New step button is selected.

  9. Enter Functions in the filter box, then select the Azure Functions connector.

    Under Choose an action, Functions is typed in the search box. Under Connectors, Azure Functions is selected.

  10. Select your Function App whose name ends in FunctionApp, or contains the ExportLicensePlates function.

    Under Azure Functions, in the search results list, Azure Functions (TollBoothFunctionApp) is selected.

  11. Select the ExportLicensePlates function from the list.

    Under Azure Functions, under Actions (2), Azure Functions (ExportLicensePlates) is selected.

  12. This function does not require any parameters that need to be sent when it gets called. Select + New step, then search for condition. Select the Condition Control option from the Actions search result.

    Under ExportLicensePlates, the field is blank. Under the + New step button, Add a condition is selected.

  13. For the value field, select the Status code parameter. Make sure the operator is set to is equal to, then enter 200 in the second value field.

    Note: This evaluates the status code returned from the ExportLicensePlates function, which will return a 200 code when license plates are found and exported. Otherwise, it sends a 204 (NoContent) status code when no license plates were discovered that need to be exported. We will conditionally send an email if any response other than 200 is returned.

    The first Condition field displays Status code. The second, drop-down menu field displays is equal to, and the third field is set to 200.

  14. We will ignore the If true condition because we don't want to perform an action if the license plates are successfully exported. Select Add an action within the If false condition block.

    Under the Conditions field is an If true (green checkmark) section, and an if false (red x) section. In the If false section, Add an action is selected.

  15. Enter Send an email in the filter box, then select the Send an email action.

    From the Actions list, Office 365 Outlook (Send an email) is selected.

  16. Click Sign in and sign into your Office 365 Outlook account.

    In the Office 365 Outlook - Send an email prompt, the Sign in button is selected.

  17. In the Send an email form, provide the following values:

    1. Enter your email address in the To box.

    2. Provide a subject, such as Toll Booth license plate export failed.

    3. Enter a message into the body, and select the Status code from the ExportLicensePlates function so that it is added to the email body.

    Under Send an email, fields are set to the previously defined settings.

  18. Select Save in the tool bar to save your Logic App.

  19. Select Run to execute the Logic App. You should start receiving email alerts because the license plate data is not being exported. This is because we need to finish making changes to the ExportLicensePlates function so that it can extract the license plate data from Azure Cosmos DB, generate the CSV file, and upload it to Blob storage.

    The Run button is selected on the Logic Apps Designer blade toolbar.

  20. While in the Logic Apps Designer, you will see the run result of each step of your workflow. A green checkmark is placed next to each step that successfully executed, showing the execution time to complete. This can be used to see how each step is working, and you can select the executed step and see the raw output.

    In the Logic App Designer, green check marks display next to Recurrence, ExportLicensePlates, Condition, and Send an email.

  21. The Logic App will continue to run in the background, executing every 15 minutes (or whichever interval you set) until you disable it. To disable the app, go to the Overview blade for the Logic App and select the Disable button on the taskbar.

    The Disable button is selected on the TollBoothLogic blade top menu.

Exercise 7: Configure continuous deployment for your Function App

Duration: 40 minutes

In this exercise, configure your Function App that contains the ProcessImage function for continuous deployment. You will first set up a GitHub source code repository, then set that as the deployment source for the Function App.

Help references

Description Links
Creating a new GitHub repository https://help.github.com/articles/creating-a-new-repository/
Continuous deployment for Azure Functions https://docs.microsoft.com/azure/azure-functions/functions-continuous-deployment

Task 1: Create a GitHub repository

  1. Navigate to https://github.com and sign in.

  2. From the top corner of any page (once signed in), select the + menu item, then select New repository.

    From the Plus icon drop-down menu, New repository is selected.

  3. Select your owner account, enter a unique repository name, make sure it is set to Public, and then select Create repository.

    Note: When selecting Public, any connection information will be published to the GitHub repo. Ensure you follow the delete steps at the end of this to remove any sensitive information.

    In the New Repository section, the Repository name field is set to serverless-architecture-lab, and the Public radio button is selected. At the bottom, the Create repository button is selected.

  4. On your new repository page, copy the HTTPS git path to your clipboard, using the button provided.

    On the Repository page, the HTTPS git path copy to clipboard button is selected.

Task 2: Add GitHub repository to your Visual Studio solution

  1. Open the TollBooth project in Visual Studio.

  2. Right-click the TollBooth solution in Solution Explorer, then select Add Solution to Source Control.

    In Solution Explorer, TollBooth solution is selected. From its right-click menu, Add Solution to Source Control is selected.

  3. Select View in Visual Studio's top menu, then select Team Explorer.

    The View, Team Explorer menu item is highlighted.

  4. Click Sync under the Project section of the Team Explorer pane.

    The Sync button is highlighted.

  5. Select the Publish Git Repo button, then paste the git URL for your new repository you copied from GitHub. Finally, select Publish.

    In the Team Explorer - Synchronization window, the Publish Git Repo button is selected, and in the field below, the copied URL displays.

  6. Refresh your GitHub repository page in your browser. You should see that the project files have been added. Navigate to the TollBooth folder of your repo. Notice that the local.settings.json file has not been uploaded. That's because the .gitignore file of the TollBooth project explicitly excludes that file from the repository, making sure you don't accidentally share your application secrets.

    On the GitHub Repository page for serverless-architecture-lab, on the Code tab, project files display.

Task 3: Configure your Function App to use your GitHub repository for continuous deployment

  1. Open the Azure Function App you created whose name ends with FunctionApp, or the name you specified for the Function App containing the ProcessImage function.

  2. Select Deployment Center underneath the Platform features tab.

    In the TollBoothFunctionApp blade, on the Platform features tab, under Code Deployment, Deployment Center is selected.

  3. Select GitHub in the Deployment Center blade. Enter your GitHub credentials if prompted. Select Continue.

    In the Deployment Center blade, the GitHub icon is selected.

  4. Select App Service Kudu build server, then select Continue.

    Under the Build Provider step, App Service Kudu build server is selected.

  5. Choose your organization.

  6. Choose your new repository under Choose project. Make sure the master branch is selected.

    Fields in the Deployment option blade set to the following settings: Choose your organization, Personal; Choose repository, serverless-architecture-lab; Choose branch, master.

  7. Select Continue.

  8. On the Summary page, select Finish.

  9. After continuous deployment is configured, all file changes in your deployment source are copied to the function app and a full site deployment is triggered. The site is redeployed when files in the source are updated.

    The Deployment Center is shown with a pending build.

Task 4: Finish your ExportLicensePlates function code and push changes to GitHub to trigger deployment

  1. Navigate to the TollBooth project using the Solution Explorer of Visual Studio.

  2. From the Visual Studio View menu, select Task List.

    Task List is selected from the Visual Studio View menu.

  3. There you will see a list of TODO tasks, where each task represents one line of code that needs to be completed.

  4. Open DatabaseMethods.cs.

  5. The following code represents the completed task in DatabaseMethods.cs:

// TODO 5: Retrieve a List of LicensePlateDataDocument objects from the collectionLink where the exported value is false.
licensePlates = _client.CreateDocumentQuery<LicensePlateDataDocument>(collectionLink,
        new FeedOptions() { EnableCrossPartitionQuery=true,MaxItemCount = 100 })
    .Where(l => l.exported == false)
    .ToList();
// TODO 6: Remove the line below.
  1. Make sure that you deleted the following line under TODO 6: licensePlates = new List<LicensePlateDataDocument>();.

  2. Save your changes then open FileMethods.cs.

  3. The following code represents the completed task in DatabaseMethods.cs:

// TODO 7: Asyncronously upload the blob from the memory stream.
await blob.UploadFromStreamAsync(stream);
  1. Save your changes.

  2. Right-click the TollBooth project in Solution Explorer, then select Commit... under the Source Control menu item.

    In Solution Explorer, TollBooth is selected. From its right-click menu, Source Control and Commit are both selected.

  3. Enter a commit message, then select Commit All.

    In the Team Explorer - Changes window,

  4. After committing, select the Sync link. This will allow us to add the remote GitHub repository.

    Under Team Explorer - Changes, in the informational message Commit 02886e85 created locally. Sync to share, the Sync link is selected.

  5. Select the Sync button on the Synchronization step.

    Under Synchronization in the Team Explorer - Synchronization window, the Sync link is selected.

    Afterward, you should see a message stating that the incoming and outgoing commits were successfully synchronized.

  6. Go back to Deployment Center for your Function App in the portal. You should see an entry for the deployment kicked off by this last commit. Check the timestamp on the message to verify that you are looking at the latest one.

    The latest deployment is displayed in the Deployment Center.

Exercise 8: Rerun the workflow and verify data export

Duration: 10 minutes

With the latest code changes in place, run your Logic App and verify that the files are successfully exported.

Task 1: Run the Logic App

  1. Open your ServerlessArchitecture resource group in the Azure portal, then select your Logic App.

  2. From the Overview blade, select Enable.

    In the TollBoothLogic, the Enable enable button is selected.

  3. Now select Run Trigger, then select Recurrence to immediately execute your workflow.

    In the TollBoothLogic blade, Run Trigger / Recurrence is selected.

  4. Select the Refresh button next to the Run Trigger button to refresh your run history. Select the latest run history item. If the expression result for the condition is true, then that means the CSV file should've been exported to Blob storage. Be sure to disable the Logic App so it doesn't keep sending you emails every 15 minutes. Please note that it may take longer than expected to start running, in some cases.

    In Logic App Designer, in the Condition section, under Inputs, true is circled.

Task 2: View the exported CSV file

  1. Open your ServerlessArchitecture resource group in the Azure portal, then select your Storage account you had provisioned to store uploaded photos and exported CSV files.

  2. In the Overview pane of your storage account, select Blobs.

    Under Services, Blobs is selected.

  3. Select the export container.

    Export is selected under Name.

  4. You should see at least one recently uploaded CSV file. Select the filename to view its properties.

    In the Export blade, under name, a .csv file is selected.

  5. Select Download in the blob properties window.

    In the Blob properties blade, the Download button is selected.

    The CSV file should look similar to the following:

    A CSV file displays with the following columns: FileName, LicensePlateText, TimeStamp, and LicensePlateFound.

  6. The ExportLicensePlates function updates all of the records it exported by setting the exported value to true. This makes sure that only new records since the last export are included in the next one. Verify this by re-executing the script in Azure Cosmos DB that counts the number of documents in the Processed collection where exported is false. It should return 0 unless you've subsequently uploaded new photos.

After the hands-on lab

Duration: 10 minutes

In this exercise, attendees will deprovision any Azure resources that were created in support of the lab.

Task 1: Delete the resource group in which you placed your Azure resources

  1. From the Portal, navigate to the blade of your Resource Group and select Delete in the command bar at the top.

  2. Confirm the deletion by re-typing the resource group name and selecting Delete.

  3. If you created a different resource group for your virtual machined, be sure to delete that as well.

  4. Optionally, delete the GitHub repository you created for this lab by selecting settings and then Delete this repository from the GitHub website.

Task 2: Delete the GitHub repo

  1. Open https://www.github.com, then select your profile icon and select Your repositories

  2. Navigate to your repo and select it

  3. Click the Settings tab, scroll to the bottom, select Delete this repository

You should follow all steps provided after attending the Hands-on lab.

Attribution

This content was originally posted here:
https://github.com/Microsoft/MCW-Serverless-Architecture

License

This content is licensed with the MIT License license.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE