How To: Retrieve from CosmosDB using Azure API Management

[Guest Jeffrey Chilberto]

Overview

 

 

In this How To, I will show a simple mechanism for reading items from CosmosDB using Azure API Management (APIM). Azure Cosmos DB is a fully managed, serverless NoSQL database for high-performance applications of any size or scale. APIM is a API management service offering many features for manging APIs and exposing backend services.

 

 

 

Example Use Case

 

 

The example use case involves using CosmosDB to maintain a list of information and exposing the information using APIM as a public RSS feed. This will provide a user to illustrate APIM caching and transformation, and how APIM can call CosmosDB securely using managed identity.

 

 

 

Setup

 

 

The first step is to provision a CosmosDB and APIM service instance. In my resource group, I created the following services:

 

 

 

 

The first step is to get the URI for the new CosmosDB instance.

 

 

 

 

I also created several items that would be retrieved in the data explorer as illustrated below:

 

 

 

 

Over in the APIM, we will create a new HTTP API for our RSS Feed.

 

 

 

 

The Web Service URL is the CosmosDB URI copied earlier.

 

 

In the policy section, we will complete the API. Next though, let's setup the identity of the APIM.

 

 

 

Permissions

 

 

The next step is to enable the APIM to access CosmosDB. To do this, navigate to the Managed identities blade:

 

 

 

 

You will want the System assigned on:

 

 

 

 

To enable access to CosmosDB for APIM, we need to setup role-based access to the built-in role: Cosmos DB Built-in Data Reader. This article does a good job of explaining the setup. In the end, you will need a snippet to similar to the following:

 

 

 

 

 

$resourceGroupName = "<myResourceGroup>"

$accountName = "<myCosmosAccount>"

$readOnlyRoleDefinitionId = "<roleDefinitionId>" # as fetched above

# For Service Principals make sure to use the Object ID as found in the Enterprise applications section of the Azure Active Directory portal blade.

$principalId = "<aadPrincipalId>"

New-AzCosmosDBSqlRoleAssignment -AccountName $accountName `

-ResourceGroupName $resourceGroupName `

-RoleDefinitionId $readOnlyRoleDefinitionId `

-Scope "/" `

-PrincipalId $principalId

 

 

 

 

 

 

 

 

 

The $resourceGroupName and $accountName can be taken from the CosmosDB overview:

 

 

 

 

The $principalId is taken from Managed identities Object (principal) ID shown previously. The $readOnlyRoleDefinitionId can be retrieved using the following command:

 

 

 

 

 

az cosmosdb sql role definition list --account-name $accountName --resource-group $resourceGroupName

 

 

 

 

 

 

 

 

 

With the RBAC in place, let's setup the CosmosDB request in the policy.

 

 

Policy

 

 

Now back on the API, select the operation and click the Policy code editor:

 

 

 

 

This will provide an xml editor. Insert the following xml in the inboud element.

 

 

 

 

 

 

 

<inbound>

<base />

<set-variable name="requestDateString" value="@(DateTime.UtcNow.ToString("r"))" />

<authentication-managed-identity resource="https://cosmosrssfeed.documents.azure.com" output-token-variable-name="msi-access-token" ignore-error="false" />

<set-header name="Authorization" exists-action="override">

<value>@("type=aad&ver=1.0&sig=" + context.Variables["msi-access-token"])</value>

</set-header>

<set-header name="x-ms-date" exists-action="override">

<value>@(context.Variables.GetValueOrDefault<string>("requestDateString"))</value>

</set-header>

<set-header name="x-ms-version" exists-action="override">

<value>2018-12-31</value>

</set-header>

<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">

<value>false</value>

</set-header>

<rewrite-uri template="/dbs/RssFeedDB/colls/RssFeed/docs/" copy-unmatched-params="false" />

</inbound>

 

 

 

 

 

 

 

 

 

This was derived from the List Documents Rest API and the Common Rest Request Headers. Take note of the authentication-managed-identity step in the policy to set teh msi-access-token variable used in the Authorization header.

 

 

 

An RSS feed is a perfect candidate for caching so I added the following to the inbound element:

 

 

 

 

 

 

 

<cache-lookup vary-by-developer="false" vary-by-developer-groups="false" downstream-caching-type="none">

<vary-by-header>Accept</vary-by-header>

<vary-by-header>Accept-Charset</vary-by-header>

<vary-by-header>Authorization</vary-by-header>

</cache-lookup>

 

 

 

 

 

 

 

And, the following to the outbound policy.

 

 

 

 

 

<cache-store duration="20" />

 

 

 

 

 

 

 

To alter the result to XML, I added the following to the outbound element:

 

 

 

 

 

<set-body template="liquid">

<rss version="2.0">

<channel>

<title>My Information Feed</title>

<link>https://apim-rssfeed-au-se.azure-api.net</link>

<description>Super duper RSS Feed</description>{% for document in body.Documents %}<item><title>{{document.title}}</title><link>{{document.link}}</link><description>{{document.description}}</description></item>{% endfor %}</channel>

</rss>

</set-body>

<set-header name="Content-Type" exists-action="override">

<value>application/xml</value>

</set-header>

 

 

 

 

 

 

 

The body is transformed to XML using set-body and the liquid template. I originally tried using json-to-xml but found this too cumbersome. Also note the header is overridden to XML and be sure to put this before the base policy is called.

 

 

 

Test

 

 

Testing the API is nice and simple using the Test tab.

 

 

 

 

If the Azure gods are with you, you should not see an error and instead see a response transformed to XML.

 

 

 

 

 

 

Now, what if something goes wrong, or if you are not familiar with the Trace feature, read on.

 

 

 

Troubleshooting

 

 

The best way to troubleshoot is using Trace. This is located next to the send button.

 

 

 

 

For example, if you find the trace is not working you can determine if an entry was found in the cache as you will see a message similar to the following:

 

 

 

 

And subsequently, when there is a matching cache hit, the following message will be shown:

 

 

 

 

Trace is your friend

 

 

Summary

 

 

This How To illustrated a simple way to expose CosmosDB using API Management. We followed the best practice of using managed identities to setup trust between APIM and CosmosDB. The scenario also incorporated transforming the message from json to XML and illustrated a basic use of caching. Let me know if this saves you some time!

 

 

 

Cheers

 

Continue reading...