T
tsunomur
Overview
There are multiple methods available to monitor applications deployed on Azure Container Apps. While the official documentation outlines these techniques, this article aims to organize them for better understanding.
Although this guide is focused on Java Spring Boot applications, the sections regarding Azure Container Apps are applicable to other programming languages and frameworks too.
How to Monitor Applications on Azure Container Apps
The following approaches can be employed to monitor applications running on Azure Container Apps:
- Using Application Insights (Azure Monitor OpenTelemetry Distro)
- Using the OpenTelemetry Collector of the Container Apps Environment/a>
- Using Dapr
- Forwarding system logs/console logs to Log Analytics
We will explain these monitoring methods in more detail later in this article.
In the latter section of this blog, we have provided concise guidance on using Application Insights and OpenTelemetry to monitor applications. If you're not well-versed in this topic, please refer to “Monitoring Applications with Application Insights and OpenTelemetry” section.
Demo Application Specifications
For this article, we created a simple demo application that performs a quadrature operation on two GET request values and returns the result. The
/calc endpoint takes two parameters, and the controller sequentially invokes addition, subtraction, multiplication, and division APIs.The API endpoints are within one Java app, capable of self-calling or being deployed as separate Container Apps.
In this demo, Container Apps are deployed for the front end (app1) and each operation (add, sub, mul, div). Spring Boot's Controller in the front end calls each Container App endpoint.
1. Using Application Insights (Azure Monitor OpenTelemetry Distro)
This method requires incorporating the Application Insights agent either within the application's container or directly into the application itself. Refer Enable Azure Monitor OpenTelemetry for .NET, Node.js, Python, and Java applications for more details.
It doesn't depend on Azure Container Apps, so if you're already monitoring with Application Insights, no changes are needed. Naturally, you can use all the features available in Application Insights.
However, each application (each Container App) needs to specify the Application Insights agent or embed it in the code, which could increase the container image size and require connection string management.
Transaction view
Here's how a transaction and its exception appear.
You can observe that app1's Container App calls the add, sub, mul, div Container Apps. The division by zero result is displayed, showing where and what type of exception occurred.
You can see a larger image by clicking on each screenshot.
Metric
The metrics produced by the OpenTelemetry Meter are visible in Application Insights.
2. Using the OpenTelemetry Collector of the Container Apps Environment
In the Container Apps Environment, you can enable the OpenTelemetry Collector to receive data output by the OpenTelemetry SDK from Container Apps.
Similar to using Application Insights, this method also requires specifying or implementing the OpenTelemetry agent, but it eliminates the need to manage connection strings for each Container App by not using Application Insights.
Although you won't use Application Insights per Container App, you need to specify the Application Insights connection string per Container Apps Environment if you use it as an export destination.
This option is the easiest if you're already using OpenTelemetry. However, currently, metrics cannot be sent to Application Insights. The following table is on Collect and read OpenTelemetry data in Azure Container Apps (preview)
How to Enable OpenTelemetry collector
Run the following command while setting up the Container Apps Environment to enable this feature. Once activated, the
OTEL_EXPORTER_OTLP_ENDPOINT environment variable will be automatically configured for the application.- Azure CLI
az containerapp env telemetry app-insights set -n aca-otelsample -g rg-opentelemetry --connection-string <Connection String> --enable-open-telemetry-traces true --enable-open-telemetry-logs true
- Azure Portal
Transaction view
Similar to method #1, it displays the service call relationships. You can observe that the application sequentially calls the add, sub, mul, and div Container Apps.
3. Using Dapr
Using Container Apps with Dapr allows you to make requests between Container Apps, like
http://localhost:3500/v1.0/invoke/checkout/method/checkout/100. The Dapr container, working as a sidecar in Container Apps, intercepts the request and directs it to the appropriate Container App.Currently, Application Insights can receive traces of requests passing through Dapr. It leverages the Application Insights connection string configured in the Container Apps Environment. To enable this, you simply need to activate Dapr in Container Apps, without needing an Application Insights (OpenTelemetry) agent configuration. This method is the simplest if using Dapr is assumed.
It's important to note that Dapr can only monitor inter-service communication; it does not capture intra-service traces or metrics. This means activities such as an application within Container Apps accessing its own API endpoint or generating logs are not monitored.
However, if the API is built as a separate Container App and the communication between them uses Dapr, it will be tracked. To gather intra-service data, you must use Application Insights or OpenTelemetry agents as outlined in method #1 and #2.
Please note that you cannot use method #2 at the same time (an error will occur when setting the connection string). This limitation applies only to sending data to Application Insights and does not affect the use of Dapr itself.
How to enable Dapr based instrumentation
The command below sets up a Container Apps Environment. The flag
--dapr-instrumentation-key is used to indicate that Dapr should utilize the Instrumentation key from Application Insights.- Azure CLI
az containerapp env create --name aca-otelsample --resource-group rg-opentelemetry --location japaneast --dapr-connection-string <Connection String>
Details of the command can be found at az containerapp env create.
- Bicep
This Bicep code corresponds to the Azure CLI
--dapr-instrumentation-key option. You can include a connection string within the properties section.resource env 'Microsoft.App/managedEnvironments@2024-03-01' = {
name: aca_env_name
location: location
properties: {
appLogsConfiguration: {
destination: 'log-analytics'
logAnalyticsConfiguration: {
customerId: la.properties.customerId
sharedKey: la.listKeys().primarySharedKey
}
}
zoneRedundant: false
daprAIConnectionString: <Connection String> // <----- Setting the Application Insights connection string
workloadProfiles: [
{
workloadProfileType: 'Consumption'
name: 'Consumption'
}
]
peerAuthentication: {
mtls: {
enabled: false
}
}
}
}
Transaction view
In method #1 and #2, the front-end application (app1) sequentially invoked REST APIs for each quadratic operation. Conversely, in the method involving Dapr, each REST API call seems isolated. This occurs because Dapr handles communication between Container Apps without visibility into the internal processing of the application.
4. Forwarding System Logs/Console Logs to Log Analytics
Within the Container Apps Environment, you have the ability to forward both system and console logs to Log Analytics. Even though only console logs are collected (excluding events or metrics), this method is the simplest for outputting platform logs without altering the application's code or Container Apps settings.
The following document details the steps for activating logging in Container Apps to Log Analytics: Log storage and monitoring options in Azure Container Apps
How to view logs
As observed, without a corresponding ID, it becomes challenging to track application transactions based on the logs produced in Log Analytics.
Monitoring Applications with Application Insights and OpenTelemetry
For those new to Application Insights and OpenTelemetry, here is a brief introduction. If you are already knowledgeable about these topics, feel free to skip ahead to the Conclusion.
What Application Insights Can Do
Application Insights is an Azure-native service designed for application monitoring. This tool allows you to collect logs, metrics, and traces from your applications and visualize the data.
Refer to the Application Insights overview for a summary of the Application Insights.
Application Insights is an Azure service compatible with OpenTelemetry, the open standard for monitoring. It supports various language SDKs and can receive data from OpenTelemetry, enabling application monitoring without code changes. This service conveniently handles traces, logs, and metrics all in one place.
Here is a brief guide on how to monitor your Java application using Application Insights. There are two ways to utilize Application Insights from Java:
- Specifying the Azure Monitor OpenTelemetry Distro as a Java agent
- Integrating it directly in the source code
The agent method requires substituting the OpenTelemetry agent with the Azure Monitor OpenTelemetry Distro. Before launching your Java applications with the Java agent, you need to set up the connection string in an environment variable or in configuration file as illustrated below.
export APPLICATIONINSIGHTS_CONNECTION_STRING=<Your Connection String>
java -javaagent:"path/to/applicationinsights-agent-3.5.3.jar" -jar myapp.jar
Refer to the "Enable using code" section of the following document for implementation details.
Using Azure Monitor Application Insights with Spring Boot
The Java agent for Application Insights acts as a wrapper around OpenTelemetry components, offering comparable capabilities. In addition, it includes extended implementations that leverage the features available in Application Insights. For further details, refer Why should I use the "Azure Monitor OpenTelemetry Distro"?
What OpenTelemetry Can Do
OpenTelemetry enables the collection of logs, metrics, and traces, which can be integrated with various compliant tools. For an overview, visit the official site.
Key points are:
You retain ownership of your data without vendor lock-in
You need to learn only one set of APIs and conventions.
Similar to Application Insights integration discussed above, you can add OpenTelemetry to a Java app in two ways
- Using the OpenTelemetry agent as a Java agent via a command-line argument (codeless instrumentation)
- Adding it directly into the source code (code-based instrumentation)
To use codeless instrumentation approach, just add the Java agent as a java command argument for seamless integration without modifying your code:
# If Application Insights is used, this OpenTelemetry Java agent can be replaced by an Application Insights agent as discussed earlier.
java -javaagent:./opentelemetry-javaagent.jar -jar myapp.jar
You will also need to create an OpenTelemetry instance using Java Agent if you want to send metrics and logs, using the following code:
package com.example.otelsample;
import io.opentelemetry.api.OpenTelemetry;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import io.opentelemetry.api.GlobalOpenTelemetry;
@Configuration
public class OpenTelemetryConfig {
@Bean
public OpenTelemetry openTelemetry() {
return GlobalOpenTelemetry.get();
}
}
After that step, you can output logs, generate events, and create metrics with OpenTelemetry using the code below:
// Start an OpenTelemetry span and send an event
Span span = tracer.spanBuilder("CalcController").startSpan();
span.addEvent("Start calc");
// Output a log
logger.info("Dummy log message");
// Create a counter
counter = meter.counterBuilder("calc.counter")
.setDescription("How many times the calculator has been run.")
.setUnit("runs")
.build();
counter.add(1);
For code-based instrumentation implementation, refer to the documentation at Instrumentation
You need to collect and visualize data from OpenTelemetry. Open-source tools like Jaeger and Zipkin handle traces and logs, while Prometheus collects metrics that can be visualized with Grafana.
Using the OpenTelemetry Collector lets you gather data without embedding code in your application for each tool. It acts as an interface between your app and various tools. The demo architecture in the official documentation is easy to understand.
"Demo Architecture", OpenTelemetry.io, Demo Architecture
Applications send data to the OpenTelemetry Collector's endpoint (http://localhost:<port>/, where port is 4317 or 4318), and the backend endpoints are specified in the OpenTelemetry Collector's configuration.
For example, here is a configuration for the OpenTelemetry Collector:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
exporters:
debug:
verbosity: detailed
otlp/jaeger:
endpoint: jaeger-all-in-one:4317
tls:
insecure: true
prometheus:
endpoint: "0.0.0.0:8889"
const_labels:
label1: value1
zipkin:
endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"
format: proto
processors:
batch:
extensions:
health_check:
pprof:
endpoint: :1888
zpages:
endpoint: :55679
service:
telemetry:
logs:
level: DEBUG
sampling:
enabled: false
extensions: [pprof, zpages, health_check]
pipelines:
traces:
receivers: [otlp]
exporters: [debug, zipkin, otlp/jaeger]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug, prometheus]
logs:
receivers: [otlp]
exporters: [debug]
This configuration is based on "Configuration", OpenTelemetry.io, Configuration
In the receivers, set the receiving endpoints, and in the exporters, configure the export destinations. By configuring services in the pipelines, you can receive traces at the OTLP endpoint and export to Zipkin.
OpenTelemetry avoids vendor lock-in, offering developers choices but requiring various tools. For example, you might use Jaeger for logs and Prometheus for metrics, each needing specific configurations.
Application Insights simplifies this by providing data collection to visualization in one service, negating the need to learn multiple services.
Conclusion
As demonstrated, there are multiple ways to monitor Azure Container Apps. We hope you find the method that fits your application needs and preferred implementation approach.
Continue reading...