S
Shikhaghildiyal
Reusable Templates for Sending Logs to Log Analytics Workspace
Challenge:
When it comes to developing custom policies for sending logs to a Log Analytics workspace for any Azure resource, there are multiple approaches one can take. However, if a customer wants a custom policy for all Azure resources in their environment, simply copying and pasting the same policy definition can lead to issues. We might miss copying it fully, or the way it works for one Azure resource to another may vary, requiring us to change the logic accordingly.
Solution:
To address above mentioned challenge, we can define standardized templates that can be reused, with specific parts of the template being modified as per the Azure product where it needs to be applied. This approach ensures consistency and reduces the risk of errors.
Below are two different templates created with this in mind. If Template 1 does not work, then Template 2 can be reused and tested accordingly.
Template 1 (Basic)
This Template is a basic template which can be used for majority of the Azure Resources where we have the option to enable diagnostic settings. If this template does not work, then please proceed with Template 2 as defined in this document.
Reference For Template 1:
Use Case for Template 1
We will use Application Insights for testing the above template and see how it works
Step 1: Create Policy Definition in Azure Portal, Provide Subscription and policy name
Step 2: Copy the template and add it under policy rule inside Policy Definition
Step 3: Please update Policy name, Resource type as given in template wherever applicable and then save the policy
Step 4: Once it is saved, click on assign, and assign the policy.
Step 5: Now, check the compliance report and see if resource is marked as complaint or non- compliant and take necessary actions accordingly.
Template 2(With Count Variable)
In certain scenarios, especially when dealing with compliance issues, the basic template (Template 1) may not be sufficient. One common issue you might encounter is misleading compliance results, where Azure resources return ambiguous or incorrect compliance states like [true, false]. This inconsistency can make it difficult to correctly assess resource compliance, especially in environments with a large number of Azure resources.
To mitigate this, we introduce Template 2, which utilizes a count variable to provide a more accurate and reliable logging mechanism for sending logs to your Log Analytics Workspace. By including a count variable, this template ensures that compliance statuses are clearly and correctly represented in your logs, eliminating false or misleading messages.
Reference For Template 2:
Use Case for Template 2
We will use Function Apps for testing the above template because if you use template 1 you will see issues in compliance reasons
Step 1: Create Policy Definition in Azure Portal, Provide Subscription and policy name
Step 2: Copy the template and add it under policy rule inside Policy Definition
Step 3: Please update Policy name, Resource type as given in template wherever applicable and then save the policy
Step 4: Once it is saved, click on assign, and assign the policy.
Step 5: Now, check the compliance report and see if resource is marked as complaint or non- compliant and take necessary actions accordingly.
Continue reading...
Challenge:
When it comes to developing custom policies for sending logs to a Log Analytics workspace for any Azure resource, there are multiple approaches one can take. However, if a customer wants a custom policy for all Azure resources in their environment, simply copying and pasting the same policy definition can lead to issues. We might miss copying it fully, or the way it works for one Azure resource to another may vary, requiring us to change the logic accordingly.
Solution:
To address above mentioned challenge, we can define standardized templates that can be reused, with specific parts of the template being modified as per the Azure product where it needs to be applied. This approach ensures consistency and reduces the risk of errors.
Below are two different templates created with this in mind. If Template 1 does not work, then Template 2 can be reused and tested accordingly.
Template 1 (Basic)
This Template is a basic template which can be used for majority of the Azure Resources where we have the option to enable diagnostic settings. If this template does not work, then please proceed with Template 2 as defined in this document.
Reference For Template 1:
Code:
{
"name": "(Name of Policy)",
"type": "Microsoft.Authorization/policyDefinitions",
"apiVersion": "2022-09-01",
"scope": null,
"properties": {
"displayName": "(Policy Display name )",
"policyType": "Custom",
"description": "(policy description)",
"mode": "All",
"metadata": {
"version": "1.0.0",
"category": "(as per product)"
},
"parameters": {
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"AuditIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"profileNameLAW": {
"type": "String",
"metadata": {
"displayName": "Profile name",
"description": "The diagnostic settings profile name"
},
"defaultValue": "setbypolicyLAW"
},
"logAnalytics": {
"type": "String",
"metadata": {
"displayName": "Log Analytics workspace",
"description": "Select Log Analytics workspace from dropdown list. ",
"strongType": "omsWorkspace",
"assignPermissions": true
}
},
"logsEnabledLAWBoolean": {
"type": "Boolean",
"metadata": {
"displayName": "Enable logs",
"description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
true,
false
],
"defaultValue": true
},
"metricsEnabledLAWBoolean": {
"type": "Boolean",
"metadata": {
"displayName": "Enable logs",
"description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
true,
false
],
"defaultValue": true
},
"evaluationDelay": {
"type": "String",
"metadata": {
"displayName": "Evaluation Delay",
"description": "Specifies when the existence of the related resources should be evaluated. The delay is only used for evaluations that are a result of a create or update resource request. Allowed values are AfterProvisioning, AfterProvisioningSuccess, AfterProvisioningFailure, or an ISO 8601 duration between 0 and 360 minutes."
},
"defaultValue": "AfterProvisioning"
}
},
"policyRule": {
"if": {
"field": "type",
"equals": "(please put resource type here)"
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"evaluationDelay": "[parameters('evaluationDelay')]",
"name": "[parameters('profileNameLAW')]",
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Insights/diagnosticSettings/logs[*].enabled",
"equals": "[parameters('logsEnabledLAWBoolean')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/metrics[*].enabled",
"equals": "[parameters('metricsEnabledLAWBoolean')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/workspaceId",
"equals": "[parameters('logAnalytics')]"
}
]
},
"roleDefinitionIds": [
"/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
"/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
],
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"resourceName": {
"type": "string"
},
"location": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"logsEnabledLAWBoolean": {
"type": "bool"
},
"metricsEnabledLAWBoolean": {
"type": "bool"
},
"profileNameLAW": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "(please put resource type here)/providers/diagnosticSettings",
"apiVersion": "2021-05-01-preview",
"name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileNameLAW'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"logs": [
{
"categoryGroup": "allLogs",
"enabled": "[parameters('logsEnabledLAWBoolean')]"
}
],
"metrics": [
{
"category": "AllMetrics",
"enabled": "[parameters('metricsEnabledLAWBoolean')]"
}
]
}
}
],
"outputs": {}
},
"parameters": {
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('fullName')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"logsEnabledLAWBoolean": {
"value": "[parameters('logsEnabledLAWBoolean')]"
},
"metricsEnabledLAWBoolean": {
"value": "[parameters('metricsEnabledLAWBoolean')]"
},
"profileNameLAW": {
"value": "[parameters('profileNameLAW')]"
}
}
}
}
}
}
}
}
}Use Case for Template 1
We will use Application Insights for testing the above template and see how it works
Step 1: Create Policy Definition in Azure Portal, Provide Subscription and policy name
Step 2: Copy the template and add it under policy rule inside Policy Definition
Step 3: Please update Policy name, Resource type as given in template wherever applicable and then save the policy
Step 4: Once it is saved, click on assign, and assign the policy.
Step 5: Now, check the compliance report and see if resource is marked as complaint or non- compliant and take necessary actions accordingly.
Template 2(With Count Variable)
In certain scenarios, especially when dealing with compliance issues, the basic template (Template 1) may not be sufficient. One common issue you might encounter is misleading compliance results, where Azure resources return ambiguous or incorrect compliance states like [true, false]. This inconsistency can make it difficult to correctly assess resource compliance, especially in environments with a large number of Azure resources.
To mitigate this, we introduce Template 2, which utilizes a count variable to provide a more accurate and reliable logging mechanism for sending logs to your Log Analytics Workspace. By including a count variable, this template ensures that compliance statuses are clearly and correctly represented in your logs, eliminating false or misleading messages.
Reference For Template 2:
Code:
{
"name": "(Name of Policy)",
"type": "Microsoft.Authorization/policyDefinitions",
"apiVersion": "2022-09-01",
"scope": null,
"properties": {
"displayName": "(Policy Display Name)",
"policyType": "Custom",
"description": "(policy description )",
"mode": "All",
"metadata": {
"version": "1.0.0",
"category": "(as per product)"
},
"parameters": {
"profileNameLAW": {
"type": "String",
"metadata": {
"displayName": "Profile name",
"description": "The diagnostic settings profile name"
},
"defaultValue": "setbypolicyLAW"
},
"evaluationDelay": {
"type": "String",
"metadata": {
"displayName": "Evaluation Delay",
"description": "Specifies when the existence of the related resources should be evaluated. The delay is only used for evaluations that are a result of a create or update resource request. Allowed values are AfterProvisioning, AfterProvisioningSuccess, AfterProvisioningFailure, or an ISO 8601 duration between 0 and 360 minutes."
},
"defaultValue": "AfterProvisioning"
},
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"AuditIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"logAnalytics": {
"type": "String",
"metadata": {
"displayName": "Log Analytics workspace",
"description": "Select the Log Analytics workspace from dropdown list",
"strongType": "omsWorkspace",
"assignPermissions": true
}
},
"metricsEnabledLAWBoolean": {
"type": "Boolean",
"metadata": {
"displayName": "Enable metrics",
"description": "Whether to enable metrics stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
true,
false
],
"defaultValue": true
},
"logsEnabledLAWBoolean": {
"type": "Boolean",
"metadata": {
"displayName": "Enable logs",
"description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
true,
false
],
"defaultValue": true
}
},
"policyRule": {
"if": {
"field": "type",
"equals": "(please put resource type here)"
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"name": "[parameters('profileNameLAW')]",
"evaluationDelay": "[parameters('evaluationDelay')]",
"roleDefinitionIds": [
"/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
],
"existenceCondition": {
"allOf": [
{
"count": {
"field": "Microsoft.Insights/diagnosticSettings/logs[*]",
"where": {
"allOf": [
{
"field": "Microsoft.Insights/diagnosticSettings/logs[*].enabled",
"equals": "true"
},
{
"field": "microsoft.insights/diagnosticSettings/logs[*].categoryGroup",
"equals": "allLogs"
}
]
}
},
"greaterorEquals": 1
},
{
"field": "Microsoft.Insights/diagnosticSettings/metrics[*].enabled",
"equals": "[parameters('metricsEnabledLAWBoolean')]"
}
]
},
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"resourceName": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"logsEnabledLAWBoolean": {
"type": "bool"
},
"metricsEnabledLAWBoolean": {
"type": "bool"
},
"profileNameLAW": {
"type": "string"
},
"location": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "(please put resource type here)/providers/diagnosticSettings",
"apiVersion": "2021-05-01-preview",
"name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"metrics": [
{
"category": "AllMetrics",
"timeGrain": null,
"enabled": "[parameters('metricsEnabledLAWBoolean')]"
}
],
"logs": [
{
"categoryGroup": "allLogs",
"enabled": "[parameters('logsEnabledLAWBoolean')]"
}
]
}
}
],
"outputs": {}
},
"parameters": {
"profileNameLAW": {
"value": "[parameters('profileNameLAW')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"metricsEnabledLAWBoolean": {
"value": "[parameters('metricsEnabledLAWBoolean')]"
},
"logsEnabledLAWBoolean": {
"value": "[parameters('logsEnabledLAWBoolean')]"
},
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('name')]"
}
}
}
}
}
}
}
}
}Use Case for Template 2
We will use Function Apps for testing the above template because if you use template 1 you will see issues in compliance reasons
Step 1: Create Policy Definition in Azure Portal, Provide Subscription and policy name
Step 2: Copy the template and add it under policy rule inside Policy Definition
Step 3: Please update Policy name, Resource type as given in template wherever applicable and then save the policy
Step 4: Once it is saved, click on assign, and assign the policy.
Step 5: Now, check the compliance report and see if resource is marked as complaint or non- compliant and take necessary actions accordingly.
Continue reading...