D
davidfrazee
The Azure Web Application Firewall (WAF) on Azure Front Door offers centralized protection for your web applications against vulnerabilities and threats. The effectiveness of your Azure WAF in managing traffic can be assessed through WAF logs stored in specified locations such as a Log Analytics Workspace or Storage Accounts. These logs document requests that have been either matched or blocked by WAF rules. This data is crucial for monitoring, auditing, and resolving issues. By default, WAF logs are maintained in a plain text format for user convenience and analysis. However, these client requests might include sensitive personal data, like personally identifiable information (PII), which can include names, addresses, contact details, and financial information. Without proper sanitization, logs containing such PII could be exposed to unauthorized access. To address this, Azure Front Door WAF now offers sensitive data protection through log scrubbing. WAF log scrubbing employs a customizable rules engine to pinpoint and redact sensitive portions within the requests, replacing them with a series of asterisks (******) to prevent data exposure. This blog will explains the log scrubbing process and provides practical examples for a more comprehensive understanding.
The Azure Front Door WAF’s sensitive data protection feature using log scrubbing is compatible with all WAF policy rule sets including Default Rule Set (DRS), Bot Manager Ruleset, and any Custom rules. It utilizes a range of match variables, such as client IP, headers, cookies, and request arguments, to identify data for scrubbing. Rule creation involves selecting a match variable, an operator, and defining a selector, which determines the specific key to be cleansed from the logs. Take a simple login process, which generally involves username and password fields; these are two distinct keys that can be targeted as selectors. Should a dubious login attempt activate the WAF, it records the username and password if they contain the suspect string or code injection. The log scrubber then anonymizes these details, obscuring the malicious content while preserving the attack's characteristics and significantly reducing the risk of personal data exposure.
See below for the full list of Match Variables:
Request IP, Request URI & Request Header
In our initial scenario, we examine the log scrubbing engine's capability to conceal the requester's IP address, the request URI, and the User-Agent that activated the WAF rule. In this example, the User-Agent is a carrier for a SQL injection string within the User-Agent key's value. The log scrubbing rules are configured so that the selector targets User-Agent within the Request Header Names, ensuring every User-Agent occurrence, including the embedded SQL injection string, is cleansed from the WAF logs. This scenario serves as an example on how to write log scrubbing rules to cleanse potentially sensitive values. It is best practice to specify only those values that contain potential personally identifiable information (PII) or other sensitive data, rather than generic request headers like User-Agent.
With the rules defined and the feature enabled, we’ll send a request using Postman that will trigger a block by the WAF and then check on the logs. Our screenshot below shows a 403 Forbidden status code returned from the Azure WAF policy.
Upon examining the logs, the fields labeled clientIP_s, requestUri_s, and matchVariableValue within details_matches_s are now populated with ***** as the full value. While the specific User-Agent involved in this incident is not visible in the logs, the presence of an attack can still be inferred through the analysis of the request headers. This method allows for the identification of unauthorized attempts to access the site, despite certain data being obscured.
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.CDN" and Category == "FrontDoorWebApplicationFirewallLog"
| project TimeGenerated, Resource, policy_s, clientIP_s, clientPort_d, requestUri_s, details_matches_s, details_msg_s, action_s, trackingReference_s
Request Body JSON
The below example shows screenshots of Request Body JSON argument identified for scrubbing. This example shows how to scrub a JSON argument within the WAF logs and uses Postman to generate the request traffic. A log scrubbing rule uses 'password' as the identifier within Request Body JSON Argument Name. This ensures only the 'password' value in the JSON payload is scrubbed rather than every value transmitted to the website. In our scenario the JSON payload triggers SQL injection WAF rules because of the embedded command string, prompting the Azure WAF to record the event in the logs.
{
“email”: “admin@juice-sh.op”,
“password”: “’ or 1=1—”
}
The WAF logs indicate that the JSON value, along with the requester's IP and the request URI, are effectively sanitized.
The below screenshots provide additional examples of how Azure WAF’s log scrubbing rules will remove sensitive data from WAF logs.
Request Cookie:
Request Body Post:
Query String:
The Azure Front Door WAF’s log scrubbing tool for sensitive data protection provides the same functionality as the current log scrubbing feature available for Application Gateway. This tool provides organizations with a robust solution for safeguarding sensitive data and personally identifiable information within their logs. Sensitive data protection is critical in reducing the likelihood of incurring legal or regulatory repercussions due to the inadvertent disclosure of private or confidential information. Implementing log scrubbing rules is a recommended best practice for all system administrators managing log data. For additional insights into Azure WAF, please refer to the accompanying resources.
Continue reading...
Log Scrubbing
The Azure Front Door WAF’s sensitive data protection feature using log scrubbing is compatible with all WAF policy rule sets including Default Rule Set (DRS), Bot Manager Ruleset, and any Custom rules. It utilizes a range of match variables, such as client IP, headers, cookies, and request arguments, to identify data for scrubbing. Rule creation involves selecting a match variable, an operator, and defining a selector, which determines the specific key to be cleansed from the logs. Take a simple login process, which generally involves username and password fields; these are two distinct keys that can be targeted as selectors. Should a dubious login attempt activate the WAF, it records the username and password if they contain the suspect string or code injection. The log scrubber then anonymizes these details, obscuring the malicious content while preserving the attack's characteristics and significantly reducing the risk of personal data exposure.
See below for the full list of Match Variables:
| Match Variable | Operator | Selector |
| Request IP Address | Equals any | <None> |
| Request URI | Equals any | <None> |
| Request Header Names | Equals/Equals any | <Custom> |
| Request Cookie Names | Equals/Equals any | <Custom> |
| Request Body Post Arg Names | Equals/Equals any | <Custom> |
| Request Body Json Arg Names | Equals/Equals any | <Custom> |
| Query String Arg Names | Equals/Equals any | <Custom> |
Request IP, Request URI & Request Header
In our initial scenario, we examine the log scrubbing engine's capability to conceal the requester's IP address, the request URI, and the User-Agent that activated the WAF rule. In this example, the User-Agent is a carrier for a SQL injection string within the User-Agent key's value. The log scrubbing rules are configured so that the selector targets User-Agent within the Request Header Names, ensuring every User-Agent occurrence, including the embedded SQL injection string, is cleansed from the WAF logs. This scenario serves as an example on how to write log scrubbing rules to cleanse potentially sensitive values. It is best practice to specify only those values that contain potential personally identifiable information (PII) or other sensitive data, rather than generic request headers like User-Agent.
With the rules defined and the feature enabled, we’ll send a request using Postman that will trigger a block by the WAF and then check on the logs. Our screenshot below shows a 403 Forbidden status code returned from the Azure WAF policy.
Upon examining the logs, the fields labeled clientIP_s, requestUri_s, and matchVariableValue within details_matches_s are now populated with ***** as the full value. While the specific User-Agent involved in this incident is not visible in the logs, the presence of an attack can still be inferred through the analysis of the request headers. This method allows for the identification of unauthorized attempts to access the site, despite certain data being obscured.
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.CDN" and Category == "FrontDoorWebApplicationFirewallLog"
| project TimeGenerated, Resource, policy_s, clientIP_s, clientPort_d, requestUri_s, details_matches_s, details_msg_s, action_s, trackingReference_s
Request Body JSON
The below example shows screenshots of Request Body JSON argument identified for scrubbing. This example shows how to scrub a JSON argument within the WAF logs and uses Postman to generate the request traffic. A log scrubbing rule uses 'password' as the identifier within Request Body JSON Argument Name. This ensures only the 'password' value in the JSON payload is scrubbed rather than every value transmitted to the website. In our scenario the JSON payload triggers SQL injection WAF rules because of the embedded command string, prompting the Azure WAF to record the event in the logs.
{
“email”: “admin@juice-sh.op”,
“password”: “’ or 1=1—”
}
The WAF logs indicate that the JSON value, along with the requester's IP and the request URI, are effectively sanitized.
The below screenshots provide additional examples of how Azure WAF’s log scrubbing rules will remove sensitive data from WAF logs.
Request Cookie:
Request Body Post:
Query String:
Conclusion
The Azure Front Door WAF’s log scrubbing tool for sensitive data protection provides the same functionality as the current log scrubbing feature available for Application Gateway. This tool provides organizations with a robust solution for safeguarding sensitive data and personally identifiable information within their logs. Sensitive data protection is critical in reducing the likelihood of incurring legal or regulatory repercussions due to the inadvertent disclosure of private or confidential information. Implementing log scrubbing rules is a recommended best practice for all system administrators managing log data. For additional insights into Azure WAF, please refer to the accompanying resources.
Resources
- Azure WAF Overview - Introduction to Azure Web Application Firewall | Microsoft Learn
- What is Azure Web Application Firewall on Azure Front Door? - What is Azure Web Application Firewall on Azure Front Door? | Microsoft Learn
- What is Azure Web Application Firewall Sensitive Data Protection (Preview)? - Azure Web Application Firewall on Azure Front Door Sensitive Data Protection (preview) | Microsoft Learn
- How to mask sensitive data on Azure Web Application Firewall - How to mask sensitive data on Azure Web Application Firewall on Azure Front Door (preview) | Microsoft Learn
Continue reading...