F
flanakin
If you haven't heard about the FinOps Open Cost and Usage Specification (FOCUS), check out our FOCUS 1.0 introduction blog post. But if you're here, I'm guessing you've at least looked into the FOCUS 1.0 preview which we announced in November 2023. We were diligently working alongside the FOCUS community to implement feedback and solidify the general availability release of the specification, which we were excited to finally release at FinOps X this year. But you probably know all that by now, so let's focus on the core changes you'll find if you're already using FOCUS 1.0 preview:
In FOCUS 1.0 preview, we had a Region column to indicate what region resources were deployed to. The specification wasn't clear as to whether this was the ID or name and, after discussion, it was determined that both are valuable for FinOps practitioners. With that, Region was renamed and split into separate RegionId and RegionName columns. In Azure, this will return lowercase values without spaces, like "brazilsouth" for the ID, and English display names, like "Brazil South", for the name. For anyone new to our FOCUS dataset, it's also important to note that FOCUS leverages the standard Azure region ID and display names, which are slightly different than the actual and amortized datasets. As an example, you'll see "Brazil South" instead of "BR South".
Related to the resource region, FOCUS 1.0 has declared that some columns, like AvailabilityZone are optional, and when not specified, they should not be included in the dataset. With that, we've removed the empty AvailabilityZone column from the FOCUS 1.0 dataset. Per the specification, we'll re-add this column when we can include the value.
In FOCUS, the amount you've used or purchased is tracked with the PricingQuantity, which is coupled with the PricingUnit to explain what the quantity refers to. Some services are priced in "blocks", like per 1,000 tokens for Azure OpenAI Service. Due to this, PricingQuantity is tracked separately from what was called UsageQuantity, which specified the distinct units without "blocking". In FOCUS 1.0, UsageQuantity and UsageUnit were renamed to ConsumedQuantity and ConsumedUnit and changed to only represent usage amounts and explicitly not cover purchase amounts. This means, ConsumedQuantity will only have a value when ChargeCategory == "Usage". To determine the amount purchased, please use the PricingQuantity.
FOCUS 1.0 preview didn't define a way to indicate what the on-demand unit price or cost was for a service after negotiated discounts were applied, but without commitment discounts. Given how important it is to calculate savings, we added extended columns for x_OnDemandCost and x_OnDemandUnitPrice to include this important detail. After much deliberation, FOCUS 1.0 introduced standard ContractedCost and ContractedUnitPrice columns for the same purpose. With that, we've renamed x_OnDemandCost and x_OnDemandUnitPrice columns to ContractedCost and ContractedUnitPrice.
The biggest change in FOCUS 1.0 was a reclassification of the ChargeSubcategory column from 1.0 preview. ChargeSubcategory was used to indicate what type of usage or adjustment the charge (row) represents. Specifically, this indicated whether a charge related to a commitment discount was used or unused and whether an adjustment was a credit, refund, etc. In FOCUS 1.0, ChargeSubcategory was removed in favor of the following changes:
Slightly tangential to these, PricingCategory values were also updated from "On-Demand" and "Commitment-Based" to "Standard" and "Committed". The purpose and intent remains the same for these. Only the values changed.
Beyond these core changes to the specification, we also updated the mappings used for regions, pricing units, service categories, and resource types to address a few gaps. These mapping files are hosted in the FinOps toolkit and available for anyone interested in data cleansing and normalization outside of FOCUS, like for historical datasets. If you notice anything missing, please do create an issue or submit a PR. To learn more, see the FinOps toolkit contribution guide.
Lastly, we also removed the empty x_ChargeId column. This column was discussed for inclusion, but didn't ultimately make it into the specification, so we never implemented it. If you'd like to see this added as an official column, please do submit feedback to the FOCUS community, which I'll cover in a bit.
Given FOCUS how supports a set of conditional columns which may not always be present, depending on what's supported by the provider, it's more important than ever to have a clear understanding of what columns are provided in the dataset, what data type they are, etc. This includes extended columns in addition to the FOCUS columns. To support this, we've added 2 new metadata files for FOCUS 1.0 and 1.0 preview in the FinOps toolkit.
We'd love to get your feedback on these to understand how you'll use them. We've already identified a use within FinOps hubs as part of our data ingestion pipeline. Let us know how you use it and if there's anything else you'd like to see. We're considering expanding this to document data transformation steps to convert actual and amortized data to FOCUS and also providing these metadata files for non-FOCUS datasets to provide the same programmatic level of detail for data integration scenarios.
So, what's next for FOCUS? The main thing we (as the FOCUS community) are looking for is your feedback. Learn about FOCUS, leverage reusable FOCUS reports (like the Power BI reports in the FinOps toolkit), align your existing reporting to FOCUS, educate stakeholders, and most importantly, if you have questions or feedback about the columns or values you see in your FOCUS dataset, let us know!
If you have questions about the Microsoft implementation, feel free to ask here on Tech Community, contact the Cost Management team through your Microsoft Services account team, or ask in the #microsoft-cloud channel in the FinOps Foundation Slack community.
If you have questions or feedback about the specification itself, use the #chat-focus channel in the FinOps Foundation Slack community or create an issue in the FOCUS GitHub repository, which will be triaged by maintainers within a few days to determine if it should be added to the 1.1 release, which is coming in November 2024 at FinOps X Europe!
These are exciting times! It's amazing to see 4 of the biggest cloud providers all adopting this revolutionary new data format! Let us know what you'd like to see as we plot out the future direction of FOCUS and our tooling to support your FinOps efforts.
Continue reading...
- Resource region
- Usage amounts
- Contracted (on-demand) cost
- Charge categorization
- Dataset metadata
Resource region
In FOCUS 1.0 preview, we had a Region column to indicate what region resources were deployed to. The specification wasn't clear as to whether this was the ID or name and, after discussion, it was determined that both are valuable for FinOps practitioners. With that, Region was renamed and split into separate RegionId and RegionName columns. In Azure, this will return lowercase values without spaces, like "brazilsouth" for the ID, and English display names, like "Brazil South", for the name. For anyone new to our FOCUS dataset, it's also important to note that FOCUS leverages the standard Azure region ID and display names, which are slightly different than the actual and amortized datasets. As an example, you'll see "Brazil South" instead of "BR South".
Related to the resource region, FOCUS 1.0 has declared that some columns, like AvailabilityZone are optional, and when not specified, they should not be included in the dataset. With that, we've removed the empty AvailabilityZone column from the FOCUS 1.0 dataset. Per the specification, we'll re-add this column when we can include the value.
Usage amount
In FOCUS, the amount you've used or purchased is tracked with the PricingQuantity, which is coupled with the PricingUnit to explain what the quantity refers to. Some services are priced in "blocks", like per 1,000 tokens for Azure OpenAI Service. Due to this, PricingQuantity is tracked separately from what was called UsageQuantity, which specified the distinct units without "blocking". In FOCUS 1.0, UsageQuantity and UsageUnit were renamed to ConsumedQuantity and ConsumedUnit and changed to only represent usage amounts and explicitly not cover purchase amounts. This means, ConsumedQuantity will only have a value when ChargeCategory == "Usage". To determine the amount purchased, please use the PricingQuantity.
Contracted (on-demand) cost
FOCUS 1.0 preview didn't define a way to indicate what the on-demand unit price or cost was for a service after negotiated discounts were applied, but without commitment discounts. Given how important it is to calculate savings, we added extended columns for x_OnDemandCost and x_OnDemandUnitPrice to include this important detail. After much deliberation, FOCUS 1.0 introduced standard ContractedCost and ContractedUnitPrice columns for the same purpose. With that, we've renamed x_OnDemandCost and x_OnDemandUnitPrice columns to ContractedCost and ContractedUnitPrice.
Charge categorization
The biggest change in FOCUS 1.0 was a reclassification of the ChargeSubcategory column from 1.0 preview. ChargeSubcategory was used to indicate what type of usage or adjustment the charge (row) represents. Specifically, this indicated whether a charge related to a commitment discount was used or unused and whether an adjustment was a credit, refund, etc. In FOCUS 1.0, ChargeSubcategory was removed in favor of the following changes:
- Refunds are now identified by ChargeClass == "Correction".
- ChargeCategory will be "Usage", "Purchase", etc. for any "correction" charges, allowing you to sum the total usage costs, including refunds.
- Credits are now identified by ChargeCategory == "Credit" in FOCUS, but note that credits are not currently included in Cost Management data.
- To determine whether a charge represents a used or unused portion of a commitment discount, use CommitmentDiscountStatus == "Used" or "Unused".
Slightly tangential to these, PricingCategory values were also updated from "On-Demand" and "Commitment-Based" to "Standard" and "Committed". The purpose and intent remains the same for these. Only the values changed.
Other updates
Beyond these core changes to the specification, we also updated the mappings used for regions, pricing units, service categories, and resource types to address a few gaps. These mapping files are hosted in the FinOps toolkit and available for anyone interested in data cleansing and normalization outside of FOCUS, like for historical datasets. If you notice anything missing, please do create an issue or submit a PR. To learn more, see the FinOps toolkit contribution guide.
Lastly, we also removed the empty x_ChargeId column. This column was discussed for inclusion, but didn't ultimately make it into the specification, so we never implemented it. If you'd like to see this added as an official column, please do submit feedback to the FOCUS community, which I'll cover in a bit.
Dataset metadata
Given FOCUS how supports a set of conditional columns which may not always be present, depending on what's supported by the provider, it's more important than ever to have a clear understanding of what columns are provided in the dataset, what data type they are, etc. This includes extended columns in addition to the FOCUS columns. To support this, we've added 2 new metadata files for FOCUS 1.0 and 1.0 preview in the FinOps toolkit.
We'd love to get your feedback on these to understand how you'll use them. We've already identified a use within FinOps hubs as part of our data ingestion pipeline. Let us know how you use it and if there's anything else you'd like to see. We're considering expanding this to document data transformation steps to convert actual and amortized data to FOCUS and also providing these metadata files for non-FOCUS datasets to provide the same programmatic level of detail for data integration scenarios.
What's next
So, what's next for FOCUS? The main thing we (as the FOCUS community) are looking for is your feedback. Learn about FOCUS, leverage reusable FOCUS reports (like the Power BI reports in the FinOps toolkit), align your existing reporting to FOCUS, educate stakeholders, and most importantly, if you have questions or feedback about the columns or values you see in your FOCUS dataset, let us know!
If you have questions about the Microsoft implementation, feel free to ask here on Tech Community, contact the Cost Management team through your Microsoft Services account team, or ask in the #microsoft-cloud channel in the FinOps Foundation Slack community.
If you have questions or feedback about the specification itself, use the #chat-focus channel in the FinOps Foundation Slack community or create an issue in the FOCUS GitHub repository, which will be triaged by maintainers within a few days to determine if it should be added to the 1.1 release, which is coming in November 2024 at FinOps X Europe!
These are exciting times! It's amazing to see 4 of the biggest cloud providers all adopting this revolutionary new data format! Let us know what you'd like to see as we plot out the future direction of FOCUS and our tooling to support your FinOps efforts.
Continue reading...