Deprecating bad item limit and large item limit migration parameters

[Guest The_Exchange_Team]

During a mailbox migration to Exchange Online, the migration process can expose some data inconsistencies in the form of bad (missing) items and/or large items. A bad item is a corrupt item/property in the source mailbox that cannot be copied to the target mailbox. A large item is an item in the source mailbox that is larger than the maximum message size in Exchange Online. Bad items and large items cannot be migrated to Exchange Online. Bad items and large items can also be encountered during onboarding public folders as well as off-boarding.

 

Bad item limits (BIL) and large item limits (LIL) are used to set the maximum number of items allowed before a migration request fails. The BIL and LIL parameters are available in the Exchange admin center (EAC) and Exchange PowerShell cmdlets namely, Set-MoveRequest and New-MoveRequest. See this document about message size limits that could impact migrations.

 

Data Consistency Score (DCS) can help alert the administrator when the data fidelity score during migration is less than optimal based on historical data for different contextual elements in a mailbox item (like property, permissions, calendar item, metadata, etc.) On the other hand, BIL requires admins to estimate a BIL/LIL limit at the start of the process without any data. BIL also doesn’t have contextual or historical indicators for migration health and treats every type of corruption similarly. Thus, DCS not only helps eliminate corruption from user’s mailbox, but also empowers users with data to identify items that can’t be migrated. DCS empowers administrators to pass this information to the user being migrated to take action to preserve large items (as well as some corrupt items) in offline systems and prevents surprises after a migration is completed.

 

DCS is being used as the default engine to track and report on data inconsistencies during migrations to Exchange Online. DCS considers the quantity of bad, missing, and large items encountered along with the understanding of message elements (e.g., message content vs metadata) in the current migration as well as historic migration data to compute and describe the data fidelity score of a migration. Links to documents describing DCS and scoring details are in the Resources section below.

 

Currently, more than 90% of all onboarding migration jobs use DCS (instead of BIL/LIL) and approximately 75% of these jobs get a perfect score. It is encouraging to see that users are benefiting from the improvements DCS provides. Thus, we have decided to fully deprecate BIL/LIL and use DCS to measure and guide migration with data fidelity.

 

What to expect?

 

 

With this change,

 

• The modern EAC does not support BIL/LIL parameters, and the classic EAC is in the process of being deprecated.
  
• For migration cmdlets, values for BIL/LIL parameters would be ignored.
  
• Any migration that was started with BIL/LIL values will continue to respect these values. However, no new migrations will be able to use BIL/LIL parameters.
  

Timeline

 

 

The BIL/LIL parameters will be deprecated in the November/December 2022 timeframe.

 

Frequently asked questions

 

 

Where does DCS system fit into the migration pipeline?

 

Migrations start with DCS by default and a DCS score is presented to the admin towards the end of the migration, during the finalization step before completing the migration. If the DCS score is not ideal, the admin can review the items and determine next steps.

 

What can I do to discover bad item issues early?

As a best practice, before starting a migration, check for items that might cause issues.

 

• For mailbox integrity, see New-MailboxRepairRequest and Troubleshooting Failed Migrations.
  
• For large Items, use a script to identify blocking large items, as illustrated in the example below:
  

 

Get-Mailbox -ResultSize Unlimited | Get-MailboxFolderStatistics -IncludeAnalysis -FolderScope All | Where-Object {(($_.TopSubjectSize -Match "MB") -and ($_.TopSubjectSize -GE 35.0)) -or ($_.TopSubjectSize -Match "GB")} | Select-Object Identity, TopSubject, TopSubjectSize, DisplayName | Export-CSV -path "C:\temp\report.csv" -notype

 

• For calendar items inconsistencies, use the CalCheck tool.
  

 

What are the new bad item limits for DCS system?

DCS is a complex multi-factor system with different limits attributed to each factor based on historic data. We are constantly tuning the various thresholds used to determine the DCS score to ensure that problematic data loss does not occur during migrations. Currently, these limits are not exposed to administrators.

 

What can I do if I get a score of “investigate” in DCS?

DCS documentation talks about how DCS helps admins (1) communicate the issue to the affected user, and (2) move ahead with migration either by manually addressing the item or resuming after resolving the issues. Other than the documentation, please see Migrations with Data Consistency Score (DCS) – more than you ever wanted to know!

 

What can I do if I get a score of “Poor” in DCS?

If the job has score Poor, we recommend contacting support. We will work with you to identify the steps to resolve the issues and resume your migration jobs.

 

Do I need to change my scripts that I use for migration?

In phase 1, if your scripts use BIL/LIL parameters to run migration, you will notice that the BIL/LIL parameters are ignored for new migration jobs. The scripts do not fail in phase 1 due to this change. We would communicate when the parameters are invalid and can lead to failure. We recommend updating the scripts as we are rolling out this change to make transition to phase 2 seamless.

 

Additional resources

 

• Exchange online limits: Exchange Online limits - Service Descriptions | Microsoft Learn
  
• Identifying integrity issues: New-MailboxRepairRequest and Troubleshooting Failed Migrations
  
• Identifying permissions issue: Who is linked to whom in your spider web of mailbox permissions. Migration batch permissions analysis. | Microsoft Learn
  
• Identifying corrupt calendar items: CalCheck
  

 

Exchange Online Migration Team

 

Continue reading...