I tweeted and blogged about policy storm problems caused by using the following MBAM script
Invoke-MbamClientDeployment.ps1 during OSD with Configuration Manager version 2103 when the BitLocker Management feature is enabled, you can read about that here.
My original tweet about the issue was retweeted many times.
A number of customers were affected by the problem so Microsoft investigated it and produced a hotfix, which is now available for download in your Configuration Manager 2103 console. You can read the official information about that hotfix here.
Why the fuss ?
Many people are asking why did this become a problem and additionally why were people using the script in the first place. The two points below should cover that.
- When ConfigMgr 2103 was released it changed the way the recovery service was used for BitLocker recovery keys and that had a knock on affect for clients that used the MBAM script to escrow recovery information. The knock on affect was issues on the primary servers due to policy storms.
- People used the script to upload the recovery info to the database while the Operating System was being installed, this is done to minimize any security impact (time based) due to a missing key in the database. Without the script, the key could be missing from the database for hours (depending on policy) and that could mean a locked out system with no way of recovering it.
I’ll refer you to this tweet from the great Jason Sandys about the issue.
Check your task sequences to see if any of them use the Invoke-MbamClientDeployment.ps1 powershell script. Below is an example:
If you have deployed task sequences that use that script and if you are on ConfigMgr version 2103 then any computers deployed using this method can cause policy storms on the primary if the hotfix is not installed.
Identifying the problem
Before applying the fix, please verify if you actually have the issue. Run the following query on your Primary database.
SELECT PA.PolicyID, RPM.* FROM PolicyAssignment PA JOIN ResPolicyMap RPM ON PA.PADBID = RPM.PADBID WHERE PA.PolicyID like 'TPM%' AND RPM.MachineID = 0 AND RPM.IsTombstoned = 0
The screenshot below reveals that no devices were impacted by the issue, therefore the hotfix is not needed.
The screenshot below, shows one device that indeed has ‘reported in’. And that means you need to apply the hotfix to avoid policy storms. This is a lab so there is one device, imagine hundreds/thousands…
Installing the hotfix
Now that you know how to identify the problem, you’ll want to install the hotfix on affected sites, however it won’t show up unless you’ve installed the previous update rollup available for ConfigMgr version 2103, namely
- KB10036164: Update rollup for Microsoft Endpoint Configuration Manager version 2103
Below you can see the new hotfix (KB10372804) is listed with a state of ‘Ready to install‘.
The CMUpdate.log reveals lots of interesting changes related to BitLocker Management.
After the hotfix is listed as installed, any new OSD implementations that still use the Invoke script, will no longer show up using the SQL query, however, any previously installed computers that used that script will still show those records in the database and you should contact Microsoft to get information about removing those records.
Below I re-imaged a computer in the lab after the hotfix was installed and used the invoke script during OSD, this did allow the recovery key to get escrowed (using the MBAM powershell script) but more importantly it didn’t add a new
PA.PolicyID record to SQL, meaning no more policy storm. Awesome !
It’s great that Microsoft has released a hotfix for this issue however the hotfix will only address new OSD installs that use the invoke script after the hotfix has been applied, any pre-existing records will still suffer from the problem and necessitate a call to Microsoft support for removal/clean up.
Thanks to Frederic for his clarification for this blog post.