How To: Get Exchange 2013 back up after failed CU install

Exchange 2013 is a whole new beast when it comes to a stand alone Exchange server. A failed Exchange 2013 CU (Cumulative Update) can leave your Exchange server unresponsive to outside requests even though all Services are up and Powershell diagnostics are reporting everything working correctly. It leaves many Exchange Admins new to 2013 scratching their heads and attempting to restore from backup only to learn that a restore from backup does not work! What is this sorcery?! Stay calm. This is an easy fix and a lesson for new 2013/2016 Exchange admins that are used to the previous versions.

Brave New World

These are dark days for Exchange admins that work for small businesses with less than 75 users or so. Exchange is becoming more of an enterprise product and much less of a small business email platform. Microsoft wants you to go to Office 365 if you’re small and leave Exchange to the “big boys”. This is made evident by the level of internal complexity Exchange has accrued in the last two releases (2013, 2016). A new concept that was introduced with Exchange 2013 is Server Component States (Read this to get more details). This new function is largely a Datacenter enhancement for managing unhealthy members of an Exchange cluster to prevent bottlenecks in performance as a result of a single unhealthy server grabbing client and database requests.

Cut to the end, will ya? How do I kill it?

So with the explanation of Server Component States behind us, we come to the one time this becomes a reality for an Exchange Admin with one Exchange server. A failed CU update. During a failed CU update the “Requesters” (Typically Maintenance and Functional) shutdown all Server Component States. What happens when this is the case? Well, the Exchange server appears to be functioning exactly like it should. All Services are up, all databases are mounted, and all web resources respond normally. However, mail flow is dead, no Outlook clients can connect or they do sporadically, and all Activesync functions are stalled. So how do you check for this state? One simple command:

Get-ServerComponentState –Identity <Server>

This will list all of the component states for that server. After a failed CU update it will show many “inactive”, if not all of them.

So how do I get it back? With a couple commands depending on the “Requesters” involved. (This example is with Functional and Maintenance)

Set-ServerComponentState -Identity <Server> -Component ServerWideOffline -State Active -Requester Functional

Set-ServerComponentState -Identity <Server> -Component ServerWideOffline -State Active -Requester Maintenance

How do you check which requester’s caused the inactive states? Well that is a bit tougher to find out. Two commands, one to set a variable to the Component ServerWideOffline and the second to pipe that into a call to Active Directory to get the value of the Requesters.

<variable> = Get-ServerComponentState –Identity <Server> -Component ServerWideOffline

<variable>.RemoteStates

You can also find the details in the Registry of the Exchange server or in Active Directory.

Registry Location =

\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\ServerComponentStates

Active Directory (ADSI Edit) Location =

Configuration > Services > Microsoft Exchange > Organization Name > Administrative Groups > Exchange Administrative Group (FYDIBOHF23SPDLT) > Servers > (Server Name).

Right click the Server Name and select Properties.  Scroll down to “msExchComponentStates”. Double click the entry and it will give you all the values added to this attribute. You want to look for the entries 1:ServerWideOffline:(Requester):(NumberStrings). Those are the Requesters you will need to re-enable the Exchange Server Component States.

ADSI_Edit_msExchComponentStates

This is why a restore from backup does not work. The component states are in Active Directory and the Exchange server will use the newest Component state entry between the Registry and Active Directory. So if you restore from backup, the registry entry will get overwritten with the entry from Active Directory because the Registry Entry from the backed up Exchange server will have an older Component State entry than the Active Directory one.

I hope this helps some Exchange Admin that is freaking out about their Exchange Server after a failed CU update. I was there once. It is not a fun place to be.

Cheers.

 

 

Advertisements

4 thoughts on “How To: Get Exchange 2013 back up after failed CU install

  1. After a failed CU 16 this has resolved the issue of no mailflow. Thanks so much for posting I can now troubleshoot why it failed in peace knowing that things are still working. All the best.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s