Tuesday, September 14, 2010

Exchange 2010 ActiveSync Issue (or the dreaded HTTP Error 500)

Exchange ActiveSync (EAS for us smart people) is a pretty rocking way to get Smart phones hooked up to your Exchange Server.  9 times out of ten it "just works" if you follow the docs, but sometimes it doesn't.  Otherwise I wouldn't writing about it now, would I?

I had a rather perplexing case where EAS only worked for some users.  On the non-working users the only error message displayed was the rather unhelpful HTTP Error 500.  The only suggestion the usually excellent www.testexchangeconnectivity.com could offer was an old article applicable to Exchange 2003.

Diagnosing

The next step was to scour the Application log on the Exchange Server, and lo and behold, the following error was logged:

Source: MSExchange ActiveSync
Event ID: 1053
Level: Error


Exchange ActiveSync doesn't have sufficient permissions to create the "CN=User Name,OU=Admins,OU=Employees,OU=People,DC=domain,DC=local" container under Active Directory user "Active Directory operation failed on server01.domain.local. This error is not retriable. Additional information: Access is denied.
Active directory response: 00000005: SecErr: DSID-031521D0, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
".
Make sure the user has inherited permission granted to domain\Exchange Servers to allow List, Create child, Delete child of object type "msExchangeActiveSyncDevices" and doesn't have any deny permissions that block such operations.


Righto.  So it would appear that to actually use EAS, information has to be written to the user account and to do this the Exchange Servers group needs to have permissions to the user object.  Further investigation revealed that the only accounts giving the error was Admin accounts.

AdminSDHolder and SDPROP are the culprits

Because the failing accounts has domain admins rights their User Object permissions were being reset by
AdminSDHolder, via a process called SDPROP.  I quote "Each Active Directory domain has an object called AdminSDHolder, which resides in the System container of the domain. The AdminSDHolder object has a unique Access Control List (ACL), which is used to control the permissions of security principals that are members of built-in privileged Active Directory groups (what I like to call "protected" groups). Every hour, a background process runs on the domain controller that holds the PDC Emulator operations master role. It compares the ACL on all security principals (users, groups and computer accounts) that belong to protected groups against the ACL on the AdminSDHolder object. If the size or the binary string is different, the security descriptor on the object is overwritten by the security descriptor from the AdminSDHolder object.."

Resolution

Because SDPROP runs every hour, the fix needs to be applied and a EAS sync done before an hour has gone past.  Here is how:
  1. Launch the EMC from your Exchange CAS Server
  2. Browse to Server Configuration and select your server
  3. Go to the System Settings tab to see which Domain Controller is in use
  4. Launch ADUC from the DC located in step 3 and make sure Advanced View is enabled
  5. Right-click the user account and go to Properties
  6. Click the Security tab, then click Advanced
  7. Check the Include inheritable permission.... checkbox
  8. OK out of all the dialog boxes
  9. Sync your device
The sync should be successful, providing not more than an hour has passed between the sync and step 1.  Whilst this has been a very difficult issue to nail down, there is a lesson in there somewhere...DON'T USE PRIVILEGED ACCOUNTS FOR DAY-TO-DAY USE!!!