Windows Explored

One of the most common complaints your going to encounter in the desktop support role is when Outlook becomes unresponsive while a user is performing any random task in Outlook, whether it be switching between folders, going into a delegated mailbox, or sorting messages. Often, the knee-jerk reaction by the front line technical support is to assume something is wrong with Outlook and begin the gamut of what I term “blind-troubleshooting”. This usually involves running an range of scripted fixes (that is meant to address issues with some of our in-house or 3rd party add-ins), followed by a repair, reinstall, and/or creation of a new Outlook profile. Over the lifetime of this incident(s), the issue can drag out to several days because usually the issue cannot be reproduced immediately after the first fix, so each additional fix is tried at some point later when the user calls back. In some cases, this drags out for a couple weeks to where the user’s workstation is replaced or Windows profile is recreated.

And all to no avail.

Hangs are inevitable. That’s the nature of email in a large enterprise environment. While some hangs can be the result of problems in Outlook (such as a corrupt OST) or with its add-ins, a lot of the time these hangs are the result of Outlook waiting on the Exchange server to process some operation and then return the result. Often, the user might encounter the a warning from the Outlook icon in the system tray that states “Outlook is trying to retrieve data from the Microsoft Exchange Server <Server Name>”. But many times, the user is not going to see this message and assume it’s Outlook, or will simply ignore it, putting you on the spot to explain it. And when this happens, I have seen many a tech go about the gamut of attempted fixes mentioned above in absence of words.

I personally have dealt with many of these requests from certain irate user’s who became special projects of mine. I approach these all the same way: I eliminate unapproved 3rd party add-ins from the equation and then set up a method for gathering process dumps of Outlook when it encounters a hang. The easiest way to do this is to drop Procdump on the user’s workstation along with a small bat file on the desktop that I ask the user to run after they open Outlook, which contains the command to start Procdump and watch Outlook for a hang (e.g. Start c:\sysinternals\procdump -h outlook.exe outlookhang.dmp). When procmon encounters a hung window of 5 seconds, it will dump out the process to a .dmp file that can be examined in WinDbg.

This was the case recently when one of my colleagues was asked to look into a complaint from a user in a remote office about her Outlook getting hung up at random times. I outlined the process above for him and we waited for a few days as dumps were being collected on the user workstation. When all was said and done, I opened the dumps with the Windows Debugging Tools to confirm my suspicion (using the !analyze –v –hang command) that issue was not with Outlook directly but with it waiting on the Exchange server to process some operation:

Most of the dumps were of the same nature. In fact, just to show how common this can be, I reproduced this exact same stack by capturing the hang of another user in a different office complaining of hangs when switching between mailboxes and yet another in yet another office when switching between folders in her mailbox.

You don’t need to be especially adept at reading dumps to understand what is happening here. I’ve been using basic hang analysis of dumps to piece together many hang issues in various apps and don’t have any special training. You would read the stack from the bottom up. You can see MSMAPI (Outlook’s Messaging Application Programming Interface) doing some work, then EMSMDB (the Electronic Messaging System Microsoft Data Base) unloading before various RPC calls take place. In this case, RPC is used for passing some operation to the remote system (the exchange server), where work is waiting to be done. Ideally, upon completion of this work on the remote system, Outlook would then return to a responsive state.

Here is a short description of the remote procedure call:

An RPC is initiated by the client, which sends a request message to a known remote server to execute a specified procedure with supplied parameters. The remote server sends a response to the client, and the application continues its process.

Hopefully, this makes your job easier and you can then go off and tell the Network or Exchange admins to start looking at their servers and address the issue there instead of blaming Outlook.

If you want to troubleshoot RPC hangs further, take a look at this MSDN article: Analyzing a Stuck Call Problem.