Windows Explored

It was reported by a local office technician that over the course of a few days several users were connecting to a site over the web that used Remote Desktop Connection to connect a remote desktop/terminal services session. Shortly after connecting, users complained the their session was being disconnected. The issue was initially troubleshot as possibly a local setting in the OS, such as the IE proxy or maybe the TMG firewall client, but switching to different proxy made no difference. It was then assumed that perhaps our network work was part of the problem. Port issue? Not likely, ports 443 and 3389 are too common and since the users were able to connect initially, this could be eliminated as the cause. Last, the remote site technical support was contacted and asked at what point inactive sessions were being dropped. The answer to that was 1 hour and so this, too, was eliminated as the cause.

Eventually, the issue made it my way, and the first thing I thought was that this was not specifically a “disconnect”. When I think disconnect, I’m thinking along the lines of excessive packet loss or corruption between the client and the server which results in a dropped connection. Another cause for a disconnected application could be that the client app or one of its components that handles the connection is crashing. To confirm my suspicion, I asked the local tech to provide me the name of one of the affected workstations. All the affected workstation were running Windows XP, which meant that if the RDP client was crashing, the post mortem debugger might be capturing this. I navigated across the network to \\computername\c$\Documents and Settings\All Users\Application Data\Microsoft\Dr Watson and saw a recent drwtsn32.log and user dump.

I love Dr Watson because it makes basic crash analysis so easy. Bypassing the dump, you can just jump into the plain text log. The log contains a history of application crashes that the it has captured over the course of the operating system’s install-lifetime experienced in any user account. This makes it an excellent troubleshooting tool for going back into time (the user.dmp is overwritten each time it is generated). Scrolling to the bottom of the log and searching up for the last application crash confirms the RDP client, mstsc.exe, was encountering an access violation exception:

The next step is to find the thread stack where the FAULT is occurring, perhaps some 3rd party module might allude to the cause:

Nothing 3rd party here, mstscax is the Terminal Services ActiveX client. Assuming the problem is with the Remote Desktop Connection client itself or one of its other components, I recommended that the users needing access to the remote server upgrade to the latest RDC client for Windows XP, 7.0. This can be found here: http://support.microsoft.com/kb/969084. From previous experience with a similar issue, installing the latest RDC client was enough to correct; and after a follow up with the local tech, the problem was reported as resolved.

The important lesson here is that being disconnected from network resource doesn’t necessarily mean that this is due to packet loss or instability. This kind of assumption can send you on a wild goose chase through the OS networking subsystem or the larger network itself. With a little knowledge of crash dumps, and more specifically the post-mortem debugger in XP, the problem was quickly identified.

That being said, Windows Vista/7 has done away with a default post-mortem debugger of any kind. However, you can enable crash dump archiving in all Windows OS’ after XP by simply creating a registry key. See here or here for details.