Windows Explored

Everyday Windows Desktop Support, Advanced Troubleshooting & Other OS Tidbits

Posts Tagged ‘Process Explorer’

The Case of the Failed MSI

Posted by William Diaz on December 3, 2015


I recently began encountering a problem where one of our application packages was getting hung during the install process. I could see the program directory correctly populated with application files but it seemed to get hung at end, and msiexec processes continued to run indefinitely. After manually killing each process, I checked to see if the application actually completed installing. When trying to run one of the applications that has a dependency on one of the failed MSI’s APIs, I encountered the following:

image

Further, Outlook was crashing when trying to load the application’s main component.

Faulting application name: OUTLOOK.EXE, version: 14.0.7012.1000, time stamp: 0x514a1b69
Faulting module name: imFileSite.dll, version: 8.5.3006.93, time stamp: 0x52d811c0
Exception code: 0xc0000005
Fault offset: 0x0008637e
Faulting process id: 0xd18
Faulting application start time: 0x01d12d4e055d29fc
Faulting application path: C:\Program Files (x86)\Microsoft Office\Office14\OUTLOOK.EXE
Faulting module path: C:\Program Files (x86)\Interwoven\WorkSite\iOutlook\imFileSite.dll

 

My guess was that the application’s components were not getting registered. To confirm, I manually registered all the components in the program directly via the command line:

FOR /R "C:\Program Files (x86)\ApplicationFolder" %G IN (*.dll) DO "%systemroot%\system32\regsvr32.exe" /s "%G").

Afterwards, the application with the dependency launched without error and Outlook was no longer crashing. However, the root issue still needed to be identified as I had no idea what else in the MSI was failing to complete. After manually deleting the components from the file system (it was missing from Programs and Features), I ran the MSI again and this time used Process Explorer to look inside the hung msiexec processes. There were multiple processes running, but I could see one process, though, eating CPU resources, and this was likely where I might find the culprit:

image

Opening the process details, I select the Threads tab. The thread that is doing all the work (or in this case getting hung up) is indicated by the Cycles tab. From here, I opened the thread by selecting it and clicking Stack:

image

Stacks are read from the bottom up. You can see a 3rd party component here (HCApi – McAfee HIPS) at play. Note, this might not be entirely abnormal as you can always expect any anti-virus suite to be hooking itself into any number of processes. So, to confirm what I was seeing, I used the Task Manager to dump the msiexec process that was using the CPU time by right-clicking it and select Create dump file (which can also be done via Process Explorer):

image

Dumps can be analyzed with WinDbg or a tool like DebugDiag 2.0. I have grown increasingly lazy and forgetful over the years with WinDbg. It has a very high learning curve and if you don’t use it much, its easy to forget everything except the old faithful !analyze –v or !analyze –v –hang. I used DebugDiag instead. When it is installed, simply right-click the dump file and select Analyze Crash/hang Issue from the context menu and point it to the crash file.

image

image

It will do its best to figure out the issue in the most vague way and you almost always need to do some interpretation of your own. For the most part, the analysis summary can be ignored.

image

A little bit down in the report points me to what I was seeing in Process Explorer with the problem thread that was using all the CPU time:

image

You can click the Thread ID like a hyperlink to follow it down in the report, expanding the thread. It reveals a familiar site. However, there is a bit more insight as the entry point is revealed and I can see the problem has something to do with the CustomAction table of the MSI itself.

image

Through a quick process of elimination using Orca to remove some of the rows in the CustomAction table, I narrow down the cause specifically down to action ISSelfRegisterCosting.

image

Without that row, the MSI install completes normally; or so at least it seems. I have no idea what removing this action might have elsewhere so this merely a hack. To further confirm the problem is being caused by McAfee, I perform the install on a virtual machine where McAfee is not installed and it proceeds normally. I then reach out to our McAfee enterprise admin and ask him to disable HIPS on one of my workstation. After doing so, the MSI runs and the application installs as expected. He inherits the problem.


Update

This has since been corrected by MacAfee with a HIPS update. One of the DLLs trying to get registered was getting blocked.

Posted in Troubleshooting, Troubleshooting Tools | Tagged: , | 9 Comments »

The Case of the Slowly Opening Or Unresponsive Office Files

Posted by William Diaz on August 15, 2012


After a recent security update for our XP workstations, a couple complaints came in where user’s were having difficulty opening Microsoft Office files across the network. In some cases, the file would open, but only after a delay of a few minutes. In other cases, the file would not open at all, causing the Office application (Word, Excel) to become unresponsive and hung up. The files were not ridiculously large, and opening the same files locally did not present a problem. Identifying the cause was a simple matter of turning to Process Explorer and examining the stack of the working program thread:

image

You can see the stack growing with a couple dozen calls to some component named GKExcel.dll. Turning on the Lower Pane to view DLLs (View > Lower Pane View > DLLs), I can see it is described as Microsoft Component, but the description is too generic to make out the purpose:

image

However, one of the functions may allude to what its purpose is and how it got here. Searching FValidateExcelFile takes me to this MS KB article Excel workbooks may open slowly over the network:

After you install MS11-021 and the Office File Validation (OFV) Add-in for Microsoft Office 2003 (KB 2501584), workbooks stored in a network location open more slowly over the network in Excel 2003 than they did without the OFV installed. The decrease in performance depends on the size of the workbook and bandwidth of the network, and in some scenarios, can seem to cause Excel to crash.

The issue is not specific to Excel, however. Word files were taking several minutes to open as well. Resolving is a simple matter of uninstalling the Microsoft Office File Validation Add-in or modifying the registry to make an exception for the application opening the file. To do this:

  • Go to HKCU\Software\Policies\Microsoft\Office\<ver>\<application>\Security.
  • Create a new key called FileValidation
  • Create a DWORD value called EnableOnLoad with a value of 0

If uninstalling across the enterprise, then: msiexec / x {90140000-2005-0000-0000-0000000FF1CE} / quiet.

Posted in Office, Troubleshooting, Troubleshooting Tools | Tagged: , , | Leave a Comment »

Help! Everything Is Crashing

Posted by William Diaz on July 25, 2012


This is an XP workstation so likely the post-mortem default debugger is capturing the exception. I UNC-navigate to \\computername\Documents and Settings\All Users\Application Data\Microsoft\DrWatson. I grab both the drwtsn32.log and user.dmp files. They have recent time stamps of the day before which means that they were likely created as a result of the issue the user was experiencing. I start by examining the log file, starting from the bottom working my way up. The user’s initial complaint was the IE was crashing when going to various websites. I expected to find iexplore.exe process crashing in the log. A few searches in the text file later, I find IE crashing on that day:

Application exception occurred:
        App: C:\Program Files\Internet Explorer\iexplore.exe (pid=6828)
        When: 7/24/2012 @ 11:28:13.701
        Exception number: c0000005 (access violation)

Read the rest of this entry »

Posted in Troubleshooting | Tagged: , , , , | Leave a Comment »

Quickly Configure Symbol Information for Process Monitor & Process Explorer

Posted by William Diaz on May 1, 2012


I often find myself running Process Monitor and Process Explorer on user workstations. But to get the most of either of these tools, you really need to configure symbols so you can accurately read thread and stack information when doing a deeper analysis of a process. This can often be a nuisance because I am a cut and paste type of guy and even after doing it numerous times, I still have trouble recalling the Microsoft symbol path. On top of that, many times I run these tools with the user connected so speed is a necessity. To work around that, I decided to write a small script that I can run from a file server that will do it for me. Run the script before your start either tool.

There is one prerequisite, however: you need the full dbghelp.dll from the Windows Debugging Tools as the debug help DLL in system32 is not sufficient. If you are running on both 32 and 64 bit systems, you will need to get both the 64 bit and 32 bit versions. Store them away on network share and modify the script below to look to that share. In my case, we are still a mixed environment so I renamed the 32 bit dbghelp to dbghelp32.dll while the 64 bit version remains unchanged and created two different scripts. The respective DLL will copied into a folder C:\DbgHelp on the local system .

I also added a 5 second duration for new open and close processes and threads.

const HKEY_CURRENT_USER = &H80000001
strComputer = "."
 
Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\" &_
strComputer & "\root\default:StdRegProv")
 
‘Process Explorer 64
 
strKeyPath = "Software\Sysinternals\Process Explorer"
oReg.CreateKey HKEY_CURRENT_USER,strKeyPath

strValueName = "DbgHelpPath"
strValue = "C:\DbgHelp\dbghelp.dll"
oReg.SetStringValue HKEY_CURRENT_USER,strKeyPath,strValueName,strValue

strValueName = "SymbolPath"
strValue = "http://msdl.microsoft.com/download/symbols&quot;
oReg.SetStringValue HKEY_CURRENT_USER,strKeyPath,strValueName,strValue

strValueName = "HighlightDuration"
dwValue = 5000
oReg.SetDWORDValue HKEY_CURRENT_USER,strKeyPath,strValueName,dwValue

‘Process Monitor 64

strKeyPath = "Software\Sysinternals\Process Monitor"
oReg.CreateKey HKEY_CURRENT_USER,strKeyPath

strValueName = "DbgHelpPath"
strValue = "C:\DbgHelp\dbghelp.dll"
oReg.SetStringValue HKEY_CURRENT_USER,strKeyPath,strValueName,strValue

strValueName = "SymbolPath"
strValue = "http://msdl.microsoft.com/download/symbols&quot;
oReg.SetStringValue HKEY_CURRENT_USER,strKeyPath,strValueName,strValue

‘Copy full dbghelp.dll 64 bit to folder DbgHelp.

Set oFSO = CreateObject("Scripting.FileSystemObject")
 
If Not oFSO.FolderExists( "C:\DbgHelp") Then
Set objFolder = oFSO.CreateFolder("C:\DbgHelp")
End If

Set FSO = CreateObject("Scripting.FileSystemObject")
FSO.CopyFile "\\Server1\TechTools\dbghelp.dll", "C:\DbgHelp\"

Posted in Troubleshooting Tools | Tagged: , | Leave a Comment »

Investigating & Resolving Message Delays in Outlook with WinDbg

Posted by William Diaz on May 1, 2012


The problem starts after the user has had their Windows profile recreated but continues to experience some Outlook performance issues. Each time she clicks the New button in Outlook to create a new message, there is a 5 to 10 second delay before the blank message opens. There is also a 10+ second delay after she clicks send and Outlook becomes unresponsive for that time. I connect to her and quickly look at the Office Outlook registry for any unapproved 3rd party add-ins and see nothing out of the ordinary. I decide to dump the Outlook process when the hang is encounter and turn to Process Explorer. This is a simple matter of right-clicking the process and choosing Create Dump when it is in an unresponsive state. Actually, I collect two dumps, the other with Procdump in hang mode. I do this to ensure that the dumps are consistent; if the dumps are not similar then I may need to collect more until a pattern emerges.

I copied the dumps to my workstation and opened them with WinDbg and run the !analyze –v –hang command. The stack for both dumps are similar. With the exception of the NRTExchn component (our DMS add-in for Outlook), these are all MS modules. Normally, this wouldn’t excite me but something stands out like a sore thumb as I move up the stack, the presence of msi.dll in the stack of the current examine thread.

Read the rest of this entry »

Posted in Office, Troubleshooting, Troubleshooting Tools | Tagged: , , , , | Leave a Comment »

Quickly Find Those File Handles part II (Remote Handles)

Posted by William Diaz on April 4, 2012


In addition to local processes locking up files and preventing their usage by other processes, files can also become locked by remote processes, too. Finding the remote system that has a handle(s) on the file can be a little bit more more involved. I mean this literally: a little a bit more involved. Knowing which tools to use or where to look can make this task just as simple as isolating it to a local process as outlined in part I.

This example is a recreation of an issue I encountered while working on a workstation remotely. It manifested itself as a failure to logon as the user: “Windows cannot load the locally stored profile…”

Posted in Troubleshooting Tools | Tagged: , | 1 Comment »

Quickly Find Those File Handles

Posted by William Diaz on April 3, 2012


Every now and then some app somewhere is going to hook into some user file and prevent it from loading when its host application is opening up, resulting in some kind of error. A common one is when opening Outlook: “The file C:\Documents and Settings\username\Local Settings\Application Data\Microsoft\Outlook\outlook.ost is in use and could not be accessed. Close any application that is using this file, and then try again.
image

Read the rest of this entry »

Posted in Troubleshooting Tools | Tagged: , | 1 Comment »

Resolve Symbols in Process Explorer-Monitor Without Installing the Debugging Tools

Posted by William Diaz on January 31, 2012


Sometimes when you are troubleshooting with Process Explorer, it’s helpful to be able to view functions in threads to isolate a problem. The same goes for Process Monitor when viewing the Stack tab in the properties of an operation. By default, Process Explorer and Process Monitor will point to the dbghelp.dll in the windows\system32 folder, but this is a stripped down version and doesn’t support symbol server functionality. Instead, you need the dbghelp.dll from the Windows Debugging Tools to properly resolve symbols, otherwise you will encounter the Process Explorer Warning box when you click the Threads tab in process properties:
image
Read the rest of this entry »

Posted in Uncategorized | Tagged: , , | 1 Comment »

The Case of the IE Hangs and Missing PNG Images (or Killing Two Birds with One Stone)

Posted by William Diaz on January 9, 2012


The initial issue I was asked to look at started with Internet Explorer failing to gracefully exit. Instead, it would just hang and required manual intervention via the Task Manager to kill the iexplore.exe process. I connected remotely to the workstation and ran Process Explorer so I could examine IE’s threads for anything out of the ordinary. Sure enough, I saw the presence of a .tmp file:
Read the rest of this entry »

Posted in Troubleshooting | Tagged: , , , , , | 5 Comments »

Exploring Video-Graphics Performance – Event IDs 500 and 501

Posted by William Diaz on December 8, 2011


If you had a chance to review the Diagnostics-Performance logs, you may have encountered warning events coming from event ID 500 “The Desktop Window Manager is experiencing heavy resource contention. Video memory resources are over-utilized and there is thrashing happening as a result…” or 501 “…Graphics subsystem resources are over-utilized.”
Read the rest of this entry »

Posted in Inside Windows, Troubleshooting Tools | Tagged: , | 2 Comments »