Windows Explored

Everyday Windows Desktop Support, Advanced Troubleshooting & Other OS Tidbits

Archive for December 3rd, 2015

The Case of the Phantom Proxy Settings

Posted by William Diaz on December 3, 2015


Not too long ago we began to notice that users who were opening IE after logging onto some workstations or Citrix servers they did not have a previously existing profile on were getting proxy settings applied in IE that were not being defined in any group policy object. Our local office proxies are applied later in the GPO processing so if it was anything coming before that, it should have been overridden by local GPO. The quick remedy was simply wait for policy to apply in the background or manually run gpupdate. But we still needed to address the cause as in some cases the proxy server was no longer reachable, preventing users from getting out to the Internet.

To get an idea of what might be happening, we deleted a profile from a machine and logged onto it. We then fired up Process Monitor and left it running while launching Internet Explorer for the first time. After verifying that incorrect proxy setting was getting applied, we stopped the trace and the log was sent my way. Proxy settings are applied in the registry so I knew I would be looking only at registry events, searching for the string of proxy server name in the log, and specifically looking at the operation for RegSetValue. From here, it was a simple matter of opening the Properties for that Event and going to the Stack tab.

clip_image001

The component setting the phantom proxy server (iedkcs.dll) was coming from the Internet Explorer Admin Kit or IEAK. This is otherwise known as Active Setup and its what IE will use when it is initially launched by a new user logon if the IE install was configured using IEAK. Also known as Branding, Active Setup for IE looks for INSTALL.INS if it defined in the registry. You can find this file in Program Files (x86)\Internet Explorer\CUSTOM. Looking in the INSTALL.INS, I could see under [ConnectionSettings] the phantom proxy server. You can read about the internals of this here: https://support.microsoft.com/en-us/kb/2029043

At first this was a bit baffling because I had setup the rollout for IE11 using the standalone IE11 MSU and required dependencies and not the IEAK. However, after glancing at the INSTALL.INS file again, I noticed this was actually coming from an earlier IE 9 deployment, which was created using IEAK by my IE predecessor:

image

To correct, I referred back to https://support.microsoft.com/en-us/kb/2029043 and concluded that this could be avoided by going to [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\>{2A2F9EE5-7FE3-44E8-AD75-72B5704DBCB4}] and deleting "StubPath"="RunDLL32 IEDKCS32.DLL,BrandIE4 CUSTOM".

To correct on new workstations going forward, I simply created a REG DELETE command in the Task Sequence after the IE11 setup to remove the value. For existing machines, a GPP registry is the easiest way to go.

Posted in Inside Windows, Troubleshooting Tools | Tagged: | Leave a Comment »

The Case of the Failed MSI

Posted by William Diaz on December 3, 2015


I recently began encountering a problem where one of our application packages was getting hung during the install process. I could see the program directory correctly populated with application files but it seemed to get hung at end, and msiexec processes continued to run indefinitely. After manually killing each process, I checked to see if the application actually completed installing. When trying to run one of the applications that has a dependency on one of the failed MSI’s APIs, I encountered the following:

image

Further, Outlook was crashing when trying to load the application’s main component.

Faulting application name: OUTLOOK.EXE, version: 14.0.7012.1000, time stamp: 0x514a1b69
Faulting module name: imFileSite.dll, version: 8.5.3006.93, time stamp: 0x52d811c0
Exception code: 0xc0000005
Fault offset: 0x0008637e
Faulting process id: 0xd18
Faulting application start time: 0x01d12d4e055d29fc
Faulting application path: C:\Program Files (x86)\Microsoft Office\Office14\OUTLOOK.EXE
Faulting module path: C:\Program Files (x86)\Interwoven\WorkSite\iOutlook\imFileSite.dll

 

My guess was that the application’s components were not getting registered. To confirm, I manually registered all the components in the program directly via the command line:

FOR /R "C:\Program Files (x86)\ApplicationFolder" %G IN (*.dll) DO "%systemroot%\system32\regsvr32.exe" /s "%G").

Afterwards, the application with the dependency launched without error and Outlook was no longer crashing. However, the root issue still needed to be identified as I had no idea what else in the MSI was failing to complete. After manually deleting the components from the file system (it was missing from Programs and Features), I ran the MSI again and this time used Process Explorer to look inside the hung msiexec processes. There were multiple processes running, but I could see one process, though, eating CPU resources, and this was likely where I might find the culprit:

image

Opening the process details, I select the Threads tab. The thread that is doing all the work (or in this case getting hung up) is indicated by the Cycles tab. From here, I opened the thread by selecting it and clicking Stack:

image

Stacks are read from the bottom up. You can see a 3rd party component here (HCApi – McAfee HIPS) at play. Note, this might not be entirely abnormal as you can always expect any anti-virus suite to be hooking itself into any number of processes. So, to confirm what I was seeing, I used the Task Manager to dump the msiexec process that was using the CPU time by right-clicking it and select Create dump file (which can also be done via Process Explorer):

image

Dumps can be analyzed with WinDbg or a tool like DebugDiag 2.0. I have grown increasingly lazy and forgetful over the years with WinDbg. It has a very high learning curve and if you don’t use it much, its easy to forget everything except the old faithful !analyze –v or !analyze –v –hang. I used DebugDiag instead. When it is installed, simply right-click the dump file and select Analyze Crash/hang Issue from the context menu and point it to the crash file.

image

image

It will do its best to figure out the issue in the most vague way and you almost always need to do some interpretation of your own. For the most part, the analysis summary can be ignored.

image

A little bit down in the report points me to what I was seeing in Process Explorer with the problem thread that was using all the CPU time:

image

You can click the Thread ID like a hyperlink to follow it down in the report, expanding the thread. It reveals a familiar site. However, there is a bit more insight as the entry point is revealed and I can see the problem has something to do with the CustomAction table of the MSI itself.

image

Through a quick process of elimination using Orca to remove some of the rows in the CustomAction table, I narrow down the cause specifically down to action ISSelfRegisterCosting.

image

Without that row, the MSI install completes normally; or so at least it seems. I have no idea what removing this action might have elsewhere so this merely a hack. To further confirm the problem is being caused by McAfee, I perform the install on a virtual machine where McAfee is not installed and it proceeds normally. I then reach out to our McAfee enterprise admin and ask him to disable HIPS on one of my workstation. After doing so, the MSI runs and the application installs as expected. He inherits the problem.


Update

This has since been corrected by MacAfee with a HIPS update. One of the DLLs trying to get registered was getting blocked.

Posted in Troubleshooting, Troubleshooting Tools | Tagged: , | 9 Comments »