I am a big fan of debugging tools, especially WinDbg. These tools can be quite complicated but most of the time I use them simply for a “one-dimensional” analysis of an application or system crash and use the information from the dump to find a cause and hopefully the most logical fix. In the example here, a user complains that PowerPoint-Excel is often crashing or becoming hung. I was able to locate a couple of errors in the Event Viewer > Application Logs that indicated a crash had taken place within the MS Office product, but it was too generic and failed to resolve to the failing component. I asked the user to manually create a crash dump for me the next time the issue happened by using the Task Manager > Applications > right-clicking the unresponsive PowerPoint or Excel application > Select Create Dump File. The dump file gets sent to the user’s temp directory, e.g. C:\Users\username\AppData\Local\Temp\AppName.DMP.
I copied the dmp and opened with with WinDbg. Note, if you are analyzing a 32bit application dump (often the case with MS Office – because who actually uses 64bit office?) you will want to use the 32bit version of WinDbg. Or, if you are incredibly lazy like me (can’t figure out what happened to my x86 version), you can use the 64bit WinDbg and change it use x86-based processor mode. Just run .load wow64exts followed by .effmach x86, followed by the always faithful !analyze –v –hang.
This resulted in following output (Including only the stack text since everything before this is not useful) :
|
STACK_TEXT:
00392774 75751605 00000001 003927c4 00000001 ntdll_776f0000!ZwWaitForMultipleObjects+0x15
00392810 769619f8 003927c4 00392838 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x100
00392858 5e558ac9 00000001 7efde000 00000000 kernel32!WaitForMultipleObjectsExImplementation+0xe0
WARNING: Stack unwind information not available. Following frames may be wrong.
00392898 5e58f314 00000001 02b68204 ffffffff MSO!Ordinal10580+0x28e
003928d0 5e589c1c 00000000 003928ec 02b69140 MSO!Ordinal4721+0x114
00392900 5e58a80a 02421880 ffffffff 02b68200 MSO!Ordinal5739+0x18a
00392914 2d5b9771 02421880 2d4d05b6 00392948 MSO!Ordinal9231+0x2d6
00392924 2d4d1ebe 00000001 2dc983fc 00000000 PPCORE!DllGetLCID+0xc2eb7
00392948 2d5666de 00000001 84008571 0af19000 PPCORE!PPMain+0x4b010
0039427c 2d5660c3 02b69140 00000425 84009249 PPCORE!DllGetLCID+0x6fe24
00395544 2d563d9f 8400a15d 003966d4 02c40000 PPCORE!DllGetLCID+0x6f809
00396650 2d4a2c41 02c40000 2d4a2be8 02c40000 PPCORE!DllGetLCID+0x6d4e5
00396668 2d5c949b 003966d4 00000000 00000000 PPCORE!PPMain+0x1bd93
00399a64 2d5c90e3 00399a80 84005dbd 0af1928c PPCORE!DllGetLCID+0xd2be1
00399ab0 2d7beff6 0af19000 00399b7c 00399b08 PPCORE!DllGetLCID+0xd2829
0039be34 5f1ea45e 0039bf00 5f1ec17c 76a3030c PPCORE!DllGetLCID+0x2c873c
0039be48 769a03ab 0039bf00 8405ee7f 00000000 MSO!MsoCrashMainThread+0x21f
0039bed0 77763b8f 0039bf00 77763a6c 00000000 kernel32!UnhandledExceptionFilter+0x127
0039bed8 77763a6c 00000000 0039f934 7771c500 ntdll_776f0000!__RtlUserThreadStart+0x62
0039beec 77763911 00000000 00000000 00000000 ntdll_776f0000!_EH4_CallFilterFunc+0x12
0039bf14 777533dd fffffffe 0039f924 0039c050 ntdll_776f0000!_except_handler4+0x8e
0039bf38 777533af 0039c000 0039f924 0039c050 ntdll_776f0000!ExecuteHandler2+0x26
0039bf5c 77753350 0039c000 0039f924 0039c050 ntdll_776f0000!ExecuteHandler+0x24
0039bfe8 77700133 0139c000 0039c050 0039c000 ntdll_776f0000!RtlDispatchException+0x127
0039bff4 0039c000 0039c050 c0000005 00000000 ntdll_776f0000!KiUserExceptionDispatcher+0xf
0039c4b4 588a6fa9 00000001 00000800 0a346c84 0x39c000
0039c4cc 588a6f52 0a346cac 0039c4e8 5e567954 GFX!Ordinal818+0x1cf
0039c4f4 588a6ea1 00000020 00000800 00000008 GFX!Ordinal818+0x178
0039c5a0 588a6e0a 004066c0 0039c990 00000001 GFX!Ordinal818+0xc7
0039c5cc 51333fb1 0039c644 0039c990 00000001 GFX!Ordinal818+0x30
0039c638 51333f12 00000001 004066c0 0039c990 OART!Ordinal910+0x4f8
0039c668 63818573 0039c8ac 00000000 00000001 OART!Ordinal910+0x459
0039c8d8 6381848e 0039c910 00000000 00000001 RICHED20!RichListBoxWndProc+0xb6d2
0039c918 6aebac5f 05755318 0578b628 00000000 RICHED20!RichListBoxWndProc+0xb5ed
0039c984 6aebadf3 05757e58 00000020 00000000 MSPTLS!LssbFIsSublineEmpty+0xb13c
0039c9a8 6aeb1aea 0000000d 00000004 0039cae8 MSPTLS!LssbFIsSublineEmpty+0xb2d0
0039ca58 6aeb2295 05755f08 0039cd58 0001165b MSPTLS!LssbFIsSublineEmpty+0x1fc7
0039cae0 6aebf7fb 00000000 0039cd58 00000000 MSPTLS!LssbFIsSublineEmpty+0x2772
0039cb44 6aebff10 05755cc8 00000000 0039cd74 MSPTLS!LssbFIsSublineEmpty+0xfcd8
0039cb74 6aebf0cc 00000000 00000000 0c6651f0 MSPTLS!LssbFIsSublineEmpty+0x103ed
0039cd80 6aea4c98 057556e8 00000111 00010dec MSPTLS!LssbFIsSublineEmpty+0xf5a9
0039cdb4 6380f006 057556e8 00000111 00010dec MSPTLS!LsCreateLine+0x23
0039ced4 638005df 00000003 00000000 ffffffff RICHED20!RichListBoxWndProc+0x2165
0039cf2c 638002da 0039d894 00000003 00000000 RICHED20!IID_ITextServices2+0x65f3
0039cf68 63816f3b 0039d894 00000003 00000000 RICHED20!IID_ITextServices2+0x62ee
0039cf94 63816aa1 0c694c00 00000111 ffffffff RICHED20!RichListBoxWndProc+0xa09a
0039cfe4 63816987 00000001 ffffffff 00000111 RICHED20!RichListBoxWndProc+0x9c00
0039d060 6af33a5d 057552c0 00000009 00000009 RICHED20!RichListBoxWndProc+0x9ae6
0039d0f0 6af3ec94 0577ffe8 00000009 00000009 MSPTLS!LsLwMultDivR+0x13786
0039d1a8 6af3854d 0577ffe8 00000000 00000000 MSPTLS!LsLwMultDivR+0x1e9bd
0039d264 6af39b13 0c6534f8 00000000 00000000 MSPTLS!LsLwMultDivR+0x18276
0039d388 6af185ed 0039d448 0c6534f8 0039d49c MSPTLS!LsLwMultDivR+0x1983c
0039d404 6af18f8a 0039d448 0c6534f8 0000ad40 MSPTLS!FsTransformBbox+0xf44e
0039d4ec 6af316a1 0577ffe8 00000000 0000ad40 MSPTLS!FsTransformBbox+0xfdeb
0039d55c 6af31913 0c6534f8 00000009 00000004 MSPTLS!LsLwMultDivR+0x113ca
0039d5f0 6af176b9 00000001 0c6534f8 0c652544 MSPTLS!LsLwMultDivR+0x1163c
0039d644 6af0d8a1 0c652528 00000000 00000000 MSPTLS!FsTransformBbox+0xe51a
0039d6d0 6af0decb 00000000 0c6534f8 00000000 MSPTLS!FsTransformBbox+0x4702
0039d79c 6af02099 0577fa10 00000000 0039d874 MSPTLS!FsTransformBbox+0x4d2c
0039d7b4 63814a25 0577fa10 00000000 0039d874 MSPTLS!FsCreatePageBottomless+0x2d
0039d850 6381478b 0039d874 00010dec 0c6af128 RICHED20!RichListBoxWndProc+0x7b84
0039d87c 638145ef 057552c0 00010dec 0c6af128 RICHED20!RichListBoxWndProc+0x78ea
0039d9bc 63813bac 3fffffff 00400000 0c679bb8 RICHED20!RichListBoxWndProc+0x774e
0039dca0 6380e3b1 0039dcf0 00000185 00000185 RICHED20!RichListBoxWndProc+0x6d0b
0039dcd0 637fa38d 0039dcf0 00000185 00000185 RICHED20!RichListBoxWndProc+0x1510
0039dd4c 63812db5 0578eaec 0039dd8c 63812b45 RICHED20!IID_ITextServices2+0x3a1
0039dd58 63812b45 0c67dd40 0039dda0 00000000 RICHED20!RichListBoxWndProc+0x5f14
0039dd8c 51332427 0039ddcc 0e0d0dd0 0e0d0dd8 RICHED20!RichListBoxWndProc+0x5ca4
0039de50 51332095 0050b0d0 0dc6de00 0dc6def8 OART!Ordinal7653+0x502
0039de68 51331e34 0e0d0c40 0039de88 84001d0e OART!Ordinal7653+0x170
0039df84 5132f344 0e0d0c40 0dc6de60 0e0dc0c4 OART!Ordinal525+0xb7b
0039dfc4 51337c8c 0039e058 0039dffc 51337c59 OART!Ordinal6914+0x2c0
0039dfd0 51337c59 0dc6de00 84001d76 0dc6de00 OART!Ordinal3680+0xe8f
0039dffc 5133b804 0dc6de00 840022a2 0dc6de00 OART!Ordinal3680+0xe5c
0039e028 5133b7a6 0dc6de00 840022e6 0dc6de00 OART!Ordinal5150+0x240
0039e06c 513aeed3 0000000e 0039e088 00000000 OART!Ordinal5150+0x1e2
0039e090 513af827 0039e138 840023c2 027fe980 OART!Ordinal389+0x119
0039e148 518017a0 007fe964 0000b298 0039e198 OART!Ordinal1309+0x69d
0039e174 2d803b5a 02808460 0039e198 0039e1a0 OART!Ordinal5501+0x2d
0039e1c8 2d606bb6 02808464 0039e47c 840023a5 PPCORE!DllGetLCID+0x30d2a0
0039e4a8 2d4d2181 02220b44 02808464 0039e4c8 PPCORE!DllGetLCID+0x1102fc
0039e4b8 2d4d1cb4 02220a00 029a5600 0039e4fc PPCORE!PPMain+0x4b2d3
0039e4c8 2d4cf07c 00000004 029a5600 00000000 PPCORE!PPMain+0x4ae06
0039e4fc 2d4cf049 00000004 029a5600 0039e51c PPCORE!PPMain+0x481ce
0039e50c 2d4cf02a 00000004 0285c860 0039e52c PPCORE!PPMain+0x4819b
0039e51c 2d4ced04 00000004 0039e594 0039e564 PPCORE!PPMain+0x4817c
0039e52c 2d49492f 00000004 84002269 2df23040 PPCORE!PPMain+0x47e56
0039e564 2d49385d 2df23040 005849e0 00000000 PPCORE!PPMain+0xda81
0039e578 2d4937ea 0039e594 840022cd 2df23040 PPCORE!PPMain+0xc9af
0039e5c0 2d486f1e 84003f59 2df23040 005849e0 PPCORE!PPMain+0xc93c
0039f854 2df215ae 2df21575 2df20000 00000000 PPCORE!PPMain+0x70
0039f8e8 7696338a 7efde000 0039f934 777297f2 POWERPNT+0x15ae
0039f8f4 777297f2 7efde000 767616c8 00000000 kernel32!BaseThreadInitThunk+0xe
0039f934 777297c5 2df210c8 7efde000 ffffffff ntdll_776f0000!__RtlUserThreadStart+0x70
0039f94c 00000000 2df210c8 7efde000 00000000 ntdll_776f0000!_RtlUserThreadStart+0x1b
|
Stacks are read from the bottom up so you can get a very basic idea of what is happening as you climb the ladder. We are really only interested in the component details … and maybe the method-routine of that component (assuming you have the symbols). For example, we start with the main PowerPoint component, PPCORE (everything after the ! is the method or routine). That calls into OART, which is the Office Art components of the MS Office suite, which goes into RICHED20, which I am guessing is something like rich text box component, then to another office component, MSPTLS, and again into another MS Office art component, GFX. After GFX you can see a couple exceptions (errors) are encountered before passing back into the main PowerPoint dll.
Hint: if you are not sure what these components are or belong to, open a new command browser in WinDbg and use the lmvm command to get the details of it, e.g. lmvm GFX:
|
start end module name
588a0000 58a4d000 GFX (export symbols) GFX.DLL
Loaded symbol image file: GFX.DLL
Image path: C:\Program Files (x86)\Microsoft Office\Office14\GFX.DLL
Image name: GFX.DLL
Timestamp: Fri Jun 28 18:43:07 2013 (51CE117B)
CheckSum: 001B6FED
ImageSize: 001AD000
File version: 14.0.7104.5000
Product version: 14.0.7104.0
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 1.0 App
File date: 00000000.00000000
Translations: 0000.04e4
CompanyName: Microsoft Corporation
ProductName: Microsoft Office 2010
InternalName: GFX
OriginalFilename: GFX.DLL
ProductVersion: 14.0.7104.5000
FileVersion: 14.0.7104.5000
FileDescription: Microsoft OfficeArt
LegalCopyright: © 2010 Microsoft Corporation. All rights reserved.
|
So, from the stack above, my first guess is that we have an issue with some graphic module(s) – and maybe to a greater part the video subsystem (not necessarily the fault of MS Office). To confirm, I can look at all the thread stacks by dumping them with ~*kb. Reading from the bottom up, it looks like the problem starts with thread 15 (similar to what !analyze –v –hang shows):
|
15 Id: 944.1ffc Suspend: 0 Teb: 7efd8000 Unfrozen
ChildEBP RetAddr Args to Child
0e9ec8c8 7772c81a 00000bd4 00000000 00000000 ntdll_776f0000!ZwWaitForSingleObject+0x15
0e9ec92c 7772c6fe 00000000 00000000 0000000e ntdll_776f0000!RtlpWaitOnCriticalSection+0x13e
0e9ec954 637f3de0 63913530 05750f1c 637f837e ntdll_776f0000!RtlEnterCriticalSection+0x150
WARNING: Stack unwind information not available. Following frames may be wrong.
0e9ec974 63809412 0000000e ffffffff 057542c8 RICHED20+0x3de0
0e9ec98c 637fa8a3 ffffffff 00000000 0c66df94 RICHED20!IID_ITextServices2+0xf426
0e9eca10 6380a7d3 ffffffff 00000016 0c66df94 RICHED20!IID_ITextServices2+0x8b7
0e9eca38 637f9c7d 00000000 00000016 00000016 RICHED20!IID_ITextServices2+0x107e7
0e9eca5c 637f951a 0e9ecd64 00000000 00000016 RICHED20!IID_ITextHost+0xaf1
0e9ecb20 637ff8c9 00000016 00000016 0e9ece80 RICHED20!IID_ITextHost+0x38e
0e9ecbc4 63806564 00000016 0e9ece80 00000000 RICHED20!IID_ITextServices2+0x58dd
0e9ed000 63805843 0e9ed21c 0e9ed290 0c6660e4 RICHED20!IID_ITextServices2+0xc578
0e9ed1fc 638057bf 00000527 0e9ed22c 0e9ed21c RICHED20!IID_ITextServices2+0xb857
0e9ed240 51330ed8 0e9ed290 009ed274 51330ee8 RICHED20!IID_ITextServices2+0xb7d3
0e9ed254 51330821 0e9ed290 0e9ed274 8aa7103a OART!Ordinal582+0x86a
0e9ed2b0 5133e25c 02ec3c40 00000000 00000017 OART!Ordinal582+0x1b3
0e9ed2f4 5133e0a9 02ec3c40 8aa711ea 00000001 OART!Ordinal3092+0x234
0e9ed360 5133de56 0e9ed43c 8aa71112 00000001 OART!Ordinal3092+0x81
0e9ed398 5156385e 0e9ed43c 8aa71c22 02caed40 OART!Ordinal5804+0x49
0e9edea8 51c8ac06 027fdda8 000da976 001ecfe4 OART!Ordinal3710+0x1f0b
0e9ee010 51c8e83f 0e9ee1ac 0e9ee194 02d47a00 OART!Ordinal3134+0xdad1
0e9ee1e0 51c8af32 0000005e 02d47a00 0e9ee2e4 OART!Ordinal3134+0x1170a
0e9ee1f0 51c84008 0000005e 8aa7206e 0dc531d8 OART!Ordinal3134+0xddfd
0e9ee2e4 51c8477e 029b7e00 0dc53000 00000000 OART!Ordinal3134+0x6ed3
0e9ee338 51c76c9c 00000000 0dc6d000 029b7e00 OART!Ordinal3134+0x7649
0e9ee358 51c53869 00000000 0dc6d000 029b7e00 OART!Ordinal7663+0x111f5
0e9ee388 51c53d8f 029b7e00 0af6a780 00000000 OART!Ordinal5848+0x31bbe
0e9ee46c 51c541f7 01f6a780 00000007 00000000 OART!Ordinal5848+0x320e4
0e9ee494 51c513be 0af6a780 ffffffff 00000001 OART!Ordinal5848+0x3254c
0e9ee510 5166ed08 00000001 023d5100 00000000 OART!Ordinal5848+0x2f713
0e9ee650 5157b3e7 fffffffe 00000000 0dc70500 OART!Ordinal4701+0x7d347
0e9ee6b8 5157af1c 027fdda8 027fdd80 0e9ee798 OART!Ordinal1690+0x6a98
0e9ee750 51565415 0e9ee790 0e9eed80 027fde94 OART!Ordinal1690+0x65cd
0e9ee764 5156534d 00000000 0e9eed80 027fde94 OART!Ordinal2289+0xadd
0e9eebd4 515651db 0e9eed80 00000000 0e9eecf8 OART!Ordinal2289+0xa15
0e9eed2c 51564f8e 003d9fc8 00000000 00000000 OART!Ordinal2289+0x8a3
0e9eeedc 51564b24 0e9eeefc 00000000 8aa72d96 OART!Ordinal2289+0x656
0e9eef1c 51307adf 0e9eefa0 0e9ef050 0e9eefc4 OART!Ordinal2289+0x1ec
0e9eef2c 513077ae 0e9eefa0 8aa72d4e 00000000 OART!Ordinal4963+0x46
0e9eefc4 5132afe4 0e9ef050 0e9ef068 8aa732ea OART!Ordinal2541+0xfd
0e9ef060 5132afe4 02b61720 0e9ef104 8aa73276 OART!Ordinal4433+0x3e9
0e9ef0fc 5132afe4 02b615d0 0e9ef1a0 8aa73312 OART!Ordinal4433+0x3e9
0e9ef198 513075f0 02b613f0 0e9ef1d8 8aa7335a OART!Ordinal4433+0x3e9
0e9ef1d0 51303605 02b6aab0 02de3c84 8aa7308e OART!Ordinal2385+0x142
0e9ef204 513035b2 02a95dc0 00000000 51303401 OART!Ordinal2513+0xde
0e9ef240 2d4d96eb 8aa7342d 0e9ef560 0e9ef4cc OART!Ordinal2513+0x8b
0e9ef320 2d4d941b 03382e44 0e9ef240 af01221d PPCORE!PPMain+0x5283d
0e9ef404 2d4d91af 0e9ef338 0e9ef530 af01221d PPCORE!PPMain+0x5256d
0e9ef554 2d4d8fbc 027fd180 027fd180 00000002 PPCORE!PPMain+0x52301
0e9ef5a4 2d4d8d6f 027fd180 0e9ef578 00000002 PPCORE!PPMain+0x5210e
0e9ef604 2d4d8c64 0e9ef5c8 0e9ef630 02423788 PPCORE!PPMain+0x51ec1
0e9ef644 2d4d8b76 8aa7317d 0ae0ef20 2dcaba78 PPCORE!PPMain+0x51db6
0e9ef670 2d4d8b43 8aa73191 0ae0aa94 0ae0aa84 PPCORE!PPMain+0x51cc8
0e9ef69c 2d48f094 0e9ef6c3 8aa731d9 0e9ef7b8 PPCORE!PPMain+0x51c95
0e9ef6d4 2d48f053 0e9ef720 5e574ea6 0ae0aa80 PPCORE!PPMain+0x81e6
0e9ef6dc 5e574ea6 0ae0aa80 02421880 8a953774 PPCORE!PPMain+0x81a5
0e9ef720 5e556cf0 02421880 0e9ef7b8 02859214 MSO!Ordinal3920+0x30
0e9ef740 5e556be1 0e9ef7b8 00000000 0e9ef79c MSO!Ordinal6535+0x5cb
0e9ef75c 5e553ba1 0e9ef7b8 00000000 003fa7f0 MSO!Ordinal6535+0x4bc
0e9ef790 5e551d40 003fa7f0 00000000 5e551d40 MSO!Ordinal3125+0x1f0
0e9ef7ec 7696338a 003fa7f0 0e9ef838 777297f2 MSO!MsoFRegGetDwCore+0x26f
0e9ef7f8 777297f2 003fa7f0 78d117c4 00000000 kernel32!BaseThreadInitThunk+0xe
0e9ef838 777297c5 5e551ce8 003fa7f0 ffffffff ntdll_776f0000!__RtlUserThreadStart+0x70
0e9ef850 00000000 5e551ce8 003fa7f0 00000000 ntdll_776f0000!_RtlUserThreadStart+0x1b
|
Immediately after, thread 14 reveals Direct 3D 9 runtime component, which is the core multimedia APIs from Microsoft:
|
14 Id: 944.218c Suspend: 0 Teb: 7ef12000 Unfrozen
ChildEBP RetAddr Args to Child
0fe7f8bc 757514b9 00000ba4 00000000 00000000 ntdll_776f0000!ZwWaitForSingleObject+0x15
0fe7f928 76961194 00000ba4 ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x98
0fe7f940 76961148 00000ba4 ffffffff 00000000 kernel32!WaitForSingleObjectExImplementation+0x75
0fe7f954 668f801f 00000ba4 ffffffff 00000000 kernel32!WaitForSingleObject+0x12
0fe7f970 668f8073 0fe7f984 7696338a 0eef0040 d3d9!CBatchFilterI::WorkerThread+0x24
0fe7f978 7696338a 0eef0040 0fe7f9c4 777297f2 d3d9!CBatchFilterI::LHBatchWorkerThread+0xd
0fe7f984 777297f2 0eef0040 79a81638 00000000 kernel32!BaseThreadInitThunk+0xe
0fe7f9c4 777297c5 668f8066 0eef0040 ffffffff ntdll_776f0000!__RtlUserThreadStart+0x70
0fe7f9dc 00000000 668f8066 0eef0040 00000000 ntdll_776f0000!_RtlUserThreadStart+0x1b
|
At this point, I decide to check for a later version of the graphics drivers for the users computer and found a newer driver release. Install, reboot, and no further Office applications reported since. Altogether, the process of identifying the basic issue itself took no more than 10 minutes, all without any software engineer credentials.
Like this:
Like Loading...
Related
Windows Explored 
Todd Walker said
HI,
How are you able to determine that it Direct 3d 9 runtime component that was causing the issue in all the debugging output?
Thanks
William Diaz said
in the last thread, component d3d9!