The call to LoadLibrary(sos2) failed, Win32 error 0n14001, “This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.”

Mike Semikin has reported this issue while executing windbg command .load sos2

The call to LoadLibrary(sos2) failed, Win32 error 0n14001
“This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.”
Please check your debugger configuration and/or network access.

Resolution
This is because visual studio 2008 is used as a development environment for sos2 and CRT is installed in assembly cache(winSxS) folder so dependent dll will be missing

As an immediate workaround, please install visual c++ 2008 runtime redistributable from Microsoft,  download from http://www.microsoft.com/Downloads/details.aspx?familyid=9B2DA534-3E03-4391-8A4D-074B9F2BC1BF&displaylang=en

Windows Forms – System.OutOfMemoryException: Screen-compatible bitmap cannot be created. The screen bitmap format cannot be determined.

Description

Occasionally, Windows Forms application user will get an error message as shown in below snapshot.
screen bitmap error

Analysis

If you have done win32 development, you will immediately suspect GDI handles. I would like to point out the steps which you can take to troubleshoot and confirm if there is a GDI Leak in a process.

When a user reports this issue which is not reproducible at will, in that case you can ask user to launch task manager and go to view->select columns and check Handles and GDI Handles. Even better, if you can use process explorer.

Things to remember

1. 32 bit Windows OS(XP and above) uses DWORD which is 4 bytes for either user handle or gdi handle. Most significant 16 bits are used for GDI handles so 2^16 = 65536 is a theoretical limit on number of GDI or USER handles a Win32 Process can have.

2. Default value of number of handles configured in a 32 bit operating system are 10k which is configurable in the registry

3. “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\GDIProcessHandleQuota” has the default value of GDI handles

4. “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\USERProcessHandleQuota” has the default value of User Handles

5. GDI handles are not just bitmaps, but it can be any device context object for example, Font, Pen, DC etc

I have not dissected GDI Handle yet so I can’t get into too much detail on GDI data structure on Windows XP. However, another point to remember is GDI kernel data structure is stored in a system paged pool which is a physical memory resident so what that means is although the theoretical limit on number of GDI handles is almost 65k per process but that doesn’t mean your process can have 65k GDI handles so before changing the registry you need to think twice and then twice again, does your application really need more than 10k GDI Handles?


Below is the number of GDI Handles as shown in a process explorer’s snapshot taken at customer site when this error occurred

screen bitmap error

As usual, we need to collect the hang dump at the time of this exception


1. I really don’t know how to get the list of gdi handles from a memory dump, there used to be a gdi windbg extension dll from microsoft but that has been discontinued with windows xp.


2. if you look at process explorer snapshot as shown above, you will notice that the number of GDI Handles are 9999 and after that it fails to create the compatible bitmap.


3. Let’s open the hang dump and you can still run !analyze -v command to get the exception record


EXCEPTION_OBJECT: !pe 1a2b14ec
Exception object: 1a2b14ec
Exception type: System.OutOfMemoryException
Message: Screen-compatible bitmap cannot be created. The screen bitmap format cannot be determined.
MANAGED_STACK:
SP               IP                                      Function
0012E46C  7AE32C56
System_Drawing_ni!System.Drawing.BufferedGraphicsContext.bFillBitmapInfo(IntPtr, IntPtr, BITMAPINFO_FLAT ByRef)
…………………………………..
System_Drawing_ni!System.Drawing.BufferedGraphicsContext.Allocate(IntPtr, System.Drawing.Rectangle)
0:000> !ip2md 7AE32C56
MethodDesc: 7adff018
Method Name: System.Drawing.BufferedGraphicsContext.bFillBitmapInfo(IntPtr, IntPtr, BITMAPINFO_FLAT ByRef)
0:000> !dumpil 7adff018
ilAddr = 7aeb0f64
IL_0000: ldsfld System.IntPtr::Zero
IL_0005: stloc.0
IL_0006: ldc.i4.0
IL_0007: stloc.1
.try
{
IL_0008: ldnull
IL_0009: ldarg.1
IL_000a: newobj System.Runtime.InteropServices.HandleRef::.ctor
IL_000f: ldc.i4.1
IL_0010: ldc.i4.1
IL_0011: call System.Drawing.SafeNativeMethods::CreateCompatibleBitmap
IL_0016: stloc.0
IL_0017: ldloc.0
IL_0018: ldsfld System.IntPtr::Zero
IL_001d: call System.IntPtr::op_Equality
IL_0022: brfalse.s IL_0034
IL_0024: ldstr “GraphicsBufferQueryFail”
IL_0029: call System.Drawing.SR::GetString
IL_002e: newobj System.OutOfMemoryException::.ctor

IL_0033: throw


Dump the IL for the corresponding IP, you will notice that it throws System.OutOfMemoryException when Native Method call ( GDI32.dll!CreateCompatibleBitmap) returns IntPtr.Zero.


4. This confirms that we have an issue here with GDI Handles, since this is a managed application so we can find the System.Drawing, Font, Pen and other GDI objects surviving the Garbage Collection.


5. You can just dump GDI objects on managed heap and find out where the objects are rooted at


6.  In this particular case, the culprit was System.Windows.Forms.Control+FontHandleWrapper

Below is the finalizequeue stats

0:000> !finalizequeue
SyncBlocks to be cleaned up: 0
MTA Interfaces to be released: 0
STA Interfaces to be released: 0
———————————-
generation 0 has 34 finalizable objects (19826184->1982620c)
generation 1 has 6 finalizable objects (1982616c->19826184)
generation 2 has 84919 finalizable objects (197d3290->1982616c)
Ready for finalization 0 objects (1982620c->1982620c)

………………………………………………………………………………………………………………….

7b225228     9811       156976 System.Windows.Forms.Control+FontHandleWrapper
7b21e930     9816      1649088 System.Windows.Forms.TextBox
……………………………………………………………………………………………………………………
Total 84959 objects

and the call stack for HFONT gdi handle leak is belowBelow is the call stack to set a new Font object for every TextBox Control

System.Windows.Forms.Control+FontHandleWrapper..ctor(System.Drawing.Font)
System.Windows.Forms.Control.get_FontHandle()
System.Windows.Forms.Control.get_FontHandle()
System.Windows.Forms.Control.SetWindowFont()
System.Windows.Forms.Control.OnHandleCreated
This Windows Forms application maintains a collection of controls and a new font gets created everytime but never gets disposed along with the controls

And the root cause for Control not getting disposed is

022d28f8(System.Windows.Forms.ToolStrip)->
022d2644(MyApp.Controls.UserControl)->
022d6ed4(MyApp.MyPanel)->
022d84d0(System.Windows.Forms.ContextMenuStrip)->
021546f4(System.Windows.Forms.TextBox)->
02154608(System.Windows.Forms.PropertyStore)->

a. Form contains ContextMenuStrip member

b. Each of the created Control has a reference to ContextMenuStrip(Control.ContextMenuStrip = Form.ContextMenuStrip)

c. Even after a Control is removed, it still holds reference to ContextMenuStrip which is a member of Main Form

d. Unless Main Form is disposed, Controls will survive GC although Dispose is called on each of the Control on OnControlRemoved Event

e. This issue can be easily resolved by setting Control.ContextMenuStrip = null when a control is removed

Not reproducible Windows Forms Hang OnUserPreferenceChanged – If your application has hung once it will hang again

Problem Description

At one of our customer site, windows forms business application hung for a business user.  This issue  occurred only on one machine out of 100s of users. The way we look at it is, If your application has hung once on any machine that means it will hang again, so I volunteered to look into his machine armed with WinDBG.

Note

Executed command to check for syncblock/locks are for illustration purpose only

1. While debugging for hang/deadlock, the very first thing you need to do is look at the call stack for each thread

2.  If call to Systen.Threading.Monitor.Enter is at the top of the managed call stack and there is a corresponding *WaitFor* at the top of unmanaged call stack which you can get either by running command kb or execute !dumpstack to get managed/unmanaged both, in that case you can suspect deadlock in managed code and resolved it using !synblk command to find out which thread is owning the object

3. If the managed thread call stackdoesn’t have Thread.Monitor that means sycnblock will be pretty useless to find out the cause of deadlock

4. If thread has WaitOne or waiting on a handle in a call stack in that case, you should use dd command to find the handles for kernel handle deadlock or use !do command in managed stack along with !handle to find out which thread owns the handle

Analysis

1. Take a memory dump of the application while it is hung, no live debugging – least convenience to user

2. Open the dump file for analysis, since this is a pure managed app so first we will look at syncblock and then criticalsection locks

0:000> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info  SyncBlock Owner
—————————–
Total           546
CCW             4
RCW             17
ComClassFactory 0
Free            460
0:000> !locks

CritSec <Unloaded_PI32.dll>+a00a92f at 0a00a930
LockCount          0
RecursionCount     0
OwningThread       0
*** Locked

3. Everything is fine so far, let’s look at all the thread stacks and the waithandles on interesting thread

0:000> ~*e!clrstack
OS Thread Id: 0xcf8 (0)

ESP       EIP
0012e4c4 792b6835 System.Threading.WaitHandle.WaitOne(Int32, Boolean)
0012e4d8 7b6f192f System.Windows.Forms.Control.WaitForWaitHandle(System.Threading.WaitHandle)
0012e4ec 7ba2d0bb System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
0012e58c 7b6f328c System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
0012e5c0 7b6f322f System.Windows.Forms.Control.Invoke(System.Delegate)
………………………………………………………………………………………………………….

0012eda4 0167834e MyApp.UI.RunStartup()
0012eff0 7b1c8906 System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)

0012f3ac 7b195911 System.Windows.Forms.Application.Run(System.Windows.Forms.Form)
0012f3c0 01676974 MyApp.Program.RunSelectPlanForm()
0012f400 0187b525 MyApp.Program.Main(System.String[])

OS Thread Id: 0xf74 (9)
ESP       EIP

0710f204 792b6835 System.Threading.WaitHandle.WaitOne(Int32, Boolean)
0710f218 7b6f192f System.Windows.Forms.Control.WaitForWaitHandle(System.Threading.WaitHandle)
0710f22c 7ba2d0bb System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
0710f2cc 7b6f328c System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
0710f300 7b920717 System.Windows.Forms.WindowsFormsSynchronizationContext.Send(System.Threading.SendOrPostCallback, System.Object)
0710f34c 7a922833 Microsoft.Win32.SystemEvents.RaiseEvent(Boolean, System.Object, System.Object[])
0710f398 7a923d2f Microsoft.Win32.SystemEvents.OnUserPreferenceChanged(Int32, IntPtr, IntPtr)
0710f3b8 7aa8f184 Microsoft.Win32.SystemEvents.WindowProc(IntPtr, Int32, IntPtr, IntPtr)
0710f670 7b1d8781 System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)
0710f6a0 7b195911 System.Windows.Forms.Application.Run(System.Windows.Forms.Form)
0710f6b4 04edf271 MyApp.MyClass.Run()
0710f6e4 792d6c74 System.Threading.ThreadHelper.ThreadStart()

4. Just to illustrate, You can get the handles on thread 0 and 9 from stack trace

0:009> ~0kb
ChildEBP RetAddr  Args to Child
0012e07c 7c90df2c 7c809574 00000002 0012e0a8 ntdll!KiFastSystemCallRet
0012e080 7c809574 00000002 0012e0a8 00000001 ntdll!NtWaitForMultipleObjects+0xc
0012e11c 7e4195f9 00000002 0012e144 00000000 kernel32!WaitForMultipleObjectsEx+0x12c
0012e178 7752ebd6 00000001 0012e478 000003e8 user32!RealMsgWaitForMultipleObjectsEx+0x13e
0012e1a0 77557237 0012e478 000003e8 0012e1cc ole32!CCliModalLoop::BlockFn+0×80

0:009> ~9kb
ChildEBP RetAddr  Args to Child
0710edbc 7c90df2c 7c809574 00000002 0710ede8 ntdll!KiFastSystemCallRet
0710edc0 7c809574 00000002 0710ede8 00000001 ntdll!NtWaitForMultipleObjects+0xc
0710ee5c 7e4195f9 00000002 0710ee84 00000000 kernel32!WaitForMultipleObjectsEx+0x12c
0710eeb8 7752ebd6 00000001 0710f1b8 000003e8 user32!RealMsgWaitForMultipleObjectsEx+0x13e
0710eee0 77557237 0710f1b8 000003e8 0710ef0c ole32!CCliModalLoop::BlockFn+0×80

WaitForMultipleObjects take the array of handles and we know the array size from function parameters which is 2. We can use dd to display memory range since we know the size

0:009> dd 0012e144 L2
0012e144  00000ac0 00000038
0:009> dd 0710ee84 L2
0710ee84  00000200 000007c4

Both of these threads are waiting on different handles.

5. After reviewing the call stack again you will see that Thread 0 is calling Invoke to send a message to a window created on Thread 9 and Thread 9 is calling Invoke to send a message to a window created on Thread 0.  After posting the message, Thread 0 is waiting so message is not getting pumped on thread 0 and the same applies to Thread 9.  When you look at MarshaledInvoke implementation , after posting a message it waits on a event handle.

Why did this occur?

Thread 0 is the main UI thread and Thread 9 is a worker thread creating a splasher screen and running in STA apartment to pump message, below is the code snippet which creates a new thread and message loop for splasher on a worker thread.

SplashThread = new Thread(threadStart);
SplashThread.IsBackground = true;
SplashThread.SetApartmentState(ApartmentState.STA);
SplashThread.Start();
By default, Controls subscribe to UserPreferenceChanged Event and this event can get fired when system wide setting is changed, for example – changing the system settings including desktop background or a theme. It just so happened that business user changed his system settings when this app was running with an active splash screen and the timing was such that controls subscribed to UserPreferenceChanged was in the middle of processing a message on MainUI thread. Splasher thread sends a message to MainUI which is not processed yet and MainUI thread sends a message to splasher window to change status which results in a deadlock.

Please refer to a very good article on UserPreferenceChanged event and other gotcha/deadlock/issues which you can run into with a window created on worker thread