Scenario
From time to time, asp.net application response is very slow on Windows Server 2003

Some rants and the resolution

After turning on recycle events, logged message in application event log was Event message: Application is shutting down. Reason: Unknown. Slow response is always timed with this message in the application event log so that confirmed that Application Pool is terminating so no wonder asp.net response is slow from time to time.

However, the only missing piece was why? Since, the Reason is unknown :-). This application pool is configured for web garden with 6 app pools in it so we decided to attach debugger in production box to 2 worker processes.

If you are just starting out with debugging or have not read John Robbins Book on debugging, I would like to stress the followings when using debugger in production environment

1. By Default, ADPlus  writes the call stack on first-chance exception. Walking call stack also results in Symbol loading, symbol loading along with the stack walking causes a performance hit when a debugger is attached. The last thing you want in production environment is to cause performance hit because of  debugger.

2. Don’t just use ADPlus script to attach a debugger to the worker process by name because it will attach the debugger to each worker process in your production server causing  further performance hit.

3. Don’t use DebugDiag in production environment unless you really have a good reason for it.

The reason I am stressing about these points is because I noticed someone following all the worst practices in production environment while debugging.

The best practice is to use adplus configuration file to attach to a worker process and preferably by worker process id.

Identifying the issue

First of all, I would like to mention that Tess has excellent blog on AppDomain recycle http://blogs.msdn.com/tess/archive/2006/08/02/asp-net-case-study-lost-session-variables-and-appdomain-recycles.aspx

We just needed to find out the reason for AppPool shutdown.

Our goal is to attach a debugger to one worker process to generate a memory dump on crash and another goal is, ctrl+c in debugger console should not kill the worker process. We used optimized ADPlus Config file for production environment as shown below

<ADPlus>
<Settings>
<RunMode>CRASH</RunMode>
<Option>Quiet</Option>
<OutputDir> c:\dumps </OutputDir>
<ProcessID> 2684 </ProcessID>
</Settings>
<Exceptions>
<Option>   NoDumpOnFirstChance    </Option>
<Config>
<Code>    AllExceptions    </Code>
<Actions1>     Log     </Actions1>
<Actions2>    FullDump;  </Actions2>
<ReturnAction1>    GN    </ReturnAction1>
<ReturnAction2>   Q   </ReturnAction2>
</Config>
</Exceptions>
</ADPlus>

once we have the crash dump, just dump the httpruntime object to find out the shutdown reason and the call stack as shown below

internal void StartListeningToLocalResourcesDirectory(VirtualPath virtualDir)

{
    if (!this.IsFCNDisabled && ((this._callbackRenameOrCriticaldirChange != null) && (this._dirMonSpecialDirs != null)))

    {

[DllImport("webengine.dll")]
internal static extern void GetDirMonConfiguration(out int FCNMode);

Since, HttpRuntime controls the file monitoring so get the DirectoryMonitor object and calls StopMonitoring on it. Another way to probably implement is getting the FileChangesMonitor object and setting the FCNMode to 1 or setting the _dirMonSpecialDirs to null.

protected void Application_Start(object sender, EventArgs e)
{
System.Reflection.PropertyInfo p = typeof(System.Web.HttpRuntime)

Twitter This

Username: Password:

0:000> .loadby sos coreclr
The call to LoadLibrary(c:\Program Files (x86)\Microsoft Silverlight\3.0.40624.0\sos) failed, Win32 error 0n193
“%1 is not a valid Win32 application.”
Please check your debugger configuration and/or network access.

Make sure you are not using WinDbg 64 bit version. Silverlight is not 64 bit yet so even if you have a browser running on 64 bit os, sos dll for silverlight coreclr will fail to load on WinDbg 64 bit. Analyze your dump with WinDbg x86 version. I have WinDbg 32 bit and 64 bit both installed on my vista 64 bit os, although I still prefer XP or may be windows 7 from now on.

Twitter This

Username: Password:

As described in Sysinternals documentation http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx

ProcDump is a command-line utility whose primary purpose is monitoring an application for CPU spikes and generating crash dumps during a spike that an administrator or developer can use to determine the cause of the spike. ProcDump also includes hung window monitoring (using the same definition of a window hang that Windows and Task Manager use) and unhandled exception monitoring. It also can serve as a general process dump utility that you can embed in other scripts.

You don’t need to write your own utility to create a memory dump by monitoring performance counter. Don’t forget to use the switch “-ma” to dump full memory(especially for .net app) because by default it only dumps thread and handle.

This is really helpful to get a memory dump based on CPU usage and we could probably get the memory dump without using ADPlus in most of the cases.

syntax to dump full memory given process id is

procdump <process id> -ma

syntax to dump full memory given process id and cpu usage 80%(threshold)

procdump <process id> -ma -c 80

Twitter This

Username: Password:

We will review the WinDbg Extension SOS.dll in .NET Framework 4.0 CTP. CLR 4.0 has renamed runtime dll from mscorwks.dll to CLR.DLL, that’s really helpful.

loading SOS dll depending on the location of .net 4.0 runtime aka CLR.DLL, execute the following command

.loadby sos clr

1.  DML Support - YES, finally.  SOS supports DML in .NET 1.1 but it was gone in clr 2.0.  Silverlight CoreCLR supports DML and now .NET framework 4.0 supports it as well.

Execute the following command to turn on DMLfor every command or use /D option

0:003> .prefer_dml 1
DML versions of commands on by default

0:003> !dumpheap /D -type Exception -stat

For people new to WinDbg, Why am I so excited about DML support in SOS?

If you look at the above snapshot, you have the link for each MethodTable address which you can just click on to execute the command. No need to type, however not every commands will have the DML support but !dumpobject is another important one, you can just click on object address to dump an object from GC Heap.

2. The following additional extension commands are added

Examining code and stacks

!ThreadState

Examining CLR data structures

!DumpSigElem

Diagnostic Utilities

!VerifyObj
!FindRoots
!HeapStat
!GCWhere
!ListNearObj (lno)
!AnalyzeOOM (ao)

Examining the GC history

!HistInit
!HistStats
!HistRoot
!HistObj
!HistObjFind
!HistClear

!ThreadState Command

When you execute !threads command, you will see the similar output as shown below

PreEmptive   GC Alloc           Lock
ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception
0    1  310 00161438      a020 Enabled  013b4c64:013b5fe8 00159230     1 MTA
2    2  8c4 0016dab0      b220 Enabled  00000000:00000000 00159230     0 MTA (Finalizer)

First column is your debugger thread id and the second column ID is ManagedThread ID, OSID column is OS thread ID so that means OSID column will be 0 or some garbage when a runtime uses Fiber.

You will see the State column which is a bit flag as shown below(taken from Shared CLI)

TS_Unknown                = 0×00000000,    // threads are initialized this way

TS_AbortRequested         = 0×00000001,    // Abort the thread
TS_GCSuspendPending       = 0×00000002,    // waiting to get to safe spot for GC
TS_UserSuspendPending     = 0×00000004,    // user suspension at next opportunity
TS_DebugSuspendPending    = 0×00000008,    // Is the debugger suspending threads?
TS_GCOnTransitions        = 0×00000010,    // Force a GC on stub transitions (GCStress only)

TS_SuspendUnstarted       = 0×00400000,    // latch a user suspension on an unstarted thread

TS_ThreadPoolThread       = 0×00800000,    // is this a threadpool thread?
TS_TPWorkerThread         = 0×01000000,    // is this a threadpool worker thread?

TS_Interruptible          = 0×02000000,    // sitting in a Sleep(), Wait(), Join()
TS_Interrupted            = 0×04000000,    // was awakened by an interrupt APC. !!! This can be moved to TSNC

TS_CompletionPortThread   = 0×08000000,    // Completion port thread
………………………………………………………………………..

SOS in CLR4.0 has !threadstate command, which tells you exactly the state of the thread given the bit field, the following output shows you the threadstate bit for Worker Thread, Completion Port Thread and Finalizer Thread

0:000> !ThreadState 1009220
Legal to Join
Background
CLR Owns
In Multi Threaded Apartment
Thread Pool Worker Thread
0:000> !ThreadState 800a220
Legal to Join
Background
CoInitialized
In Multi Threaded Apartment
Completion Port Thread
0:000> !ThreadState b220
Legal to Join
Background
CLR Owns
CoInitialized
In Multi Threaded Apartment

Other Important Commands

!findroots - This is a very powerful and interesting command, because it allows you to break into debugee when CLR garbage collect generational objects.

!GCWhere - tells you the generation number along with the GC heap segment, you no longer need to map the object address with the GC heap segment or use any other extension dll

!HeapStat- This is another cool command, this command displays the stat on generational heap including generation sizes

!AnalyzeOOM - displays the detailed informatin on Last System.OutOfMemoryException

I can’t do justice on detailed documentation for each of these commands because SOS !help documentation has done a very good job. You can either look at !help documentation  or read below. I am just copying and pasting from SOS Help documentation

0:020> !help ThreadState
——————————————————————————-
!ThreadState value

The !Threads command outputs, among other things, the state of the thread.
This is a bit field which corresponds to various states the thread is in.
To check the state of the thread, simply pass that bit field from the
output of !Threads into !ThreadState.

Example:
0:003> !Threads
ThreadCount:      2
UnstartedThread:  0
BackgroundThread: 1
PendingThread:    0
DeadThread:       0
Hosted Runtime:   no
PreEmptive   GC Alloc           Lock
ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception
0    1  250 0019b068      a020 Disabled 02349668:02349fe8 0015def0     0 MTA
2    2  944 001a6020      b220 Enabled  00000000:00000000 0015def0     0 MTA (Finalizer)
0:003> !ThreadState b220
Legal to Join
Background
CLR Owns
CoInitialized
In Multi Threaded Apartment

Possible thread states:
Thread Abort Requested
GC Suspend Pending
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Legal to Join
Yield Requested
Hijacked by the GC
Blocking GC for Stack Overflow
Background
Unstarted
Dead
CLR Owns
CoInitialized
In Single Threaded Apartment
In Multi Threaded Apartment
Reported Dead
Task Reset
Sync Suspended
Debug Will Sync
Stack Crawl Needed
Suspend Unstarted
Aborted
Thread Pool Worker Thread
Interruptible
Interrupted
Completion Port Thread
Abort Initiated
Finalized
Failed to Start
Detached
0:020> !help DumpSigElem
——————————————————————————-
!DumpSigElem <sigaddr> <moduleaddr>

This command dumps a single element of a signature object.  For most circumstances,
you should use !DumpSig to look at individual signature objects, but if you find a
signature that has been corrupted in some manner you can use !DumpSigElem to read out
the valid portions of it.

If we look at a valid signature object for a method we see the following:
0:000> !dumpsig 0×000007fe`ec20879d 0×000007fe`eabd1000
[DEFAULT] [hasThis] Void (Boolean,String,String)

We can look at the individual elements of this object by adding the offsets into the
object which correspond to the return value and parameters:
0:000> !dumpsigelem 0×000007fe`ec20879d+2 0×000007fe`eabd1000
Void
0:000> !dumpsigelem 0×000007fe`ec20879d+3 0×000007fe`eabd1000
Boolean
0:000> !dumpsigelem 0×000007fe`ec20879d+4 0×000007fe`eabd1000
String
0:000> !dumpsigelem 0×000007fe`ec20879d+5 0×000007fe`eabd1000
String

We can do something similar for fields.  Here is the full signature of a field:
0:000> !dumpsig 0×000007fe`eb7fd8cd 0×000007fe`eabd1000
[FIELD] ValueClass System.RuntimeTypeHandle

Using !DumpSigElem we can find the type of the field by adding the offset of it (1) to
the address of the signature:
0:000> !dumpsigelem 0×000007fe`eb7fd8cd+1 0×000007fe`eabd1000
ValueClass System.RuntimeTypeHandle

!DumpSigElem will also work with generics.  Let a function be defined as follows:
public A Test(IEnumerable<B> n)

The elements of this signature can be obtained by adding offsets into the signature
when calling !DumpSigElem:

0:000> !dumpsigelem 00000000`00bc2437+2 000007ff00043178
__Canon
0:000> !dumpsigelem 00000000`00bc2437+4 000007ff00043178
Class System.Collections.Generic.IEnumerable`1<__Canon>

The actual offsets that you should add are determined by the contents of the
signature itself.  By trial and error you should be able to find various elements
of the signature.

0:020> !help VerifyObj
——————————————————————————-
!VerifyObj <object address>

!VerifyObj is a diagnostic tool that checks the object that is passed as an
argument for signs of corruption.

0:002> !verifyobj 028000ec
object 0×28000ec does not have valid method table

0:002> !verifyobj 0680017c
object 0×680017c: bad member 00000001 at 06800184

0:020> !help FindRoots
——————————————————————————-
!FindRoots -gen <N> | -gen any | <object address>

The “-gen” form causes the debugger to break in the debuggee on the next
collection of the specified generation.  The effect is reset as soon as the
break occurs, in other words, if you need to break on the next collection you
would need to reissue the command.

The last form of this command is meant to be used after the break caused by the
other forms has occurred.  Now the debuggee is in the right state for
!FindRoots to be able to identify roots for objects from the current condemned
generations.

!FindRoots is a diagnostic command that is meant to answer the following
question:

“I see that GCs are happening, however my objects have still not been
collected. Why? Who is holding onto them?”

The process of answering the question would go something like this:

1. Find out the generation of the object of interest using the !GCWhere
command, say it is gen 1:
!GCWhere <object address>

2. Instruct the runtime to stop the next time it collects that generation using
the !FindRoots command:
!FindRoots -gen 1
g

3. When the next GC starts, and has proceeded past the mark phase a CLR
notification will cause a break in the debugger:
(fd0.ec4): CLR notification exception - code e0444143 (first chance)
CLR notification: GC - end of mark phase.
Condemned generation: 1.

4. Now we can use the !FindRoots <object address> to find out the cross
generational references to the object of interest.  In other words, even if the
object is not referenced by any “proper” root it may still be referenced by an
older object (from an older generation), from a generation that has not yet been
scheduled for collection.  At this point !FindRoots will search those older
generations too, and report those roots.
0:002> !findroots 06808094
older generations::Root:  068012f8(AAA.Test+a)->
06808094(AAA.Test+b)

0:020> !help HeapStat
——————————————————————————-
!HeapStat [-inclUnrooted | -iu]

This command shows the generation sizes for each heap and the total, how much free
space there is in each generation on each heap.  If the -inclUnrooted option is
specified the report will include information about the managed objects from the
GC heap that are not rooted anymore.

Sample output:

0:002> !heapstat
Heap     Gen0         Gen1         Gen2         LOH
Heap0    177904       12           306956       8784
Heap1    159652       12           12           16
Total    337556       24           306968       8800

Free space:                                                 Percentage
Heap0    28           12           12           64          SOH:  0% LOH:  0%
Heap1    104          12           12           16          SOH:  0% LOH:100%
Total    132          24           24           80

0:002> !heapstat -inclUnrooted
Heap     Gen0         Gen1         Gen2         LOH
Heap0    177904       12           306956       8784
Heap1    159652       12           12           16
Total    337556       24           306968       8800

Free space:                                                 Percentage
Heap0    28           12           12           64          SOH:  0% LOH:  0%
Heap1    104          12           12           16          SOH:  0% LOH:100%
Total    132          24           24           80

Unrooted objects:                                           Percentage
Heap0    152212       0            306196       0           SOH: 94% LOH:  0%
Heap1    155704       0            0            0           SOH: 97% LOH:  0%
Total    307916       0            306196       0

The percentage column contains a breakout of free or unrooted bytes to total bytes.

0:020> !help GCWhere
——————————————————————————-
!GCWhere <object address>

!GCWhere displays the location in the GC heap of the argument passed in.

0:002> !GCWhere 02800038
Address  Gen Heap segment  begin    allocated size
02800038 2    0   02800000 02800038 0282b740  12

When the argument lies in the managed heap, but is not a valid *object* address
the “size” is displayed as 0:

0:002> !GCWhere 0280003c
Address  Gen Heap segment  begin    allocated size
0280003c 2    0   02800000 02800038 0282b740  0

0:020> !help ListNearObj
——————————————————————————-
!ListNearObj <object address>

!ListNearObj is a diagnostic tool that displays the object preceeding and
succeeding the address passed in:

The command looks for the address in the GC heap that looks like a valid
beginning of a managed object (based on a valid method table) and the object
following the argument address.

0:002> !ListNearObj 028000ec
Before: 0×28000a4           72 (0×48      ) System.StackOverflowException
After:  0×2800134           72 (0×48      ) System.Threading.ThreadAbortException
Heap local consistency confirmed.

0:002> !ListNearObj 028000f0
Before: 0×28000ec           72 (0×48      ) System.ExecutionEngineException
After:  0×2800134           72 (0×48      ) System.Threading.ThreadAbortException
Heap local consistency confirmed.

The command considers the heap as “locally consistent” if:
prev_obj_addr + prev_obj_size = arg_addr && arg_obj + arg_size = next_obj_addr
OR
prev_obj_addr + prev_obj_size = next_obj_addr

When the condition is not satisfied:

0:002> !lno 028000ec
Before: 0×28000a4           72 (0×48      ) System.StackOverflowException
After:  0×2800134           72 (0×48      ) System.Threading.ThreadAbortException
Heap local consistency not confirmed.

0:020> !help AnalyzeOOM
——————————————————————————-
!AnalyzeOOM

!AnalyzeOOM displays the info of the last OOM occured on an allocation request to
the GC heap (in Server GC it displays OOM, if any, on each GC heap).

To see the managed exception(s) use the !Threads command which will show you
managed exception(s), if any, on each managed thread. If you do see an
OutOfMemoryException exception you can use the !PrintException command on it.
To get the full callstack use the “kb” command in the debugger for that thread.
For example, to display thread 3’s stack use ~3kb.

OOM exceptions could be because of the following reasons:

1) allocation request to GC heap
in which case you will see JIT_New* on the call stack because managed code called new.
2) other runtime allocation failure
for example, failure to expand the finalize queue when GC.ReRegisterForFinalize is
called.
3) some other code you use throws a managed OOM exception
for example, some .NET framework code converts a native OOM exception to managed
and throws it.

The !AnalyzeOOM command aims to help you with investigating 1) which is the most
difficult because it requires some internal info from GC. The only exception is
we don’t support allocating objects larger than 2GB on CLR v2.0 or prior. And this
command will not display any managed OOM because we will throw OOM right away
instead of even trying to allocate it on the GC heap.

There are 2 legitimate scenarios where GC would return OOM to allocation requests -
one is if the process is running out of VM space to reserve a segment; the other
is if the system is running out physical memory (+ page file if you have one) so
GC can not commit memory it needs. You can look at these scenarios by using performance
counters or debugger commands. For example for the former scenario the “!address
-summary” debugger command will show you the largest free region in the VM. For
the latter scenario you can look at the “Memory\% Committed Bytes In Use” see
if you are running low on commit space. One important thing to keep in mind is
when you do this kind of memory analysis it could an aftereffect and doesn’t
completely agree with what this command tells you, in which case the command should
be respected because it truly reflects what happened during GC.

The other cases should be fairly obvious from the callstack.

Sample output:

0:011> !ao
———Heap 2 ———
Managed OOM occured after GC #28 (Requested to allocate 1234 bytes)
Reason: Didn’t have enough memory to commit
Detail: SOH: Didn’t have enough memory to grow the internal GC datastructures (800000 bytes) -
on GC entry available commit space was 500 MB
———Heap 4 ———
Managed OOM occured after GC #12 (Requested to allocate 100000 bytes)
Reason: Didn’t have enough memory to allocate an LOH segment
Detail: LOH: Failed to reserve memory (16777216 bytes)

0:020> !help FAQ
——————————————————————————-
>> Where can I get the right version of SOS for my build?

If you are running version 1.1 or 2.0 of the CLR, SOS.DLL is installed in the
same directory as the main CLR dll (CLR.DLL). Newer versions of the
Windows Debugger provide a command to make it easy to load the right copy of
SOS.DLL:

“.loadby sos clr”

That will load the SOS extension DLL from the same place that CLR.DLL is
loaded in the process. You shouldn’t attempt to use a version of SOS.DLL that
doesn’t match the version of CLR.DLL. You can find the version of
CLR.DLL by running

“lmvm clr”

in the debugger.  Note that if you are running CoreCLR (e.g. Silverlight)
then you should replace “clr” with “coreclr”.

If you are using a dump file created on another machine, it is a little bit
more complex. You need to make sure the mscordacwks.dll file that came with
that install is on your symbol path, and you need to load the corresponding
version of sos.dll (typing .load <full path to sos.dll> rather than using the
.loadby shortcut). Within the Microsoft corpnet, we keep tagged versions
of mscordacwks.dll, with names like mscordacwks_<architecture>_<version>.dll
that the Windows Debugger can load. If you have the correct symbol path to the
binaries for that version of the Runtime, the Windows Debugger will load the
correct mscordacwks.dll file.

>> I have a chicken and egg problem. I want to use SOS commands, but the CLR
isn’t loaded yet. What can I do?

In the debugger at startup you can type:

“sxe clrn”

Let the program run, and it will stop with the notice

“CLR notification: module ‘mscorlib’ loaded”

At this time you can use SOS commands. To turn off spurious notifications,
type:

“sxd clrn”

>> I got the following error message. Now what?

0:000> .loadby sos clr
0:000> !DumpStackObjects
Failed to find runtime DLL (clr.dll), 0×80004005
Extension commands need clr.dll in order to have something to do.
0:000>

This means that the CLR is not loaded yet, or has been unloaded. You need to
wait until your managed program is running in order to use these commands. If
you have just started the program a good way to do this is to type

bp clr!EEStartup “g @$ra”

in the debugger, and let it run. After the function EEStartup is finished,
there will be a minimal managed environment for executing SOS commands.

>> I have a partial memory minidump, and !DumpObj doesn’t work. Why?

In order to run SOS commands, many CLR data structures need to be traversed.
When creating a minidump without full memory, special functions are called at
dump creation time to bring those structures into the minidump, and allow a
minimum set of SOS debugging commands to work. At this time, those commands
that can provide full or partial output are:

CLRStack
Threads
Help
PrintException
EEVersion

For a minidump created with this minimal set of functionality in mind, you
will get an error message when running any other commands. A full memory dump
(obtained with “.dump /ma <filename>” in the Windows Debugger) is often the
best way to debug a managed program at this level.

>> What other tools can I use to find my bug?

Turn on Managed Debugging Assistants. These enable additional runtime diagnostics,
particularly in the area of PInvoke/Interop. Adam Nathan has written some great
information about that:

http://blogs.msdn.com/adam_nathan/

>> Does SOS support DML?

Yes.  SOS respects the .prefer_dml option in the debugger.  If this setting is
turned on, then SOS will output DML by default.  Alternatively, you may leave
it off and add /D to the beginning of a command to get DML based output for it.
Not all SOS commands support DML output.

0:020> !help HistInit
——————————————————————————-
!HistInit

Before running any of the Hist - family commands you need to initialize the SOS
structures from the stress log saved in the debuggee.  This is achieved by the
HistInit command.

Sample output:

0:001> !HistInit
Attempting to read Stress log
STRESS LOG:
facilitiesToLog  = 0xffffffff
levelToLog       = 6
MaxLogSizePerThread = 0×10000 (65536)
MaxTotalLogSize = 0×1000000 (16777216)
CurrentTotalLogChunk = 9
ThreadsWithLogs  = 3
Clock frequency  = 3.392 GHz
Start time         15:26:31
Last message time  15:26:56
Total elapsed time 25.077 sec
……………………………….
—————————- 2407 total entries —————————–

SUCCESS: GCHist structures initialized

0:020> !help HistStats
——————————————————————————-
!HistStats

HistStat provides a number of garbage collection statistics obtained from the
stress log.

Sample output:

0:003> !HistStats
GCCount    Plugs Promotes   Relocs
———————————–
2296        0       35       86
2295        0       34       85
2294        0       34       85
…
2286        0       32       83
2285        0       32       83
322        0       23       55
0        0        0        0
Root 01e411b8 relocated multiple times in gc 322
Root 01e411bc relocated multiple times in gc 322
…
Root 01e413f8 relocated multiple times in gc 322
Root 01e413fc relocated multiple times in gc 322

0:020> !help histroot
——————————————————————————-
!HistRoot <root>

The root value obtained from !HistObjFind can be used to track the movement of
an object through the GCs.

HistRoot provides information related to both promotions and relocations of the
root specified as the argument.

0:003> !HistRoot 01e411b8
GCCount    Value       MT Promoted?                Notes
———————————————————
2296 028970d4 5b6c5cd8       yes
2295 028970d4 5b6c5cd8       yes
2294 028970d4 5b6c5cd8       yes
2293 028970d4 5b6c5cd8       yes
2292 028970d4 5b6c5cd8       yes
2291 028970d4 5b6c5cd8       yes
2290 028970d4 5b6c5cd8       yes
2289 028970d4 5b6c5cd8       yes
2288 028970d4 5b6c5cd8       yes
2287 028970d4 5b6c5cd8       yes
2286 028970d4 5b6c5cd8       yes
2285 028970d4 5b6c5cd8       yes
322 028970e8 5b6c5cd8       yes Duplicate promote/relocs
…

0:020> !help HistObj
——————————————————————————-
!HistObj <obj_address>

This command examines all stress log relocation records and displays the chain
of GC relocations that may have led to the address passed in as an argument.
Conceptually the output is:

GenN    obj_address   root1, root2, root3,
GenN-1  prev_obj_addr root1, root2,
GenN-2  prev_prev_oa  root1, root4,
…

Sample output:
0:003> !HistObj 028970d4
GCCount   Object                                    Roots
———————————————————
2296 028970d4 00223fc4, 01e411b8,
2295 028970d4 00223fc4, 01e411b8,
2294 028970d4 00223fc4, 01e411b8,
2293 028970d4 00223fc4, 01e411b8,
2292 028970d4 00223fc4, 01e411b8,
2291 028970d4 00223fc4, 01e411b8,
2290 028970d4 00223fc4, 01e411b8,
2289 028970d4 00223fc4, 01e411b8,
2288 028970d4 00223fc4, 01e411b8,
2287 028970d4 00223fc4, 01e411b8,
2286 028970d4 00223fc4, 01e411b8,
2285 028970d4 00223fc4, 01e411b8,
322 028970d4 01e411b8,
0 028970d4

0:020> !help HistObjFind
——————————————————————————-
!HistObjFind <obj_address>

To examine log entries related to an object whose present address is known one
would use this command. The output of this command contains all entries that
reference the object:

0:003> !HistObjFind 028970d4
GCCount   Object                                  Message
———————————————————
2296 028970d4 Promotion for root 01e411b8 (MT = 5b6c5cd8)
2296 028970d4 Relocation NEWVALUE for root 00223fc4
2296 028970d4 Relocation NEWVALUE for root 01e411b8
…
2295 028970d4 Promotion for root 01e411b8 (MT = 5b6c5cd8)
2295 028970d4 Relocation NEWVALUE for root 00223fc4
2295 028970d4 Relocation NEWVALUE for root 01e411b8
…

0:020> !help HistClear
——————————————————————————-
!HistClear

This command releases any resources used by the Hist-family of commands.
Generally there’s no need to call this explicitly, as each HistInit will first
cleanup the previous resources.

Twitter This

Username: Password:

And No I don’t have a solution for it and probably the only workaround is to make your visual studio Large Address Aware(3GB switch) on x86.

Steps to re-create

1. download and unzip http://debuggingblog.com/resources/transcripts.zip

2. open, close the xml file and try to load it the second time

3. If you load the xml file using IE8, you will see the followings once you close it

——————– State SUMMARY ————————–
TotSize (      KB)   Pct(Tots)  Usage
19e6f000 (  424380) : 20.24%   : MEM_COMMIT
f3b4000 (  249552) : 11.90%   : MEM_FREE
56dcd000 ( 1423156) : 67.86%   : MEM_RESERVE

Almost 1.4 GB Memory allocated in GC Segements for xml file is still reserved even after unloading the xml file.

However, visual studio 2008 is another story

0:000> !eeheap -gc
ephemeral segment allocation context: none
segment    begin allocated     size
01830000 01831000  027ecadc 0×00fbbadc(16497372)
12860000 12861000  137616c4 0×00f006c4(15730372)
………………………………………………………………………………..

We have bunch of 16MB GC segments and most of the objects are in gen 2.

0c55d1ec   739459     85777244 Microsoft.XmlEditor.XmlElement
0c559858  1496448     89786880 Microsoft.XmlEditor.Identifier
001f1918   105303     97472784      Free
793308ec  2369315    387475460 System.String
Total 9375571 objects

we have 90+ MB of free blocks and 380+MB in System.String. There are 2.36 million string objects, yeah so you don’t wanna pick each one of the string object to find GC root unless Microsoft or someone is paying you a dime to dump each object and aha a dump a day will make your day for sure.

0:000> !dumpheap -mt 0c9d4134
Address       MT     Size
018f241c 0c9d4134       68
5f610108 0c9d4134       68
total 2 objects
Statistics:
MT    Count    TotalSize Class Name
0c9d4134        2          136 Microsoft.XmlEditor.XmlDocumentProperties
Total 2 objects
0:000> !objsize 018f241c
sizeof(018f241c) =    507372388 (  0×1e3de364) bytes (Microsoft.XmlEditor.XmlDocumentProperties)
0:000> !objsize 5f610108
sizeof(5f610108) =    507371128 (  0×1e3dde78) bytes (Microsoft.XmlEditor.XmlDocumentProperties)

Did you just see that almost 1GB of virtual memory rooted in Microsoft.XmlEditor.XmlDocumentProperties? That’s just outrageous, I mean why would microsoft visual studio take up 1.2 GB of virtual memory to open a 58MB file, although It does make use of schema context cache.

0:000> !gcroot -nostacks 018f241c
DOMAIN(001EC570):HANDLE(RefCnt):16d1b20:Root:018f241c(Microsoft.XmlEditor.XmlDocumentProperties)

GCHandle of type RefCnt is keeping reference to Microsoft.XmlEditor.XmlDocumentProperties

There is an OutOfMemoryException thrown with the following callstack

Exception object: 5ed00a34
Exception type: System.OutOfMemoryException
Message: Insufficient memory to continue the execution of the program.
InnerException: <none>
StackTrace (generated):
SP       IP       Function
0012F5A0 0C97E8B3 Microsoft_VisualStudio_Package_LanguageService_9_0!Microsoft.VisualStudio.NativeMethods.ThrowOnFailure(Int32, Int32[])+0×3b
0012F5AC 0C9E94BB Microsoft_VisualStudio_Package_LanguageService_9_0!Microsoft.VisualStudio.Package.Source.GetText()+0×3c
0012F5DC 0C9E9360 Microsoft_VisualStudio_Package_LanguageService_9_0!Microsoft.VisualStudio.Package.Source.BeginParse()+0×55
0012F644 0C9ECF38 Microsoft_VisualStudio_Package_LanguageService_9_0!Microsoft.VisualStudio.Package.Source.OnIdle(Boolean)+0×80
0012F654 0C9ECE28 Microsoft_VisualStudio_Package_LanguageService_9_0!Microsoft.VisualStudio.Package.LanguageService.OnIdle(Boolean)+0xd8
0012F674 0C9ECCDD Microsoft_XmlEditor!Microsoft.XmlEditor.XmlLanguageService.OnIdle(Boolean)+0×35
0012F684 0C9ECC34 Microsoft_XmlEditor!Microsoft.XmlEditor.Package.FDoIdle(UInt32)+0xc4

Conclusion

I hope this is fixed in Visual Studio 2010, I do need to try it out.

Twitter This

Username: Password:

When you are starting out with Silverlight and WCF/SOAP to implement WS-Security using SOAP headers into request messages or for that matter if you are accessing SOAP services from silverlight, you may run into this silly issue that’s why I am making note of it here.

Silverlight documentation clearly says that

SOAP headers are supported via the OutgoingMessageHeaders and IncomingMessageHeaders properties inside an operation method call.

OutgoingMessageHeaders is the outgoing message headers for the active OperationContext.

Use the OperationContext from within a service operation to access the current operation execution environment.

The OperationContext has the following properties and methods.

• The Current property returns the OperationContext object that represents the current execution context.

Documentation is not very clear so this is when you may run into this issue because OperationContext is null if you directly access OperationContext.Current from silverlight client.

Let’s look at OperationContext .Current setter to find out who sets it, as shown below in reflector snapshot. Now you know  OperationContextScope sets the Context so you have to first create OperationContextScope using your ContextChannel before accessing the current Operation Execution Environment.

using (OperationContextScope contextScope = new OperationContextScope(client.InnerChannel))
{
OperationContext.Current.OutgoingMessageHeaders.Add(MessageHeader.CreateHeader
}

Twitter This

Username: Password:

I have seen numerous .net application where a background thread is created for background tasks but at the same time thread scheduling priority is never set. By default, managed thread runs with normal relative priority so now if you have a thread doing some maintenance work,  that thread has no reason to run with Normal relative priority or sometime even the thread sending occasional updates to UI will run with normal relative priority

However, managed thread doesn’t allow you to set the thread priority to true THREAD_PRIORITY_IDLE class.

If you have done Win32 development then you are used to calling ::CreateThread() with Suspended flag and ::SetThreadPriority() to start thread with some priority. The default behavior of ::CreateThread is to create a thread with Normal Priority(relative to process priority class) same as managed Thread.Start(), but you will have to pass CREATE_SUSPENDED flag first.

If you insert a breakpoint in kernel32!CreateThread, you will see that Thread.Start() will call into this Win32 API because this is the only way you can create a thread in Windows OS, even if you use CRT _beginthreadex. One important difference is you don’t have an option to specify thread CreationFlags in managed code.

visit msdn documentation http://msdn.microsoft.com/en-us/library/74169f59.aspx for more information on Managed and Unmanaged Threading.

Managed Thread doesn’t support THREAD_PRIORITY_IDLE but rather maps it to THREAD_PRIORITY_LOWEST and also it doesn’t support THREAD_PRIORITY_TIME_CRITICAL.

PAL documentation says that SetThreadPriority should set THREAD_PRIORITY_IDLE to the least-priority value the host operating system allows, as the common language runtime sets threads to THREAD_PRIORITY_IDLE then busy-waits with a Sleep(0) loop until the garbage collection thread has run, with the garbage collection thread at a priority greater than idle.

Another important aspect to managed runtime is when you write your own host, you can use fiber to map one os thread to multiple managed threads with your own scheduling algorithm since fiber is implemented in usermode. Jeffrey Richter’s book “Programming Applications for Microsoft Windows” has very good description on Fiber. Windows OS supports fiber mainly for portability since other operating systems don’t have the same threading model.

But, if you do need to map managed thread priority to supported thread priority in windows , you can use System.Diagnositcs.ProcessThread. System.Diagnostics.ThreadPriorityLevel maps one to one with thread priority class in Win32 API(windows server 2003/XP).

Windows Vista/7 and windows server 2008 introduced THREAD_MODE_BACKGROUND_BEGIN and THREAD_MODE_BACKGROUND_END see the msdn documentation http://msdn.microsoft.com/en-us/library/ms686277(VS.85).aspx

If you really want to have your application to do task in background with minimal impact to system resources, you should consider setting thread priority to THREAD_MODE_BACKGROUND_BEGIN on Windows 7/2008 through interop.

One important point is, most of the managed application don’t really need a thread with idle priority, you can meet your application’s performance objectives by manipulating the thread scheduling priority less or higher than default Normal relative priority as defined in  System.Threading.ThreadPriority Enumeration

Twitter This

Username: Password:

Problem Description

This exception is thrown in a Silverlight 2 App while trying to show and hide System.Windows.Controls.TabItem in System.Windows.Controls.TabControl

Steps to Recreate

browse to http://debuggingblog.com/sl/project1/default.html and click on button “Show Angelina”  3 times

Debugging Silverlight app using WinDbg

We will be analyzing a memory dump of IE on System.InvalidOperationException

1. We have the following two InvalidOperationException exception objects on managed heap

0:007> !dumpheap -type System.InvalidOperationException
Address       MT     Size
1013c79c 0f1ff824       72
1015addc 0f1ff824       72
total 2 objects
Statistics:
MT    Count    TotalSize Class Name
0f1ff824        2          144 System.InvalidOperationException
Total 2 objects

2. Lets get the stack trace when this exception occured

0:007> !pe 1013c79c
Exception object: 1013c79c
Exception type:   System.InvalidOperationException
Message:          Element is already the child of another element.
InnerException:   <none>
StackTrace (generated):
SP       IP       Function
021FF640 0F40F06A !MS.Internal.XcpImports.CheckHResult(UInt32)+0×32
021FF64C 0E3A4CE9 !MS.Internal.XcpImports.SetValue(MS.Internal.INativeCoreTypeWrapper, System.Windows.DependencyProperty, System.Windows.DependencyObject)+0xa9
021FF680 0E3A4118 !MS.Internal.XcpImports.SetValue(MS.Internal.INativeCoreTypeWrapper, System.Windows.DependencyProperty, System.Object)+0×100
021FF71C 0E3A3FBC !System.Windows.DependencyObject.SetObjectValueToCore(System.Windows.DependencyProperty, System.Object)+0×1c4
021FF760 0E3A1CBB !System.Windows.DependencyObject.SetValueInternal(System.Windows.DependencyProperty, System.Object, Boolean, Boolean, Boolean)+0×503
021FF834 0E3A17A1 !System.Windows.DependencyObject.SetValueInternal(System.Windows.DependencyProperty, System.Object)+0×21
021FF848 0E3A1763 !System.Windows.DependencyObject.SetValue(System.Windows.DependencyProperty, System.Object)+0×1b
021FF85C 0E3D68D7 !System.Windows.Controls.ContentControl.set_Content(System.Object)+0×37
021FF870 0E3212CF !SilverlightApplication1.Page.ShowContent()+0×57
021FF880 0E3208BE !SilverlightApplication1.Page.ShowHideAngelina_Click(System.Object, System.Windows.RoutedEventArgs)+0×76
021FF898 0E4099B5 !System.Windows.Controls.Primitives.ButtonBase.OnClick()+0×5d
021FF8B0 0E40993F !System.Windows.Controls.Button.OnClick()+0×47
021FF8C0 0E40986D !System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(System.Windows.Input.MouseButtonEventArgs)+0×85
021FF8D0 0E4097D1 !System.Windows.Controls.Control.OnMouseLeftButtonUp(System.Windows.Controls.Control, System.EventArgs)+0×41
021FF8E0 0E36C887 !MS.Internal.JoltHelper.FireEvent(IntPtr, IntPtr, Int32, System.String)+0×1b7

3. This exception occured while trying to Set the Content of a Control, so lets find out what is this content and who holds this UIElement, lets get the IL of method SilverlightApplication1.Page.ShowContent()

0:007> !dumpil 0f224060
ilAddr = 0f4607f8
IL_0000: ldarg.0
IL_0001: newobj System.Windows.Controls.TabItem::.ctor  // A new TabItem object is created
IL_0006: stfld SilverlightApplication1.Page::item
IL_000b: ldarg.0
IL_000c: ldfld SilverlightApplication1.Page::item
IL_0011: ldstr “Angelina Jolie”
IL_0016: callvirt System.Windows.Controls.TabItem::set_Header
IL_001b: ldarg.0
IL_001c: ldfld SilverlightApplication1.Page::item//Get the reference to a  TabItem object created earlier
IL_0021: ldarg.0
IL_0022: ldfld SilverlightApplication1.Page::content
IL_0027: callvirt System.Windows.Controls.ContentControl::set_Content //Set_Content is called  with SilverlightApplication1.Page object’s data member content
IL_002c: ldarg.0
IL_002d: ldfld SilverlightApplication1.Page::TabList
IL_0032: callvirt System.Windows.Controls.ItemsControl::get_Items
IL_0037: ldarg.0
IL_0038: ldfld SilverlightApplication1.Page::item

We have 2 interesting objects TabItem and SilverlightApplication1.Page::content here

4. First get the address of SilverlightApplication1.Page::content object which is being set in TabItem because that’s what resulting in exception

0:007> !do 1003b3c0
Name:        SilverlightApplication1.Page
Fields:
MT    Field   Offset                 Type VT     Attr    Value Name
0f42e730  400000d       58 …Windows.UIElement  0 instance 1011c0e8 content

0:007> !dumpheap -mt 0e396648
Address       MT     Size
1003ed3c 0e396648      140
101276a0 0e396648      140
1013c050 0e396648      140
total 3 objects
Statistics:
MT    Count    TotalSize Class Name
0e396648        3          420 System.Windows.Controls.TabItem
Total 3 objects

We have 3 TabItem objects on heap and the address of the content object(UIElement) is 1011c0e8

5. The next step to find out is which TabItem object has this UIElement and why is this TabItem object still around.

0:007> !dumpheap -mt 0e396648
Address       MT     Size
1003ed3c 0e396648      140
101276a0 0e396648      140
1013c050 0e396648      140
total 3 objects

0:007> !do 101276a0
Name:        System.Windows.Controls.TabItem
MethodTable: 0e396648
EEClass:     0e3bb768
Size:        140(0×8c) bytes
File:        System.Windows.Controls, Version=2.0.5.0, Culture=neutral
Fields:
MT    Field   Offset                 Type VT     Attr    Value Name

0ec444e8  40002b4       34        System.Object  0 instance 1011c0e8 _treeContent

0:007> !gcroot 101276a0
Note: Roots found on stacks may be false positives. Run “!help gcroot” for
more info.
Scan Thread 7 OSTHread 1710
Scan Thread 27 OSTHread 1200
Scan Thread 28 OSTHread 1278
Scan Thread 31 OSTHread 1488

6. TabItem object with address = 0e396648 has the reference to content object 1011c0e8 and this object is not rooted so that means this object is ready to be garbage collected.

There is already a TabItem object on managed heap holding reference to the same content object which is being assigned to new TabItem object on SilverlightApplication1.Page.ShowHideAngelina_Click(), that’s why we get System.InvaildOperationException with the error message “Element is already the child of another element”

7. You can get the address of Method Description for each of the methods in SilverlightApplication1.Page

0:007> !dumpmt -md 0f224094
————————————–
MethodDesc Table
Entry       MethodDesc      JIT Name
0e320848   0f224038 JIT SilverlightApplication1.Page.ShowHideAngelina_Click(System.Object,
0e321300   0f224058      JIT SilverlightApplication1.Page.HideContent()
0e321278   0f224060      JIT SilverlightApplication1.Page.ShowContent()

Let’s look at the implementation of ShowHideAngelina_Click

0:007> !dumpil 0f224038
ilAddr = 0f46071c
IL_0012: ldstr “Show”
IL_0017: callvirt System.String::Contains
IL_001c: brfalse.s IL_003f
IL_001e: ldloc.0
IL_0039: call SilverlightApplication1.Page::ShowContent
IL_003e: ret
IL_003f: ldloc.0
IL_004a: ldarg.0
IL_004b: call SilverlightApplication1.Page::HideContent
IL_0050: ret

This method checks the Text of the button and calls ‘ShowContent’ if the text =“Show” otherwise calls ‘HideContent’, so let’s look at HideContent implementation

0:007> !dumpil 0f224058
ilAddr = 0f4607d6
IL_0000: ldarg.0
IL_0001: ldfld SilverlightApplication1.Page::item
IL_0006: brfalse.s IL_001f
IL_0008: ldarg.0
IL_0009: ldfld SilverlightApplication1.Page::TabList
IL_000e: callvirt System.Windows.Controls.ItemsControl::get_Items
IL_0013: ldarg.0
IL_0014: ldfld SilverlightApplication1.Page::item
IL_0019: callvirt class [System.Windows]System.Windows.PresentationF◆䷒¥⽫Ż�::Remove
IL_001e: pop
IL_001f: ret

HideContent calls TabList.get_Items and remove the TabItem from System.Windows.Control.TabControl

Resolution

HideContent() method removes the TabItem from TabControl.Items so unless TabItem is Garbage Collected  removed TabItem will still hold the reference to Content UIElement. You can quickly fix this issue by assiging Content = null, when that item is removed as shown below in HideContent

//OLD Buggy Implementation

private void HideContent()
{
if (item != null)
this.TabList.Items.Remove(item);
}

//new implementation to make sure Content is set to null

private void HideContent()
{
item.Content = null; //assign TabItem Content = NULL
if (item != null)
this.TabList.Items.Remove(item);
}

Twitter This

Username: Password:

When you use visual studio to create a silverlight application, you will notice that it creates the application xaml, page xaml and their corresponding code behinds. When you build the application, xap package will be created inside ‘ClientBin” folder along with a test page.

I will show you how to create a silverlight xap package without visual studio with a very simple control to help you understand how silverlight control essentially works. In my previous post I mentioned how a browser based on http response loads Silverlight Plug-in(NPCTRL.DLL) with MIME type=”application/x-silverlight-2″ specified in object tag.

Let’s first look at Silverlight Application Project which gets created by default using Silverlight Application project template in visual studio. It creates the following files in a silverlight application project

-Properties
—AppManifest.xml
—AssemblyInfo.cs
-App.xaml
—App.xaml.cs
-Page.xaml
—Page.xaml.cs

AppManifest.xml contains the XAML object elements Deployment and Depolyment Parts which is used to build application package’s manifest file. When AppManifest xaml file is generated for XAP package, it also includes the followings

<            EntryPointAssembly=”MySilverlightControl”
EntryPointType=”MySilverlightControl”
RuntimeVersion=”2.0.31005″>
<AssemblyPart x:Name=”XName” Source=”MySilverlightControl.dll” />

EntryPointAssembly/Type and the Assembly Part are mandatory in a Silverlight Application because that’s how coreservices determines the entry point in an assembly.

App.xaml.cs implements the followings

public partial class App : Application

{

public App()
{
this.Startup += this.Application_Startup;
this.Exit += this.Application_Exit;
this.UnhandledException += this.Application_UnhandledException;

InitializeComponent();
}

private void Application_Startup(object sender, StartupEventArgs e)
{
this.RootVisual = new Page();
}

private void Application_Exit(object sender, EventArgs e)
private void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs         private void ReportErrorToDOM(ApplicationUnhandledExceptionEventArgs e)

Application class is essentially adding the event listener to AGCORE.DLL through Managed Runtime and sets the root as Page object in a visual tree. Below is the implementation of InitializeComponent

public void InitializeComponent()
{
    if (!this._contentLoaded)
    {
        this._contentLoaded = true;
        Application.LoadComponent(this, new Uri("/MyControl;component/App.xaml", UriKind.Relative));
    }
}

InitializeComponent implementation will fool you into believing that you will need xaml file. But you don’t even need to call InitializeComponent unless of course you are using designer generated code, behind the scene designed generates the source code for xaml file

So far, we have seen that silverlight project has an App class which derives from System.Windows.Application and apart from setting the event listeners for error handling and etc, it just sets the visual root to a Page object which is nothing but System.Windows.Control.UserControl deriving from System.Windows.Controls.Control ..and Control in turn from UIElement.

When you build the project, it will create a xap pacakge which is nothing but just a ZIP file. XAP package includes assemblies and the application manifest XAML file and that’s what get hosted in a web page

Let’s put these pieces together

1. We need a web server which can support MIME Type=application/x-silverlight-2 to serve silverlight content

2. web page hosts silverlight control by including a XAP package in object tag

3. XAP package is a zip archive format which includes application manifest(xaml file) and a managed assembly

4. managed assembly should have System.Windows.Application object

5. Application object needs a visual root tree which could be any control/UIElement

6. and of course managed assembly will need silverlight runtime

Steps to create a barebone Silverlight Control with no visual studio or xaml

1. Open your notepad and type the followings to create your barebone silverlight control deriving from System.Windows.Application and save it as MySilverlightControl.cs

public class MySilverlightControl : System.Windows.Application
{

//constructor
public MySilverlightControl()
{

//create a textblock control and assign a width,height with text =”hello silverlight”
System.Windows.Controls.TextBlock txtBlock = new System.Windows.Controls.TextBlock();
txtBlock.Text = “Hello Silverlight”;
txtBlock.Width = 800;
txtBlock.Height = 600;

//make sure it is centered
txtBlock.HorizontalAlignment = System.Windows.HorizontalAlignment.Center;
txtBlock.VerticalAlignment = System.Windows.VerticalAlignment.Center;

//margin (left,top,..)
txtBlock.Margin = new System.Windows.Thickness(400, 300, 0, 0);
txtBlock.FontSize = 24;
//set the application visual root to txtblock control
this.RootVisual = txtBlock;
}
}

2. Launch command prompt to compile this file to create a MySilverlightControl.dll

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\csc /target:library MySilverlightControl.csc.exe /reference:”C:\Program Files\Microsoft Silverlight\2.0.40115.0\System.Windows.dll”

3. Type the followings in a notepad to create application manifest xaml file

<Deployment xmlns=”http://schemas.microsoft.com/client/2007/deployment”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
EntryPointAssembly=”MySilverlightControl”
EntryPointType=”MySilverlightControl”
RuntimeVersion=”2.0.31005″>
<Deployment.Parts>
<AssemblyPart x:Name=”XName” Source=”MySilverlightControl.dll” />
</Deployment.Parts>
</Deployment>

Rememeber these attributes are mandatory in a silverlight application assembly but not in a class library. I have copied Runtime Version from http://msdn.microsoft.com/en-us/library/cc265156(VS.95).aspx

The above URL describes the MIME type and their corresponding silverlight runtime version. EntryPointAssembly and the other attributes are self explanatory and that’s what will be used by coreservices.

4. Zip those 2 files AppManifest.xaml and MySilverlightControl.dll with .xap extension

5. Create the following html file and copy it under the same folder as .xap file

<html>
<head>
<title>Barebones - Silverlight Control Testing</title>
</head>
<body>
<div id=”silverlightControlHost”>
<object data=”data:application/x-silverlight-2,” type=”application/x-silverlight-2″ width=”100%” height=”100%”>
<param name=”source” value=”MySilverlightControl.xap”/>
</object>
</div>
</body>
</html>

Below is the snapshot

This is it and your silverlight control is ready to be hosted.

Obviously, you don’t need to manually create it, but this is an attempt to explain how silverlight works and what are the components involved in a silverlight control. You can dynamically generate a code or even use Reflection.Emit to generate MSIL and above all host your content on any webserver with mime type=”application/x-silverlight-2″ for silverlight 2 runtime.

This makes Silverlight very powerful and at the same time very simple to implement and silverlight coreclr has great debugging tool in WinDbg/CoreCLR!SOS.

Kudos to Microsoft Silverlight team.

silverlight is very appealing when it comes to out of browser experience and the development platform. Of course there are some limitations in silverlight app because it runs in security sandbox model so that means no access to filesystem but the development platform along with deployment makes a great case for consideration if you are thinking about using Adobe Flash/AIR

Twitter This

Username: Password:

The purpose of this article is to demonstrate the use of WinDbg to demonstrate request processing in Silverlight Runtime
We need fiddler ,  WinDbg  and a browser in Windows OS. We have a simple html page hosted in IIS with a Silverlight control in it as shown below
<html><head>   <title>SilverlightApplication1</title></head>
<body> <div id=”silverlightControlHost”>
<object data=”data:application/x-silverlight-2,” type=”application/x-silverlight-2″>
<param name=”source” value=”ClientBin/SilverlightApplication1.xap”/>
</object>  </div></body></html>

Attach browser to WinDbg, if you are using IE8 then make sure that you are attaching the child process (tabbed browser), use process explorer to identify the child process.
Execute command “.cmdtree <pathname>sld.txt” as shown below; you can download sld.txt from here as shown below

Silverlight runtime by default gets installed under <system drive> \Program Files\Microsoft Silverlight\<version #>
As shown below

We will be interested more in native dll which makes up of Silverlight runtime. .NET library is part of Silverlight CoreCLR.
In order to get the order of the loaded modules of Silverlight runtime, we can execute command “sxe ld” or you can double click on “Break On Loaded Modules” from command tree and browse to html page containing Silverlight xap file.

You need to ignore most of the dlls but the one loaded from “C:\Program Files\Microsoft Silverlight\2.0.40115.0” (Silverlight runtime path on my machine). The order of loaded modules is
Npctrl.dll, agcore.dll, coreclr.dll, .net dlls
You can use any disassembler  or ndepends to look at the exported functions or download one from http://www.smidgeonsoft.prohosting.com/pebrowse-pro-file-viewer.html to look at these native dlls

Below is the snapshot for NPCTRL.DLL

NPCTRL.dll is a COM dll which gets loaded in browser when http request is made with silverlight control in it(object tag). Shown below in the fiddler

Browser determines this based on http response since the MIME type in html file is
type=”application/x-silverlight-2″. NPCTRL.dll gets loaded as Silverlight Plug-in.

This occurs before XAP package gets downloaded,

As shown below, you can now clear breakpoints on loaded modules and set breakpoint for each of dll one by one because debugger remembers the last ld settings.

You may want to change the symbols path to Silverlight runtime so that you can set breakpoints on all exported methods (see the above snapshot).

NPCTRL.DLL imports followings from AGCORE.DLL

NPCTRP.DLL first loads AGCORE.DLL. NPCTRL.dll implements CXcpBrowserHost/CommonBrowserHost and it makes a call to agcore!ObtainCoreServices to set the frameworkcallbacks. NPCTRL.DLL also implements CWindowsServices::CreateCLRRuntimeHost. NPCTRL.dll calls NPCTRL!GetCLRRuntTimeHost to load CORECLR.dll . CORECLR.dll is our Silverlight .NET runtime engine, basically stripped down version of MSCORWKS.dll

Once CLRRuntime is created, coreclr will create a domain and load the .net assembly as a private copy using normal assembly binding. CORECLR will use PEImage layout to map the view of file same as .net runtime. Below is the call stack when application domain for coreclr is set up

0220f4d4 04a9b399 System.Activator.CreateInstance(System.String, System.String)
0220f4e8 04a9b349 System.AppDomain.CreateInstance(System.String, System.String)
0220f504 04a9b298 System.AppDomain.CreateInstanceAndUnwrap(System.String, System.String)
0220f51c 04a9b1d8 System.AppDomain.CreateDomainManager(System.String, System.String)
0220f560 04a94d6a System.AppDomain.SetDomainManager(System.Security.Policy.Evidence, System.Security.Policy.Evidence, IntPtr, Boolean)
0220f5b0 04a98dfd System.AppDomain.Setup(System.Object)

Once app domain is set up, ManagedRuntimeHost will be created as shown below from the reflector

This is when coreclr will also set up System.Windows.Threading.DispatcherSynchronizationContext for Single Threaded Apartment (UI). At this point, it will also create a callback handler in AGCORE.DLL. AGCORE.DLL is the Unmanaged version of Silverlight. AGCORE.DLL implements event listener, Setting the visual root and the callback handler is used for communication between CORECLR and AGCORE.

The main interface for communication between CORECLR and AGCORE is MS.Internal.XcpImports(implemented in System.Windows.DLL) XcpImports is the interop layer importing the exports section in AGCORE.dll

Once XcpImports.SetFrameworkCallbacksNative is called to set the callback handler, MS.Internal.FrameworkCallbacks will create System.Windows.DependencyObject.

After dependency object is created, Application object will be created as shown below

0220f2e4 0108008d SilverlightApplication1.App..ctor()
0220f850 026b2e90 System.Reflection.ConstructorInfo.Invoke(System.Object[])
0220f860 026b2a98 MS.Internal.TypeProxy.CreateInstance(UInt32)

.net library will be responsible for creating the visual tree and passing the buffer into agcore.dll native code and so on.

when does XAP package get downloaded?

Once NPCTRL.DLL and AGCORE.DLL are loaded, NPCTRL!CommonBrowserHost implements (shown below) npctrl!CommonBrowserHost::OnGotSourceDownloadResponse and this will make a call into agcore!CCoreServices::StartDeploymentTree which in turn calls agcore!CCoreServices::CLR_Startup and finally this will be routed to  npctrl!CWindowsServices::CreateCLRRuntimeHost to load CORECLR.DLL, see below for the callstack

kernel32!LoadLibraryW - Will load CORECLR.DLL
npctrl!GetCLRRuntimeHost
npctrl!CWindowsServices::CreateCLRRuntimeHost
agcore!CCLRHost::Initialize
agcore!CRuntimeHost::Initialize
agcore!CCoreServices::CLR_Startup
agcore!CDeployment::Start
agcore!CCoreServices::StartDeploymentTree
npctrl!CXcpBrowserHost::put_Source
npctrl!CommonBrowserHost::OnGotSourceDownloadResponsetype=”application/x-silverlight-2″

Twitter This

Username: Password:

sos.dll installed with Silverlight Runtime has more extension commands compare to mscorwks version of SOS. Although you can load coreclr version of sos with command “.loadby sos coreclr” to make sure that you have the sos loaded from the same location as your coreclr.dll, but you should also get the sos.dll when you are debugging a silverlight app dump from a customer machine because silverlight has more than one runtime version.

Below are the cool/new extension commands in silverlight version of sos to make our life easier, please note that some of these extension commands and dml were supported in .NET framework 1.1 SOS version but not in 2.0

1. DML support, you can turn it on by executing command “.prefer_dml 1″ or you can add “/D” with individual command. DML will definitely save you lot of typing as shown below

This allows you to click on object link to dump the object equivalent to( “!DumpObj /d <address>”)

2. These are the new commands in silverlight version of sos. SOS help documentation has done a great job in explaining the details on each of the command, you can execute “!help <commandname>” to get details.

GCWhere and FindRoots are very handy.

GCWhere command not only outputs the Generation number of an object but also the GC Segment and the address

FindRoots with -gen option allows you to break into your application on the next collection of specified generation number in -gen option to find out the cross generation reference to the object.

—————————–
New commands in Diagnostic Utilities
—————————–
VerifyObj
FindRoots
HeapStat
GCWhere
ListNearObj (lno)
AnalyzeOOM (ao)
!FinalizeQueue [-allReady] [-short]
—————————–
New commands to Examine the GC history   (you will need stress log file, congfigured in registry. You can get more details by executing command “!help DumpLog” )
—————————–
HistInit
HistStats
HistRoot
HistObj
HistObjFind
HistClear

Twitter This

Username: Password:

0:000> .loadby sos mscorwks
Unable to find module ‘mscorwks’
0:000> .loadby sos coreclr
Unable to find module ‘coreclr’
0:000> .load C:\Program Files\Microsoft Silverlight\2.0.40115.0\sos.dll
0:000> !clrstack
Failed to find runtime DLL (mscorwks.dll), 0×80004005
Extension commands need mscorwks.dll in order to have something to do.

unable to find mscorwks.dll means there is no clr host but if you do encounter this issue while debugging Silverlight application running in IE or any browser, verify the followings

1. Have you attached WinDbg to the correct browser’s process id?

2. Have you taken the memory dump of the process running silverlight?

If you are using IE8, it runs the tabbed browser as a child process so you will see 2 instances of iexplore.exe running. You can use Process Explorer to identify which process has loaded the coreclr.dll  or agcore.dll(silverlight runtime). See the Snapshot below

Process Explorer snapshot of IE without Silverlight Runtime

Process Explorer snapshot of IE with Silverlight Runtime

Twitter This

Username: Password:

Click to go to presentation page

Twitter This

Username: Password:

I have come across this issue quite a few times and this issue will be seen more often during deployment in QA/Production machine.

Its not unusual to see the below piece of code
using (FileStream fs = new FileStream(filePath, FileMode.Open))
{
XmlReader reader = XmlReader.Create(fs);
XmlSerializer serializer = new XmlSerializer(typeof(T));
Object o = serializer.Deserialize(reader);
}

Many times this will go unnoticed but if you look closely FileStream is using a constructor with FilePath and FileMode alone. System.IO.FileStream implementation of this constructor is

public FileStream(string path, FileMode mode) : this(path, mode, (mode == FileMode.Append) ? FileAccess.Write : FileAccess.ReadWrite, FileShare.Read, 0×1000, FileOptions.None, Path.GetFileName(path), false)
{
}

Default constructor asks for ReadWrite access, so now you see why your application is hosed in production, most of the developers use their system as admin user so of course they have the write access to the requested file. This is not to blame developers who run as admin because I can totally understand the pain.

By default, FileStream needs ReadWrite access that’s why System.UnauthorizedAccessException is thrown because on a production machine, User account under which asp.net worker process runs or a windows service or for that matter any process will not have the write access to a file by default.

Make sure, you ask for what you need. If your application needs only Read access to a file, make sure to specify that in your FileStream constructor. Don’t be GREEDY

using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))

May be I would rather see a Read access by default in System.IO.FileStream implementation rather than ReadWrite.

Twitter This

Username: Password:

Mike Semikin has reported this issue while executing windbg command .load sos2

Twitter This

Username: Password: