ASP.NET App Slow Response and Application Pool/AppDomain Recycle, Event message: Application is shutting down. Reason: Unknown – Windows Server 2003

Scenario
From time to time, asp.net application response is very slow on Windows Server 2003

Rants and the resolution

After turning on recycle events, logged message in application event log was Event message: Application is shutting down. Reason: Unknown. Slow response is always timed with this message in the application event log so that confirmed that Application Pool is terminating so no wonder asp.net response is slow from time to time.

However, the only missing piece was why? Since, the Reason is unknown :-) . This application pool is configured for web garden with 6 app pools in it so we decided to attach debugger in production box to 2 worker processes.

If you are just starting out with debugging or have not read John Robbins Book on debugging, I would like to stress the followings when using debugger in production environment

1. By Default, ADPlus  writes the call stack on first-chance exception. Walking call stack also results in Symbol loading, symbol loading along with the stack walking causes a performance hit when a debugger is attached. The last thing you want in production environment is to cause performance hit because of  debugger.

2. Don’t just use ADPlus script to attach a debugger to the worker process by name because it will attach the debugger to each worker process in your production server causing  further performance hit.

3. Don’t use DebugDiag in production environment unless you really have a good reason for it.

Continue reading

Share

win32 console app catch block not catching the Access Violation exception – Why SEH(Structured Exception Handling) not working in a win32 C++ app

Description

I created a simple win32 console application using visual studio 2008, below is the code

class Person
{
private:
char* m_szFirstName;
char* m_szMiddleInitial;
public:
Person(char* szFirstName, char* szMiddleInitial)

{

m_szFirstName = szFirstName;m_szMiddleInitial = szMiddleInitial

}
char* FirstName(){return m_szFirstName;}
char* MiddleInitial(){return m_szMiddleInitial;}
};

int _tmain(int argc, _TCHAR* argv[])
{
Person* pPerson = new Person(“David”,NULL);
char msg[100];
try
{
sprintf_s(msg,”First Name = %s, Middle Initial = %s\n”, pPerson->FirstName(), pPerson->MiddleInitial());
}
catch(…)
{
sprintf_s(msg,”Exception Occured\n”);
}
printf(msg);
delete pPerson;
printf(“press key to exit”);
char* key;
scanf_s(“%c”, key);
return 0;
}

When I run this app, it crashes with Unhandled Exception – Access Violation, although I do have a catch all exception block – catch(…)

Analysis

I disassembled main function and below is what I noticed

Section .text (0×00401000)
;= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
; SYM:wmain <Int>
0×401000: 83EC6C                 SUB         ESP,0x6C           ; <==0×00401207(*+0×207)
0×401003: A100304000         MOV         EAX,DWORD PTR [__security_cookie <UInt>]; (0×403000); .data:0x4E 0xE6 0×40 0xBB
0×401008: 33C4                     XOR         EAX,ESP
0x40100A: 89442468             MOV         DWORD PTR [ESP+0x68],EAX
0x40100E: 56                          PUSH        ESI
0x40100F: 57                          PUSH        EDI
0×401010: 6A08                      PUSH        0×8
0×401012: E88F000000           CALL        __imp_??2@YAPAXI@Z ; (0x4010A6) ; operator new

In case you don’t know the internals of Win32 Structured Exception handling(SEH), please read A Crash Course on the Depths of Win32™ Structured Exception Handling by Matt Pietrek. You will notice the missing instruction to move DWORD pointer from FS:[0x0] register. FS:[0x00] -  4 bytes points to the current Structured Exception Handling (SEH) frame for each thread. Win32 SEH works on a per thread basis with each thread having its own exception handler callback function. On Intel machine, FS register points to the current TEB(Thread Environment Block). That’s why in SEH code, you will have a instruction to move DWORD ptr from FS:[0] register because that’s where you have a pointer to an EXCEPTION_REGISTRATION structure.

Why is it MISSING???

It turns out, visual studio 2008 default settings compiles with “Enable C++ Exception /EHsc” switch which will catch only c++ exceptions also called synchronous exceptions. More information on Exception handling model here http://msdn.microsoft.com/en-us/library/1deeycx5(VS.80).aspx Once you change the compiler switch to /EHa, everything is good as shown below

0×401000: 55                       PUSH        EBP                ; <==0×00401267(*+0×267)
0×401001: 8BEC                   MOV         EBP,ESP
0×401003: 6AFF                   PUSH        0xFF
0×401005: 681B194000       PUSH        __CxxFrameHandler3 + 0x001D; (0x40191B); .text:0x8B 0×54 0×24 0×08
0x40100A: 64A100000000  MOV         EAX,DWORD PTR FS:[0x0]

Share