Writing a dynamic, configurable windows service with better debugging experience

Download the source code

I was writing a media streaming service along with file watcher so I thought I will share this piece of code in case anyone finds it useful.  When developing Win32 Windows NT Service in old days, it used to be an exercise to debug a windows service from visual studio, I will always end up in writing a test harness to test those dependent dlls.

Although, managed code has made our life  easier but I still write a service with nothing in it other than service control manager startup.

Step 1 – Define a contract for dynamic and configurable service

public interface IDynamicService
{
void StartUp(NameValueCollection configData);
void Shutdown();
}
Step2 – Service is configurable from a config file

<configuration>
<configSections>
<sectionGroup name=”StreamingServices”>
<section name=”ServiceMap” type=”System.Configuration.NameValueSectionHandler” />
<!– List of services–>
<section name=”MediaService” type=”System.Configuration.NameValueSectionHandler” />
<section name=”MaintenanceService” type=”System.Configuration.NameValueSectionHandler” />
</sectionGroup>
</configSections>
<appSettings>
<add key=”AppName” value=”StreamingServices” />
<add key=”ServiceName” value=”MediaService” />
</appSettings>
<StreamingServices>
<ServiceMap>
<!–Start MediaService only–>
<add key=”MediaService”          value=”TRUE” />
</ServiceMap>
<MediaService>
<!– type name implementing the service–>
<add key=”Type”             value=”StreamingMedia” />
<!– assembly name implementing the type, assembly metadata table only contains the assembly name with path or extension–>
<add key=”AssemblyName”          value=”MediaService” />
<!–path to the assembly–>
<add key=”AssemblyPath”            value=”MediaService.dll” />
<!–this service watches the folders for filesystem events–>
<add key=”Folders”                value=”D:\StreamingService\Host” />
</MediaService>
<MaintenanceService>
<add key=”Type”                          value=”MaintenanceService” />
<add key=”AssemblyName”          value=”MaintenanceService” />
<add key=”AssemblyPath”            value=”MaintenanceService.dll” />
</MaintenanceService>
</StreamingServices>
</configuration>

Step 3 – create a struct for serviceinstance metadata

struct ServiceInstance
{
public NameValueCollection Configuration { get; set; }
public IDynamicService DynamicServiceInstance { get; set; }
}

Step 4 – Implementing the dynamic service

/// <summary>
///
/// </summary>
/// <param name=”appName”></param>
public DynamicService(string appName)
{
//read the service map section to iterate through list of services
NameValueCollection serviceMap = ConfigurationManager.GetSection(appName + “/ServiceMap”)
as NameValueCollection;

foreach (string serviceName in serviceMap.Keys)
{
try
{
//if the service name is true in the config file, add it to the list
//<add key=”MediaService”    value=”TRUE” />

if (Convert.ToBoolean(serviceMap[serviceName]))
{
ServiceInstance serviceInstance = new ServiceInstance();
serviceInstance.Configuration = ConfigurationManager.GetSection(appName + “/” + serviceName)
as NameValueCollection;
serviceList.Add(serviceInstance);
}
}
catch (Exception ex)
{
//log it
throw ex;
}
}
}

public bool Start()
{
//iterate through each service set to true in config file
//<add key=”MediaService”    value=”TRUE” />

for (int i = 0; i < serviceList.Count; i++)
{
try
{
ServiceInstance serviceInstance = serviceList[i];
//try to load the service using reflection
serviceInstance.DynamicServiceInstance = LoadServices(serviceInstance.Configuration);

if (serviceInstance.DynamicServiceInstance != null)
//service implementing the contract IDynamicService will be started here
serviceInstance.DynamicServiceInstance.StartUp(serviceInstance.Configuration);
}
catch (Exception ex)
{
//log it
throw ex;
}
}
return true;
}
/// <summary>
/// Shutdown all the services
/// </summary>
public void Shutdown()
{
//shutting down services in reverse order
for (int i = serviceList.Count – 1; i >= 0; i–)
{
try
{
ServiceInstance serviceInstance = serviceList[i];

if (serviceInstance.DynamicServiceInstance != null)
serviceInstance.DynamicServiceInstance.Shutdown();
}
catch (Exception ex)
{
//log it first
throw ex;
}
}
}
/// <summary>
/// this method is called recursively,  it  first loads the assembly as configured in .config
/// and then loads the type
/// </summary>
/// <param name=”assemblyPath”></param>
/// <param name=”typeName”></param>
/// <param name=”assemblyName”></param>
/// <returns></returns>

private object LoadType(string assemblyPath, string typeName, string assemblyName)
{
object objInstance = null;
bool assemblyLoaded = false;

try
{
//iterate through each assembly in the current domain to compare it with the
//configured assembly in app.config

Assembly[] assemblies = System.AppDomain.CurrentDomain.GetAssemblies();
foreach (Assembly assembly in assemblies)
{
//if assembly name is defined in the config file, load the type
//<add key=”AssemblyName”          value=”MediaService” />

if (assembly.GetName().Name == assemblyName)
{
assemblyLoaded = true;
Type[] types = assembly.GetTypes();
foreach (Type type in types)
{
if (type.Name == typeName)
{
//<add key=”Type”             value=”StreamingMedia” />
//if type name is defined, create the instance

objInstance = assembly.CreateInstance(type.FullName, false);
assemblyLoaded = true;
break;
}
}
break;
}
}
//called to load assembly from path
if (assemblyLoaded == false)
{
//<add key=”AssemblyPath”            value=”MediaService.dll” />

Assembly.LoadFrom(assemblyPath);
//recursive call here
objInstance = LoadType(assemblyPath, typeName, assemblyName);
}
}
catch (Exception ex)
{
//log message
throw ex;
}
return objInstance;
}
/// <summary>
/// read the config section to load the assembly and type
/// </summary>
/// <param name=”configData”></param>
/// <returns></returns>

private IDynamicService LoadServices(NameValueCollection configData)
{
IDynamicService objRet = null;

try
{
string assemblyPath = configData[ASSEMBLY_PATH];
string typeName = configData[TYPE_NAME];
string assemblyName = configData[ASSEMBLY_NAME];
objRet = LoadType(assemblyPath, typeName, assemblyName) as IDynamicService;
}
catch (Exception ex)
{
//log message
throw ex;
}
return objRet;
}
}

Step 5 – Since we are going to watch the file system, we should never process a file in the same thread as the filewatcher event handler(cause of missing events, blocking the thread for longer duration, buffer and etc), we should just follow a producer consumer model to queue the data and hand it over to consumer. we will create a generic thread safe queue  first

public class SyncQueue<T>
{
#region Private variables
private const int DEFAULT_CAPACITY = 25;
private object _syncRoot = null;
private Queue<T> _syncQueue = null;
private AutoResetEvent _resetEvent = new AutoResetEvent(false);
#endregion

#region Constructor
public SyncQueue()
: this(DEFAULT_CAPACITY)
{
}

public SyncQueue(int capacity)
{
_syncQueue = new Queue<T>(capacity);
_syncRoot = ((ICollection)_syncQueue).SyncRoot;
}
#endregion

#region public methods
public void Enqueue(T data)
{
if (data == null) new ArgumentNullException(“Null Argument”);
lock (_syncRoot)
{
_syncQueue.Enqueue(data);
}
_resetEvent.Set();
}

public T Dequeue()
{
lock (_syncRoot)
{
return _syncQueue.Dequeue();
}
}

public AutoResetEvent Signal
{
get { return _resetEvent; }
}

public int Count
{
get { return _syncQueue.Count; }
}

public Queue<T> Queue
{
get { return _syncQueue; }
}
#endregion
}

Step 6 – now we will define an abstract base class for File System Events, I have the FileSystemWatcher created in this base class to handle events but it may be better to have a composite type with filters and FileSystemWatcher created in a derived class for better reusability but it serves the purpose for me. I still prefer to suffix it with “Base” which is not preferred by many for abstract class

public abstract class FileSystemWatcherBase
{
#region Properties
private SyncQueue<string> FileQueue { get; set; }
private string RootPath { get; set; }
private string[] Filters { get; set; }
#endregion

#region constructors
protected FileSystemWatcherBase() { }

public void Initialize(FSStruct fsData)
{
CheckForNullArguments(fsData.FileQueue, “DataQueue<string> fileQueue”);
CheckForNullArguments(fsData.RootPath, “string rootPath”);
CheckForNullArguments(fsData.Filters, “string[] filters”);
FileQueue = fsData.FileQueue;
RootPath = fsData.RootPath;
Filters = fsData.Filters;
}
#endregion

#region public members
public virtual void Start()
{
string[] folders = Directory.GetDirectories(RootPath);
foreach (string folder in folders)
{
foreach (string filter in Filters)
{
CreateFSWatcher(folder, filter);
}
}
}
#endregion

#region protected members
protected void CreateFSWatcher(string folder, string filter)
{
FileSystemWatcher watcher = new FileSystemWatcher();
// Create a new FileSystemWatcher and set its properties.
watcher.Path = folder;
watcher.IncludeSubdirectories = true;
watcher.NotifyFilter = NotifyFilters.FileName;
watcher.Filter = filter;

// Add event handlers.
watcher.Created += new FileSystemEventHandler(OnCreated);
watcher.Error += new ErrorEventHandler(OnError);
// Begin watching.
watcher.EnableRaisingEvents = true;
}
#endregion

#region private methods

private void OnError(object source, ErrorEventArgs e)
{
Exception ex = e.GetException();
//to do—
}

private void OnCreated(object source, FileSystemEventArgs e)
{
FileQueue.Enqueue(e.FullPath);
}

private void CheckForNullArguments(object obj, string paramName)
{
if (obj == null)
throw new ArgumentNullException(paramName, “Null arguments in FileSystemWatcher Constructor”);
}

#endregion
}

#region data structure
public struct FSStruct
{
public SyncQueue<string> FileQueue;
public string RootPath;
public string[] Filters;
}
#endregion

Step 7 – Media Service to monitor and process the files

//implements the IDynamicService

public class MediaService : FileSystemWatcherBase, IDynamicService
{
private Thread workerThread;
private bool isRunning = false;
private SyncQueue<string> _fileQueue;
private AutoResetEvent _waitEvent;
public MediaService() { }

public void StartUp(NameValueCollection configData)
{
_waitEvent = new AutoResetEvent(false);
_fileQueue = new SyncQueue<string>();
string[] filters = { “*.mp4″, “*.mpeg” };
FSStruct fsData = new FSStruct()
{
FileQueue = _fileQueue,
Filters = filters,
RootPath = configData["Folders"]
};
Initialize(fsData);
Start();
}

public override void Start()
{
isRunning = true;
workerThread = new Thread(new ThreadStart(Run));
workerThread.Start();
_waitEvent.WaitOne();
Thread.Sleep(100);
base.Start();
}

public void Shutdown()
{
isRunning = false;
_fileQueue.Signal.Set();
ThreadHelper.WaitForThreadTermination(workerThread);
}

private void Run()
{
while (isRunning)
{
_waitEvent.Set();
_fileQueue.Signal.WaitOne(); // Wait for the file
while (_fileQueue.Count > 0)
{
try
{
//to do with file
}
catch (Exception ex)
{
//to do
}
}
}
}
}

Step 8 – Writing the service

Just 2 lines of code either in a console app or a Windows Service OnStart().

This will load numbers of services dynamically from a config file. Services can be removed or a new one can be added directly in a config file. You can debug it directly from a console appor a forms app making it reusable

protected override void OnStart(string[] args)
{
string appName = ConfigurationSettings.AppSettings["AppName"];
new DynamicService(appName).Start();

}

Download the source code

Thread Scheduling Priority in .NET and no idle priority thread

I have seen numerous .net application where a background thread is created for background tasks but at the same time thread scheduling priority is never set. By default, managed thread runs with normal relative priority so now if you have a thread doing some maintenance work,  that thread has no reason to run with Normal relative priority or sometime even the thread sending occasional updates to UI will run with normal relative priority

However, managed thread doesn’t allow you to set the thread priority to true THREAD_PRIORITY_IDLE class.

If you have done Win32 development then you are used to calling ::CreateThread() with Suspended flag and ::SetThreadPriority() to start thread with some priority. The default behavior of ::CreateThread is to create a thread with Normal Priority(relative to process priority class) same as managed Thread.Start(), but you will have to pass CREATE_SUSPENDED flag first.

If you insert a breakpoint in kernel32!CreateThread, you will see that Thread.Start() will call into this Win32 API because this is the only way you can create a thread in Windows OS, even if you use CRT _beginthreadex. One important difference is you don’t have an option to specify thread CreationFlags in managed code.

visit msdn documentation http://msdn.microsoft.com/en-us/library/74169f59.aspx for more information on Managed and Unmanaged Threading.

Managed Thread doesn’t support THREAD_PRIORITY_IDLE but rather maps it to THREAD_PRIORITY_LOWEST and also it doesn’t support THREAD_PRIORITY_TIME_CRITICAL.

PAL documentation says that SetThreadPriority should set THREAD_PRIORITY_IDLE to the least-priority value the host operating system allows, as the common language runtime sets threads to THREAD_PRIORITY_IDLE then busy-waits with a Sleep(0) loop until the garbage collection thread has run, with the garbage collection thread at a priority greater than idle.

Another important aspect to managed runtime is when you write your own host, you can use fiber to map one os thread to multiple managed threads with your own scheduling algorithm since fiber is implemented in usermode. Jeffrey Richter’s book “Programming Applications for Microsoft Windows” has very good description on Fiber. Windows OS supports fiber mainly for portability since other operating systems don’t have the same threading model.

But, if you do need to map managed thread priority to supported thread priority in windows , you can use System.Diagnositcs.ProcessThread. System.Diagnostics.ThreadPriorityLevel maps one to one with thread priority class in Win32 API(windows server 2003/XP).

Windows Vista/7 and windows server 2008 introduced THREAD_MODE_BACKGROUND_BEGIN and THREAD_MODE_BACKGROUND_END see the msdn documentation http://msdn.microsoft.com/en-us/library/ms686277(VS.85).aspx

If you really want to have your application to do task in background with minimal impact to system resources, you should consider setting thread priority to THREAD_MODE_BACKGROUND_BEGIN on Windows 7/2008 through interop.

One important point is, most of the managed application don’t really need a thread with idle priority, you can meet your application’s performance objectives by manipulating the thread scheduling priority less or higher than default Normal relative priority as defined in  System.Threading.ThreadPriority Enumeration