SOAP client in Windows Service - memory leak?
I am experiencing a very strange and annoying problem :-(. It's gonna take a while to describe it, so be ready to read a long post...
I have a Windows service written in C# that works as a scheduler - it loads objects implementing certain interface from other assemblies, and executes some of their methods when the time comes to do that. Nothing too complex... Let's say this interface looks like this:
public interface IPlugin
{
void Execute();
}
There's a core class - say SvcManager - which has Start(), Stop(), Pause() and Resume() methods that are called from the service' OnStart, OnStop, OnPause and OnContinue methods correspondingly. SvcManager.Start() creates one monitoring thread that is responsible for starting and aborting worker threads according to the schedule.
Worker thread creates an instance of an object implementing IPlugin interface, runs its Execute() method, and exits. The execution could take a while (several hours), so monitoring thread might need to stop the worker thread by calling Abort() method. SvcManager.Stop() aborts the monitoring thread and all worker threads (I keep all the plugin objects loaded, and running threads in the ArrayList), and waits for them to finish. Pause() and Resume() just pauses/resumes all these threads. It doesn't matter what the plugin objects do - they are all loaded from separate assemblies, and supposedly could do anything. Well, everything works perfect, unless any of the plugins implements SOAP client. After each SOAP call memory usage is increasing significantly, making it neccesserily to restart the service every 2 days or so, otherwise it eats all the memory on the machine at the rate of 100-150 Mb/day. It doesn't matter if I create a new instance of soap client proxy (inherited from SoapHttpClientProtocol) for each call and then explicitly call its Dispose() method in the finally block, or if I create just a single instance of a proxy which is called multiple times in a loop - in both cases memory leaks. I used .NET Memory Profiler from SciTech, and found that after each SOAP call a new set of objects appears in memory and never goes away - these are instances of AsyncCallback, HttpAbortDelegate, GCHandle[], Overlapped, OverlappedAsyncCallback, byte arrays and some other internal classes used by WebClientProtocol implementation - all referring to each other, and somehow keepeng themselfs in memory. When the thread executing SOAP calls exits normally - all these objects stay in memory. I tried to call GC.Collect() - it did not help. And if after Collect() I call GC.WaitForPendingFinalizers() method - it locks my app for several minutes, and still does not release the memory! But - and here comes interesting part - if I terminate the thread by calling Abort() method - the GC immediately destroys all these objects, and from now on the service starts working properly - all new threads executing SOAP calls do not fill up the memory anymore, and the counters for existing instances of all these classes stay below 10 all the time (although before aborting the thread they increase by 1000s/min), and memory does not grow. The tricky part is that thread must be aborted after at least one SOAP call has been made, and before it exits normally. Well, that's a very weird and unclean workaround for the problem :(...
Also, as you may notice, the implementation of SvcManager class makes it very easy to execute exactly the same process in a standalone Win32 app (not a service). Actually, it is even the same application - if there is certain parameter in a command line, it starts as a normal WinForms application. And if I set it up to call SvcManager.Start() on start, and SvcManager.Stop() on exit - it makes it to do exactly what the service does. And in the case of standalone app memory does not leak!!! No tricks needed with aborting threads - it just works perfectly fine from the beginning and not accumulating any of these internal objects after SOAP calls...
So I am absolutely lost now. I used to work with unmanaged code, and never had any memory leak problems in the last 6 (or more) years, just making sure to always call the destructor/release the resources in a finally block... Now, when managed environment is supposed to take care of everything, I ran into this issue which drives me crazy :((
Anyone from Microsoft, or just a guru who knows well the internals of .NET framework - please help! Even though without looking at the code it's really hard to figure out what's going on (well, it's not easy when you're staring at the code either!), maybe based on my description you can just give me some hints and lead me to a nicier workaround then the one with aborting the threads... And it's actually not a small app - with complex scheduling etc, so rewriting it from scratch is really not an option right now. I could not track down the reason for GC for not collecting these objects, I do not know what ThreadAbortException exactly does does to the threads so it magically makes GC to start working properly, and I do not see why it could happen only if app is running as a service. If somebody could shed light upon these questions, it might help a lot.
Thanks!
