FileSystemWatcher - Created events fires too early

I have been playing with a FileSystemWatcher, to start processing files as soon as one arrives in a folder. Specifically, I'm looking for a zip file, then unzip it.

However, when I try to unzip a file as soon as I get the Created event, the file has often not finished being written, and therefore I get an error when I try to unzip it. This time period is very short, a tight loop without pauses trying to open it for write will return OK in 3 loops.

Question
=======

How do I get FileSystemWatcher to raise an event when a file has FINISHED being written, instead of START being written?

[620 byte] By [Anarchy] at [2007-12-23]
# 1

This could be the most horrible code I've ever writ...

If anyone knows a better way of doing this, please let me know

#########################
Public Function WaitTillCanOpenFileForWrite(ByVal strFileName As String) As Boolean

Return WaitTillCanOpenFileForWrite(strFileName, 0)

End Function

Public Function WaitTillCanOpenFileForWrite(ByVal strFileName As String, ByVal intTimeOut As Integer) As Boolean

Dim blnOK As Boolean = True

Dim nI As Integer = 0

Dim dtStart As DateTime = Now

Do While CanOpenFileForWrite(strFileName) = False

nI += 1

'WriteToLog(" Trying to open file:" & strFileName & ", tries:" & nI.ToString, LogError.LogError.HIGH)

Threading.Thread.Sleep(10)

If intTimeOut > 0 AndAlso DateDiff(DateInterval.Second, dtStart, Now) >= intTimeOut Then

blnOK = False

Exit Do

End If

Loop

'WriteToLog(" file opened", LogError.LogError.HIGH)

Return blnOK

End Function

Public Function CanOpenFileForWrite(ByVal strFileName As String) As Boolean

Dim nI As Integer = 0

Dim fs As FileStream = Nothing

Dim blnOK As Boolean

Try

fs = New FileStream(strFileName, FileMode.Open, FileAccess.Write)

blnOK = True

Catch ex As Exception

blnOK = False

Finally

If fs IsNot Nothing Then

fs.Flush()

fs.Close()

fs.Dispose()

End If

End Try

Return blnOK

End Function

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 2

In short, the FileSystemWatcher will fire two created events when a new file is created. There is no way to descriminate between them. Windows does not include a CreatedFinished event unfortunately. Your workaround is similar to ones I've done in the past, essentially trying to do work in a try-catch block and sleeping and re-trying on the catch. Ugly, but effective.

Jim Wooley
http://devauthority.com/blogs/jwooley

jwooley at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 3

ugly but functional will do for me thanks Jim

You say you get 2 events? I only get one!
Have you added eventhandlers for your FSW? If your FSW is a compnent dropped on a form, you don't need to do this, as you end up with 2 eventhandlers for one event

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 4

The FileSystemWatcher component also has a "changed" event, in addition to its "created" event. After the file is created, you will receive changed events as the zip file is copied / saved into your target folder. Once the changed events stop firing, the file is done being modified.

I would recommend that you use a timer component in conjunction with the FileSystemWatcher component, setting the time to something around 1 second or so. When you receive the "created" event you can then start the timer and then start listing to "changed" events from the FileSystemWatcher. Every time you get a changed event you can then reset the timer. Once the timer fires, you can then go ahead and do your processing.

This solution is not perfect, as you do not actually know for sure that the program creating the Zip file is done creating it, only that a specific period of time (the timer interval) has elapsed since the file was last updated. In practice, however, this should work about 99% of the time.

-Scott Wisniewski
MS VB Compiler Dev

scottwis_MS at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 5

Thanks for the reply Scott, that was a method I was possibly going to use when I get round to putting a wrapper round the FSW, when I can raise my own FileCreatedStarted & FileCreatedFinished events. If/when I do this, I'll post it up, if anyone wants it.

It seems you get 1 'Created' event & 3 'Changed' events when a file is created (copied into a folder). I need to do more investigation to see if this is the same whenever a 'created' event happens - might not be the same if a file is renamed rather than copied in.

Must say I was pretty keen on using FSW when I started reading about it, but it's been really difficult! If a component doesn't work as you'd expect it, it then takes days of testing to find out how it does actually work, then figure out how to work-around a problem that works properly. I suspect that the FSW was written by a 3rd party for Microsoft, it simply doesn't 'feel' like other components, and is less logical.

This is a back-handed compliment for MS, btw

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 6

I just wanted to warn you that you can't depend on a particular number of Changed Events. The number of changed events you receive will vary based on the size of the file. For example, a fairly large file will fire many more than 3 changed events, and a small file may only fire 1.

I'm sorry to hear about your difficulties with the file system watcher control. I'll forward your feedback over to the windows forms team.

-Scott Wisniewski

scottwis_MS at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 7

thanks for the warning, I'm going to look into this next week, and will post back whatever I come up with for anyone's use/comments/feedback

gracias

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 8

Here goes.... Not fully tested, and could be a bit flaky, but I'm going home in 20 mins, so I thought I'd post this tonight

This class mimics the behaviour of the FileSystemWatcher, with these changes...

The changed event has been replaced by a StartChanged event
only 1 changed event per file is fired <-- this bit could be flaky!
After a changed event, I check to see if I can open the file for write, and when I can, I fire a EndChanged event

The created event has been replaced by a StartCreated event
After a created event, I check to see if I can open the file for write, and when I can, I fire a EndCreated event

I'd very much appreciate any feedback, thanks folks

########################
Imports System.IO

Imports System.Threading

Imports System.Runtime.InteropServices

Public Class FSW

#Region "Members"

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ The FileSystemWatcher

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Private WithEvents MyFSW As FileSystemWatcher

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ The FileSystemWatcher events & my new events (End)

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Public Event StartCreated(ByVal strFileName As String)

Public Event EndCreated(ByVal strFileName As String)

'~

Public Event StartChanged(ByVal strFileName As String)

Public Event EndChanged(ByVal strFileName As String)

'~

Public Event Renamed(ByVal strFileName As String)

Public Event Deleted(ByVal strFileName As String)

'~

Public Event Disposed(ByVal sender As Object, ByVal e As System.EventArgs)

Public Event fswError(ByVal sender As Object, ByVal e As System.IO.ErrorEventArgs)

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ When I get a changed event, store the name of the file here,

'~ & do not fire subsequent events if the filename is the same

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Private m_strFileChanged As String = String.Empty

#End Region

#Region "New/Dispose"

Public Sub New()

MyFSW = New FileSystemWatcher

End Sub

Public Sub Dispose()

If MyFSW IsNot Nothing Then

MyFSW.Dispose()

MyFSW = Nothing

End If

End Sub

#End Region

#Region "Public properties"

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Pulic properties of FSW

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Public Property Path() As String

Get

Return MyFSW.Path

End Get

Set(ByVal value As String)

MyFSW.Path = value

End Set

End Property

Public Property Filter() As String

Get

Return MyFSW.Filter

End Get

Set(ByVal value As String)

MyFSW.Filter = value

End Set

End Property

Public Property NotifyFilter() As NotifyFilters

Get

Return MyFSW.NotifyFilter

End Get

Set(ByVal value As NotifyFilters)

MyFSW.NotifyFilter = value

End Set

End Property

Public Property EnableRaisingEvents() As Boolean

Get

Return MyFSW.EnableRaisingEvents

End Get

Set(ByVal value As Boolean)

MyFSW.EnableRaisingEvents = value

End Set

End Property

#End Region

#Region "FSW Event Handlers"

Private Sub MyFSW_Created(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Created

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ When I get a CREATED event, raise my StartCreated event

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

RaiseEvent StartCreated(e.FullPath)

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Create a thread which will raise my EndCreated event

'~ When it can open the file for write

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Dim T As Thread = New Thread(New ParameterizedThreadStart(AddressOf WaitForEndOfCreate))

T.IsBackground = True

T.Start(e.FullPath)

End Sub

Private Sub MyFSW_Changed(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Changed

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ If the file is the same as for the last changed event, do nothing

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If e.FullPath.Equals(m_strFileChanged) Then

'~

Else

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Set m_strFileChanged to this file so doesn't keep

'~ raising Changed events for the same file

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

m_strFileChanged = e.FullPath

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ When I get a CHANGED event, raise my StartChanged event

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

RaiseEvent StartChanged(e.FullPath)

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Create a thread which will raise my EndChanged event

'~ When it can open the file for write

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Dim T As Thread = New Thread(New ParameterizedThreadStart(AddressOf WaitForEndOfChange))

T.IsBackground = True

T.Start(e.FullPath)

End If

End Sub

Private Sub MyFSW_Deleted(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Deleted

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Set this to "" so will raise Changed events again for this file

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If m_strFileChanged.Equals(e.FullPath) Then

m_strFileChanged = ""

End If

RaiseEvent Deleted(e.Name)

End Sub

Private Sub MyFSW_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles MyFSW.Renamed

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Set this to "" so will raise Changed events again for this file

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If m_strFileChanged.Equals(e.FullPath) Then

m_strFileChanged = ""

End If

RaiseEvent Renamed(e.Name)

End Sub

Private Sub MyFSW_Disposed(ByVal sender As Object, ByVal e As System.EventArgs) Handles MyFSW.Disposed

RaiseEvent Disposed(sender, e)

End Sub

Private Sub MyFSW_Error(ByVal sender As Object, ByVal e As System.IO.ErrorEventArgs) Handles MyFSW.Error

RaiseEvent fswError(sender, e)

End Sub

#End Region

#Region "WaitForfile"

Private Sub WaitForEndOfCreate(ByVal objFile As Object)

Dim strFile As String = TryCast(objFile, String)

If strFile Is Nothing Then

'~

Else

If WaitTillCanOpenFileForWrite(strFile) Then

RaiseEvent EndCreated(strFile)

End If

End If

End Sub

Private Sub WaitForEndOfChange(ByVal objFile As Object)

Dim strFile As String = TryCast(objFile, String)

If strFile Is Nothing Then

'~

Else

If WaitTillCanOpenFileForWrite(strFile) Then

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Set this to "" so will raise Changed events again for this file

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If m_strFileChanged.Equals(strFile) Then

m_strFileChanged = ""

End If

RaiseEvent EndChanged(strFile)

End If

End If

End Sub

Private Function WaitTillCanOpenFileForWrite(ByVal strFileName As String) As Boolean

Return WaitTillCanOpenFileForWrite(strFileName, 0)

End Function

Private Function WaitTillCanOpenFileForWrite(ByVal strFileName As String, ByVal intTimeOut As Integer) As Boolean

Dim blnOK As Boolean = True

Dim nI As Integer = 0

Dim dtStart As DateTime = Now

Do While CanOpenFileForWrite(strFileName) = False

nI += 1

'X Debug.Print(" Trying to open file:" & strFileName & ", tries:" & nI.ToString) ', LogError.LogError.HIGH)

Threading.Thread.Sleep(10)

If intTimeOut > 0 AndAlso DateDiff(DateInterval.Second, dtStart, Now) >= intTimeOut Then

blnOK = False

Exit Do

End If

Loop

Return blnOK

End Function

Private Function CanOpenFileForWrite(ByVal strFileName As String) As Boolean

Dim nI As Integer = 0

Dim fs As FileStream = Nothing

Dim blnOK As Boolean

Try

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

'~ Try to open the file for write, if it exceptions, no can do

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

fs = New FileStream(strFileName, FileMode.Open, FileAccess.Write)

blnOK = True

Catch ex As Exception

blnOK = False

Finally

If fs IsNot Nothing Then

fs.Flush()

fs.Close()

fs.Dispose()

End If

End Try

Return blnOK

End Function

#End Region

End Class

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 9

Skimming over it, I would only make a couple minor suggestions:

1) Use the standardized signature for the events using 2 parameters (sender as Object, e as FSWEventArgs). Wrap the filename into the custom event args class. This way the event signature would agree with the standard signature of events in the framework.

2) Consider an incremental addition to the sleep, that way it will sleep longer in each iteration. This would help to alleviate denial of service attacks. In addition, you may want to include a tolerance level (perhaps in a config setting) whereby the control could fire an exception if it has waited too long.

3) Consider inheriting from Component (or even FileSystemWatcher) so that the control can be dragged on the form just like the standard one. If you inherit from FileSystemWatcher, you could simply trap the events from the base class and raise your custom events as necessary. (Also, if you go the route of inheriting from FileSystemWatcher, you wouldn't need to create your own subclassed EventArgs in step 1.)

Otherwise, good work. Thank you for posting your solution so far. It might help others with similar issues.

Jim Wooley
http://devauthority.com/blogs

jwooley at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 10

Thanks for the tips Jim

1. Agreed

2. I like the incremental idea. There's a parameter called intTimeOut (seconds) which can be passed when waiting for a file. I'd thought of what to do if it's been waiting for a file for like 5 minutes, and it's not easy to work out if there is a real problem or not, I'll give it more thought...

3. Yeah that would have been easier, but wasn't sure how to do that, or overide events etc, so I did it the hard way!

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 11

I'm starting to make a new version which inherits FileSystemWatcher, as you suggested. I want to get rid of the Changed & Created events, and replace them with my own, can I do this?

Anarchy at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 12

There are some others with this question in another forum, their solution is the same, basically try to open and wait...

Sysinternals have a utility that can tell you about files that are

currently open (Process Explorer) so there ought to be an interface

somewhere, perhaps they may share the how if you ask.

http://www.thescripts.com/forum/thread225491.html

Good Luck.

Rabtok at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...
# 13

Hi,

I have created FSW application in which I have only one createde event and corresponding event handler. After a file is dropped in a designated folder it is moved to another foldere. It works fine only once. When I drop another file into designated folder it does not move to another folder. I am using follwoing code:

if(file.Exists) {

lock (m_SynLock)

{

file.MoveTo(pathToMove);

} }

Can any one please explain why created event is not fired repetativly as file are dropped into designated folder?

Thank you.

Bharat Gadhis

BharatGadhia at 2007-8-30 > top of Msdn Tech,Visual Basic,Visual Basic Language...