FileSystemWatcher - Created events fires too early
I have been playing with a FileSystemWatcher, to start processing files as soon as one arrives in a folder. Specifically, I'm looking for a zip file, then unzip it.
However, when I try to unzip a file as soon as I get the Created event, the file has often not finished being written, and therefore I get an error when I try to unzip it. This time period is very short, a tight loop without pauses trying to open it for write will return OK in 3 loops.
Question
=======
How do I get FileSystemWatcher to raise an event when a file has FINISHED being written, instead of START being written?
[620 byte] By [
Anarchy] at [2007-12-23]
This could be the most horrible code I've ever writ...
If anyone knows a better way of doing this, please let me know
#########################
Public Function WaitTillCanOpenFileForWrite(ByVal strFileName As String) As Boolean
Return WaitTillCanOpenFileForWrite(strFileName, 0)End FunctionPublic Function WaitTillCanOpenFileForWrite(ByVal strFileName As String, ByVal intTimeOut As Integer) As BooleanDim blnOK As Boolean = TrueDim nI As Integer = 0Dim dtStart As DateTime = NowDo While CanOpenFileForWrite(strFileName) = FalsenI += 1
'WriteToLog(" Trying to open file:" & strFileName & ", tries:" & nI.ToString, LogError.LogError.HIGH)Threading.Thread.Sleep(10)
If intTimeOut > 0 AndAlso DateDiff(DateInterval.Second, dtStart, Now) >= intTimeOut ThenblnOK =
FalseExit DoEnd IfLoop'WriteToLog(" file opened", LogError.LogError.HIGH)Return blnOKEnd FunctionPublic Function CanOpenFileForWrite(ByVal strFileName As String) As BooleanDim nI As Integer = 0Dim fs As FileStream = NothingDim blnOK As BooleanTryfs =
New FileStream(strFileName, FileMode.Open, FileAccess.Write)blnOK =
TrueCatch ex As ExceptionblnOK =
FalseFinallyIf fs IsNot Nothing Thenfs.Flush()
fs.Close()
fs.Dispose()
End IfEnd TryReturn blnOKEnd FunctionIn short, the FileSystemWatcher will fire two created events when a new file is created. There is no way to descriminate between them. Windows does not include a CreatedFinished event unfortunately. Your workaround is similar to ones I've done in the past, essentially trying to do work in a try-catch block and sleeping and re-trying on the catch. Ugly, but effective.
Jim Wooley
http://devauthority.com/blogs/jwooley
ugly but functional will do for me thanks Jim
You say you get 2 events? I only get one!
Have you added eventhandlers for your FSW? If your FSW is a compnent dropped on a form, you don't need to do this, as you end up with 2 eventhandlers for one event
The FileSystemWatcher component also has a "changed" event, in addition to its "created" event. After the file is created, you will receive changed events as the zip file is copied / saved into your target folder. Once the changed events stop firing, the file is done being modified.
I would recommend that you use a timer component in conjunction with the FileSystemWatcher component, setting the time to something around 1 second or so. When you receive the "created" event you can then start the timer and then start listing to "changed" events from the FileSystemWatcher. Every time you get a changed event you can then reset the timer. Once the timer fires, you can then go ahead and do your processing.
This solution is not perfect, as you do not actually know for sure that the program creating the Zip file is done creating it, only that a specific period of time (the timer interval) has elapsed since the file was last updated. In practice, however, this should work about 99% of the time.
-Scott Wisniewski
MS VB Compiler Dev
Thanks for the reply Scott, that was a method I was possibly going to use when I get round to putting a wrapper round the FSW, when I can raise my own FileCreatedStarted & FileCreatedFinished events. If/when I do this, I'll post it up, if anyone wants it.
It seems you get 1 'Created' event & 3 'Changed' events when a file is created (copied into a folder). I need to do more investigation to see if this is the same whenever a 'created' event happens - might not be the same if a file is renamed rather than copied in.
Must say I was pretty keen on using FSW when I started reading about it, but it's been really difficult! If a component doesn't work as you'd expect it, it then takes days of testing to find out how it does actually work, then figure out how to work-around a problem that works properly. I suspect that the FSW was written by a 3rd party for Microsoft, it simply doesn't 'feel' like other components, and is less logical.
This is a back-handed compliment for MS, btw
I just wanted to warn you that you can't depend on a particular number of Changed Events. The number of changed events you receive will vary based on the size of the file. For example, a fairly large file will fire many more than 3 changed events, and a small file may only fire 1.
I'm sorry to hear about your difficulties with the file system watcher control. I'll forward your feedback over to the windows forms team.
-Scott Wisniewski
thanks for the warning, I'm going to look into this next week, and will post back whatever I come up with for anyone's use/comments/feedback
gracias
Here goes.... Not fully tested, and could be a bit flaky, but I'm going home in 20 mins, so I thought I'd post this tonight
This class mimics the behaviour of the FileSystemWatcher, with these changes...
The changed event has been replaced by a StartChanged event
only 1 changed event per file is fired <-- this bit could be flaky!
After a changed event, I check to see if I can open the file for write, and when I can, I fire a EndChanged event
The created event has been replaced by a StartCreated event
After a created event, I check to see if I can open the file for write, and when I can, I fire a EndCreated event
I'd very much appreciate any feedback, thanks folks
########################
Imports System.IO
Imports
System.ThreadingImports
System.Runtime.InteropServicesPublic
Class FSW#
Region "Members"'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ The FileSystemWatcher'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Private WithEvents MyFSW As FileSystemWatcher'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ The FileSystemWatcher events & my new events (End)'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Public Event StartCreated(ByVal strFileName As String)Public Event EndCreated(ByVal strFileName As String)'~Public Event StartChanged(ByVal strFileName As String)Public Event EndChanged(ByVal strFileName As String)'~Public Event Renamed(ByVal strFileName As String)Public Event Deleted(ByVal strFileName As String)'~Public Event Disposed(ByVal sender As Object, ByVal e As System.EventArgs)Public Event fswError(ByVal sender As Object, ByVal e As System.IO.ErrorEventArgs)'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ When I get a changed event, store the name of the file here,'~ & do not fire subsequent events if the filename is the same'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Private m_strFileChanged As String = String.Empty#
End Region#
Region "New/Dispose"Public Sub New()MyFSW =
New FileSystemWatcherEnd SubPublic Sub Dispose()If MyFSW IsNot Nothing ThenMyFSW.Dispose()
MyFSW =
NothingEnd IfEnd Sub#
End Region#
Region "Public properties"'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Pulic properties of FSW'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Public Property Path() As StringGetReturn MyFSW.PathEnd GetSet(ByVal value As String)MyFSW.Path = value
End SetEnd PropertyPublic Property Filter() As StringGetReturn MyFSW.FilterEnd GetSet(ByVal value As String)MyFSW.Filter = value
End SetEnd PropertyPublic Property NotifyFilter() As NotifyFiltersGetReturn MyFSW.NotifyFilterEnd GetSet(ByVal value As NotifyFilters)MyFSW.NotifyFilter = value
End SetEnd PropertyPublic Property EnableRaisingEvents() As BooleanGetReturn MyFSW.EnableRaisingEventsEnd GetSet(ByVal value As Boolean)MyFSW.EnableRaisingEvents = value
End SetEnd Property#
End Region#
Region "FSW Event Handlers"Private Sub MyFSW_Created(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Created'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ When I get a CREATED event, raise my StartCreated event'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~RaiseEvent StartCreated(e.FullPath)'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Create a thread which will raise my EndCreated event '~ When it can open the file for write'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Dim T As Thread = New Thread(New ParameterizedThreadStart(AddressOf WaitForEndOfCreate))T.IsBackground =
TrueT.Start(e.FullPath)
End SubPrivate Sub MyFSW_Changed(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Changed'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ If the file is the same as for the last changed event, do nothing'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~If e.FullPath.Equals(m_strFileChanged) Then'~Else'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Set m_strFileChanged to this file so doesn't keep'~ raising Changed events for the same file'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~m_strFileChanged = e.FullPath
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ When I get a CHANGED event, raise my StartChanged event'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~RaiseEvent StartChanged(e.FullPath)'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Create a thread which will raise my EndChanged event '~ When it can open the file for write'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Dim T As Thread = New Thread(New ParameterizedThreadStart(AddressOf WaitForEndOfChange))T.IsBackground =
TrueT.Start(e.FullPath)
End IfEnd SubPrivate Sub MyFSW_Deleted(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles MyFSW.Deleted'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Set this to "" so will raise Changed events again for this file'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~If m_strFileChanged.Equals(e.FullPath) Thenm_strFileChanged =
""End IfRaiseEvent Deleted(e.Name)End SubPrivate Sub MyFSW_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles MyFSW.Renamed'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Set this to "" so will raise Changed events again for this file'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~If m_strFileChanged.Equals(e.FullPath) Thenm_strFileChanged =
""End IfRaiseEvent Renamed(e.Name)End SubPrivate Sub MyFSW_Disposed(ByVal sender As Object, ByVal e As System.EventArgs) Handles MyFSW.DisposedRaiseEvent Disposed(sender, e)End SubPrivate Sub MyFSW_Error(ByVal sender As Object, ByVal e As System.IO.ErrorEventArgs) Handles MyFSW.ErrorRaiseEvent fswError(sender, e)End Sub#
End Region#
Region "WaitForfile"Private Sub WaitForEndOfCreate(ByVal objFile As Object)Dim strFile As String = TryCast(objFile, String)If strFile Is Nothing Then'~ ElseIf WaitTillCanOpenFileForWrite(strFile) ThenRaiseEvent EndCreated(strFile)End IfEnd IfEnd SubPrivate Sub WaitForEndOfChange(ByVal objFile As Object)Dim strFile As String = TryCast(objFile, String)If strFile Is Nothing Then'~ ElseIf WaitTillCanOpenFileForWrite(strFile) Then'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Set this to "" so will raise Changed events again for this file'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~If m_strFileChanged.Equals(strFile) Thenm_strFileChanged =
""End IfRaiseEvent EndChanged(strFile)End IfEnd IfEnd SubPrivate Function WaitTillCanOpenFileForWrite(ByVal strFileName As String) As BooleanReturn WaitTillCanOpenFileForWrite(strFileName, 0)End FunctionPrivate Function WaitTillCanOpenFileForWrite(ByVal strFileName As String, ByVal intTimeOut As Integer) As BooleanDim blnOK As Boolean = TrueDim nI As Integer = 0Dim dtStart As DateTime = NowDo While CanOpenFileForWrite(strFileName) = FalsenI += 1
'X Debug.Print(" Trying to open file:" & strFileName & ", tries:" & nI.ToString) ', LogError.LogError.HIGH)Threading.Thread.Sleep(10)
If intTimeOut > 0 AndAlso DateDiff(DateInterval.Second, dtStart, Now) >= intTimeOut ThenblnOK =
FalseExit DoEnd IfLoopReturn blnOKEnd FunctionPrivate Function CanOpenFileForWrite(ByVal strFileName As String) As BooleanDim nI As Integer = 0Dim fs As FileStream = NothingDim blnOK As BooleanTry'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'~ Try to open the file for write, if it exceptions, no can do'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~fs =
New FileStream(strFileName, FileMode.Open, FileAccess.Write)blnOK =
TrueCatch ex As ExceptionblnOK =
FalseFinallyIf fs IsNot Nothing Thenfs.Flush()
fs.Close()
fs.Dispose()
End IfEnd TryReturn blnOKEnd Function#
End RegionEnd
ClassSkimming over it, I would only make a couple minor suggestions:
1) Use the standardized signature for the events using 2 parameters (sender as Object, e as FSWEventArgs). Wrap the filename into the custom event args class. This way the event signature would agree with the standard signature of events in the framework.
2) Consider an incremental addition to the sleep, that way it will sleep longer in each iteration. This would help to alleviate denial of service attacks. In addition, you may want to include a tolerance level (perhaps in a config setting) whereby the control could fire an exception if it has waited too long.
3) Consider inheriting from Component (or even FileSystemWatcher) so that the control can be dragged on the form just like the standard one. If you inherit from FileSystemWatcher, you could simply trap the events from the base class and raise your custom events as necessary. (Also, if you go the route of inheriting from FileSystemWatcher, you wouldn't need to create your own subclassed EventArgs in step 1.)
Otherwise, good work. Thank you for posting your solution so far. It might help others with similar issues.
Jim Wooley
http://devauthority.com/blogs
Thanks for the tips Jim
1. Agreed
2. I like the incremental idea. There's a parameter called intTimeOut (seconds) which can be passed when waiting for a file. I'd thought of what to do if it's been waiting for a file for like 5 minutes, and it's not easy to work out if there is a real problem or not, I'll give it more thought...
3. Yeah that would have been easier, but wasn't sure how to do that, or overide events etc, so I did it the hard way!
I'm starting to make a new version which inherits FileSystemWatcher, as you suggested. I want to get rid of the Changed & Created events, and replace them with my own, can I do this?
There are some others with this question in another forum, their solution is the same, basically try to open and wait...
Sysinternals have a utility that can tell you about files that are
currently open (Process Explorer) so there ought to be an interface
somewhere, perhaps they may share the how if you ask.
http://www.thescripts.com/forum/thread225491.html
Good Luck.
Hi,
I have created FSW application in which I have only one createde event and corresponding event handler. After a file is dropped in a designated folder it is moved to another foldere. It works fine only once. When I drop another file into designated folder it does not move to another folder. I am using follwoing code:
if(file.Exists) { lock (m_SynLock)
{
file.MoveTo(pathToMove);
} }
Can any one please explain why created event is not fired repetativly as file are dropped into designated folder?
Thank you.
Bharat Gadhis