473,320 Members | 1,872 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

FileSystemWatcher is unreliable

We have noticed that the FileSystemWatcher is not reliable. It is not easily
repeatable but sometime it fails to catch file system changes. When it gets
into this state it doesn't recover unless a reboot is done.

The problem is only seen when multiple files are copied to the system in
quick succession. Once the watcher misses a file *any* subsequent incoming
file isn't seen.

I found this comment on the http://www.experts-exchange.com site:

I also include a "directory sweep" that periodically checks the directories
that the filewatchers are monitoring to make certain no files have been
missed and to detect files that may have been created while the filewatchers
were not running. remember if a file is placed in a directory and the file
watcher is started afterwords - the File_Created event will not fire.

It seems that others have experienced this bugger.

Any other ideas or comments?

-jeff
Feb 23 '06 #1
2 12823
> We have noticed that the FileSystemWatcher is not reliable.

I think FileSystemWatcher is implemented using win32 api
ReadDirectoryChangesW. If that is true, then the documented limitations of
ReadDirectoryChangesW will convey to FileSystemWatcher. One such limitation
is associated with many updates in a short period of time.

If all of the above is true (sorry, I really don't know), then a belt and
suspenders design is to use a timer to trigger an examination everything in
the directory of interest, in other words, polling, and you could be smart
about the polling interval. During periods of heavy file activity, decrease
the interval. During periods of file inactivity, increase the interval. A
FileSystemWatcher could set the interval to 'immediate' when it sees
something. In this way, you would process quickly when the watcher sees
something, but you will always get around to it if the watcher fails.

In this kind of design, I would use the FileSystemWatcher only as a trigger
to look at all files - I would not use it to trigger specific files. I think
if you limit the watcher to trigger only, its reliability will improve. And
you could periodically reinstantiate the watcher object. If you really have
a failure mode that requires a reboot, then I wouldn't use it at all, but my
guess is that reinstantiating it fixes whatever failures you may be
encountering.

In programming, as in boxing, protect yourself at all times. Good luck.
Feb 24 '06 #2
Arnie wrote:
We have noticed that the FileSystemWatcher is not reliable. It is not easily
repeatable but sometime it fails to catch file system changes. When it gets
into this state it doesn't recover unless a reboot is done.

The problem is only seen when multiple files are copied to the system in
quick succession. Once the watcher misses a file *any* subsequent incoming
file isn't seen.

I found this comment on the http://www.experts-exchange.com site:

I also include a "directory sweep" that periodically checks the directories
that the filewatchers are monitoring to make certain no files have been
missed and to detect files that may have been created while the filewatchers
were not running. remember if a file is placed in a directory and the file
watcher is started afterwords - the File_Created event will not fire.

It seems that others have experienced this bugger.

Any other ideas or comments?


I wrote a service that monitored an "inbox" folder and, when a file write
occured, moved the file to a work folder for processing. After processing, it
handles the next write that might have occured, and so on.

The watcher event triggered the moment the write began, meaning that my process
had to wait until the file finished. While the thread was waiting, I might have
several more files arrive; if I wait too long, the events timeout from the queue
and the files are not processed, leaving orphaned files that weren't processed.
I tried multi-threading, but that introduced other problems (such as a shared
printer object) that I was unable to solve to my satisfaction.

In the end, I threw away the FileSystemWatcher and recreated the previous, non
..NET method I used successfully for several years: sleep the main thread for a
minute, have it process up to five files from the inbox, sleep for 60 seconds,
repeat.
--
Gregory Gadow
Feb 24 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: James R. Saker Jr. | last post by:
I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable: | qsize(self) | Return the approximate size of the queue (not reliable!). Any thoughts on why this is unreliable (and...
1
by: Troy Murphy | last post by:
How do I prevent the FileSystemWatcher event to keep firing while the file is being created? When copying a file to the watched folder, the event fires a dozen or more times! Also, the...
7
by: Allen Anderson | last post by:
I'm trying to figure out a way to catch when a file has been written to a directory. I currently have it where I can catch when the file begins writing, but this isn't helpful as I need to know...
2
by: Jet Leung | last post by:
Hi all, I had made a program to watching files in my directory. I had used a instance of FileSystemWatcher to do my work.And I had add some events of the FileSystemWatcher , for example onChange,...
13
by: David | last post by:
I have been working on trying to write a directory watcher service. One of the requirments is that it be able to watch multiple directories, not sub directories of one parent directory, but just...
3
by: Stampede | last post by:
Hi, I want to use the FileSystemWatcher in a Windows Service. I read an article, where the author created the FileSystemWatcher object in a seperate thread and when the event is fired, he started...
20
by: J-T | last post by:
We are working on an asp.net application which is a 3-tier application.I was aksed to create a component which monitors a folder and gets the file and pass them to a class library in our business...
12
by: ljh | last post by:
Has anyone else noticed that the FileSystemWatcher raises the changed event twice when a file is changed? Do you have any idea why this is the case?
5
by: =?Utf-8?B?Sm9obiBT?= | last post by:
I am trying to find out if there is a way to tell if there is already a filesystemwatcher (created by a webservice) monitoring a folder. I have a webservice that creates a filesystemwatcher,...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.