Port inverse mask / or unwanted files cleanup?
I am sure this has been addressed in the community beforehand but I could not find it...so asking.
Our application expects files fitting a particular mask (such asORDINVC*.xml) where * is the datestamp, etc in a Receive Port location.
But the client is interested to know about unwanted files in those receive ports - to report on or raise an alert. For e.g., if a file came in with a wrong mask (as a result of a typo - sayORDERINVOICE*.xmlinstead of the expectedORDINVC*.xml) - it should be caught as an exception, logged and/or redirected to an archive, so support can look into it and inform the sender.
We dont know what mistakes can be made so we cannot set a positive mask for all errors to trap files, but can we set a negative mask for a port - effectively meaning - send or redirect to another port all files which dont matchORDINVC*.xml) ?
Or something to this effect with another mechanism? This seems an interesting challenge!
CodeDigger,
I'm not aware of a way to do this directly with the built-in File adapter. Usually, I'd probably try to workaround this in one of two ways:
1- Write a simple service that preprocesses the files: it watches the main folder for new files, and copies the matching ones to one folder and the rest to another, and then use biztalk to monitor the two separate files with separate receive locations.
2- There's always the option of writing a custom receive adapter that can do this (possibly based on the sample DotnetFile adapter in the BizTalk SDK)... it can potentially be the most "elegant" solution, but it sure is a lot more complex.
Thanks Tomas,
I had thought of both but I was hoping for a minimum code solution - such as a context property filter I could set (that I did not know about) on a Send port to subscribe to those that matched the filter.
Could I create a property schema for a File mask on a pass-through port and check the value with something like (pseudocode)
BTS.FILEReceiveLocationFileMask != "ORDINVC*.xml"
This would be a valuble and useful context property for BizTalk since it does not exist yet (and all the others which no one ever uses do).
CodeDigger,
I'm not sure the property would do any good; since the send adapter doesn't really monitor files like the receive side adapter does (remember the file adapter is one way), and even if it could do it, it still would only be able to return a single message and not all matching files 
This would be monitored by the FILE (or other protocol) Receive Adapter, not the Send Adapter. The Send adapter would only subscribe to those messages picked up by the Receive adapter which match the filter set with this context property. For the same reason, the Receive adapter would pick up all messages matching the filter, not just one.
CodeDigger; Sorry, I misunderstood your last message; thought you meant doing that from an orchestration or something (which obviously makes no sense).
What you propose wouldn't quite be possible because it would mean inferring the file mask from the actual file. However, you could do it the other way around using a custom pipeline component in the receive pipeline and a custom property.
For example, you could define a custom boolean property called My.MatchesMask, and then write a custom receive pipeline component that tool the mask ("ORDINVC*.XML") in a property value. The component woiuld then evaluate the FILE.ReceivedFileName property value of the received message, and if it matches the mask in the component property, write the My.MatchesMask property into the message context with a value of true.
Then it would just be a matter of setting up a filter on the other side based on whether the My.MarchesMask property is present and has a value of true.
OK now we are getting somewhere.
But why can't I simplify further without any custom pipeline.
Can I not just use a filter like this
FILE.ReceivedFileName != "ORDINVC*.xml"
as the subscription for a SEND port - which could then send those to another receive port - which would trigger an alert.
(only question is - does a RegExp like ORDINVC*.xml work for the FILE.ReceivedFileName ?
CodeDigger,
No, you can't, as I said, that would imply that either the subscription engine inside the message box would need to know how to apply glob expressions to strings (i.e. do the * dance), which it can't do at this time.The other thing you propose would be impossible, as well, because it would mean the File adapter would need to magically figure out a mask for a received file; and there's an large number of arbitrary masks a file name could potentially match. It may not be convinient, but that's the limitations that are in place right now.
Hi,
One of the solutionis:
Pick up all files to an orchestration.
Orchestration can use RegExes or BRE to implement your custom logic.
Then this orch drops good files to the RL.
Leonid,
Yes, that's another option, thanks for putting it forward; for some reason that didn't occur to me . I still think the custom pipeline component route would be better if performance is important, though.
Tomas,
Do you mean that pipeline can avoid the persistent points which would hardly be avoided in the orchestration?
From other side the logic in BRE should be more funny implemented
and quite fast.
Leonid,
Yes, persistence points are one issue, but I was also thinking that the orchestration alternative would force all messages to go through two orchestrations (assuming the initial routing orchestration did just that), which would penalize correct messages just to detect the ones incorrectly named (which *should* be a minority). The pipeline alternative avoids that by doing a single routing, which you're doing anyway in the message box, and thus significantly reduces this overhead.
Tomas,
I think you're right!
