Dynamically re-ordering task execution?
Hi all,
I'm trying to re-implement the thread scheduling system in a windows service. The primary goal of the change is to allow prioritization of tasks. The system currently uses only the features of BeginInvoke() and its reliance on the CLR's thread pool. Here is a description of the problem.
We have two conceptual ports. Each item added to port B depends on the results of items in port A. Port A's work can be scheduled hours before port B receives any work, but that is not guaranteed. Any work scheduled for port B needs to be done as fast as possible. Work in port A can typically be done during otherwise idle CPU time.
When an item in port A finishes before port B needs the results, everything seems straightforward enough.
Now, add 1000 items to port A and begin processing them. While they are still processing, add 100 items to port B. How can I increase the priority of the related items in port A so they will execute with the same urgency as the dependent items in port B? Can I pull it out of port A and throw it into something else (maybe a port C)? Is there a way predicates could be used to bring the needed items in port A to the top of the order? Would a system using extra ports somehow achieve these goals? Should I put a duplicate request of port A's entry into port C, then have the handler somehow determine whether it's already being executed by the other port?
Too bad my first CCR project couldn't have been a bit simpler!
Thanks for any suggestions.
John
I forgot to ask... If I'm forced to use a large number of ports (a couple for each item), how would that affect memory usage and performance?
Thanks.
This all doable (although very advanced), but i need more info to reply properly.
1) When tasks execute for work A (items that get posted on port A), can they run concurrently to each other? Should they all be processed in order?
2) CCR executes task fast, very fast. Unless each taskA takes significant CPU resources (more than a few micro secs) i would not even worry about all this. Your port A will be drained fast enough that all items execute anyway and port B Tasks, dont see big delays. Only bother with any more complex solutions if your A tasks really take that long or there are millions of them,
Assuming worst case, aka you have CPU intensive tasks for A, or you have millions of them, or both, i would do the following:
Assuming you want to process tasks for port A as fast as possible, you will need to attach a persisted receiver on task A, that wakes up for each item on port A, and executes some code on the first available core. Now, the way the CCR works, is that the moment you post a message on port A, if a receiver is attached, that immediately gets converted to a Task<> that gets queued up on a dispatcher queue.
For 1.5, we added an API to dispatcherQueues, that enables you to remove pending Tasks. So when items arrive on port B, use the new Dequeue.TryDequeue() method.
When items arrive on portB, you can call the DispatcherQueue.TryDequeue api and in a tight loop drain as many tasks as you can. Then you can look at the payloads for each task (Task[] is an indexer that gives you access to the arguments for the task) and the ones that are high priority, you reschedule using a new Task that bundles the port B item. Use a different DispatcherQueue instance for the re-scheduled tasks.
The rest f the Tasks you got from dispatcherQueueA, you put back in the dispatcher queue associated with portA (taking care to preserver order, altough in the mean time newer tasks might have been added)
This is a complex scenario if you really want max concurrent on port A. It has a simple solution (using a couple of ports, and the Port.Test() method) if you just need to execute one task at a time from port A.
thanx
g
Thanks for the comment, it caused me to think in several directions I hadn't considered.
I didn't describe the whole problem previously, not being able to see it as clearly as I do now. If I only had portA and portB to deal with, your solution would likely work just great. However, the software involves several steps with sub-steps. Here is a general breakdown.
Prepare (previous portA):
Step 1 - Determine type
Sub-step X
Sub-step Y
. . .
Step 2 - Do work for type
Sub-step X
Sub-step Y
. . .
Use (previous portB):
Step 1
Sub-step X
Sub-step Y
. . .
Step 2
Sub-step X
Sub-step Y
. . .
In order to get the greatest speed, the work was divided into these discrete steps. Most of these steps can run concurrently, but not all. They include things like calls to web services, converting files of one type to another, retrieving information from a database, sending emails, and other relatively lengthy activities.
Your previous solution (obviously limited by a lack of information) would be very difficult to implement, if we had to account for each step and sub-step as we looked through the tasks in the queue.
Is this a situation where each item should have it's own port or portset? I have a much easier time imagining the solution that way, since it is very easy to write joins and choices when I know that the things in the related ports will always be for the same item. However, that doesn't seem to fit my understanding of the "right" way to use the CCR. Am I wasting time looking for a different solution?
Thanks again!
John
Hi John, what you describe is actually a common scenario for the CCR, and is widely used in DSS (our service model) and is actually a much simpler case than i originally thought.
Remember that CCR ports are essentially free (they are very small overhead) and we usually attach one per pending asynchronous operation (see the User guide for some simple service examples, where each pending request has an associated response port). I strongly recommend you also review the DSS user guide, since it shows how CCR is used in the context of asynchronous services that interact with each other using CCR ports, request, response ports for each request, etc (apologies if you have done so already)
So in your prepare phase, i would issue the operations, attach a response port on each one. Dont even bother attaching a Choice (we usually use choice, since we want to dela with success or falure from the same port) on the response ports of each pending async operation. They will complete at some time, and they will post their result on their respective response ports.
When you issue your io requests (using some wrapper around your calls to the database, web services, file i/o etc), kep track of them in some list, dictionary, etc. For examples of how to wrap non CCR async operations, with CCR apis, see the Concurrent affairs article or once again the user guide. Its quite simple: just supply a CCR PortSet to the async api, and in the callback of the non-ccr api, post a success or exception message to the CCR port...
Ok, now in your second phase, where you need to check the result of some previously submitted requests, lookup the request you need from your table, then using simply Test() on the response port, or attaching a Arbiter.Choice, wait for that request to complete. If that response is already waiting on the port, your receiver will execute right away and you can go ahead with B. Since none of your phase A operations are blocking, re-prioritizing them will not get you anything. They all complete independently anyway an dhave been submitted to external components not under your control
Thanks, George. I didn't look at the DSS user guide, since I wasnt' going to use it. I'll definitely take a look, now, since that would have corrected my assumption that many ports was undesirable.
