The hard part of nearly any parallel activity that aims for order over chaos is synchronisation. It strikes me that part of the problem with parallel programming is that the margin for error with regards to synchronisation can be very slim.
Some would say that we need to improve our code to fit into this margin for error, and I can agree with that, but what if we look at the issue from the other direction. What methods could we employ for increasing our margin of error? What might these methods look like?
If we treat the goal as improving adjustment to synchronisation mistakes, we'll first need to know when adjustments should be made. One way to do that is to predict how long each process is likely to take, then measure the success of this model against real world performance. In this way, a system could get better at synchronisation the longer it runs for. It doesn't matter if the original guess was way off, as long as you correct it with feedback.
Some would say that we need to improve our code to fit into this margin for error, and I can agree with that, but what if we look at the issue from the other direction. What methods could we employ for increasing our margin of error? What might these methods look like?
If we treat the goal as improving adjustment to synchronisation mistakes, we'll first need to know when adjustments should be made. One way to do that is to predict how long each process is likely to take, then measure the success of this model against real world performance. In this way, a system could get better at synchronisation the longer it runs for. It doesn't matter if the original guess was way off, as long as you correct it with feedback.