March 20, 2017 Ben Cotton
Solving problems by the addition of abstractions is a tried and true approach in technology. The management of high-performance computing workflows is no exception.
The Pegasus workflow engine and HTCondor’s DAGman are used to manage workflow dependencies. GridWay and DRIVE route jobs to different resources based on suitability or available capacity. Both of these approaches are important, but they share a key potential drawback: jobs are still treated as distinct units of computation to be scheduled individually by the scheduler.Read more