Makeflow and Work Queue: Scaling Up from Laptops to Supercomputers
Nate Kremer-Herman is a second year PhD student working in the Cooperative Computing Lab at the University of Notre Dame. Nate is interested in exploring storage constraints and scaling issues in parallel applications. He is currently in charge of outreach in his research group, and he helps introduce new users to the Makeflow workflow specification language and the Work Queue execution engine. These tools are designed to help scientists scale up their research, and making sure that experience is user-friendly and approachable is one of Nate's top priorities.
This seminar presentation will introduce attendees to the workflow management system Makeflow along with an associated execution engine called Work Queue. These tools are designed to provide a generic, user-friendly, and scalable interface for workflow management. During this presentation, we will cover the steps of writing a makeflow, executing a makeflow locally, and running a makeflow via batch system using Work Queue. We will make use of a brief, hands-on tutorial. By the end of the seminar, an attendee will be able to get started writing their own makeflows and use Work Queue to effectively scale their research up to their campus- or national-scale infrastructure.