Steep 5.4.0 has just been released
I’ve just released a new version of my workflow management system Steep. This version comes with two very nice new features: progress display and the possibility to resubmit workflows.
A rough estimation of the progress of process chains will now automatically be calculated during execution. This estimation is based on the number of actions already executed divided by the total number of actions in the process chain. This information is available through the processchains endpoint (see in particular the process chain data model). It will also be displayed in the web UI.
I’ve also added a new plugin type that can provide a more fine-grained estimation of the progress of process chains: progress estimator plugins can analyse the log output of running processing services and look for hints (e.g. percentages or number of bytes processed).
Apart from that, I’ve added a new page to the web UI that allows you to submit new workflows or to resubmit existing ones. It contains a full-fledged code editor with syntax highlighting.
The dropdown menu on the workflow detail page now contains a resubmit button. This allows you to submit a finished workflow again if, for example, your input data has changed or if you just want to retry a failed workflow. The button redirects you to the new page where you can edit the workflow before you submit it.
Steep 5.4.0 also comes with a few other improvements and fixes (see complete list below). The version has been thoroughly tested in practice over the last couple of months.
Steep is a scientific workflow management system that can execute data-driven workflows in the Cloud. It is very well suited to harness the possibilities of distributed computing in order to parallelise work and to speed up your data processing workflows, no matter how complex they are and regardless of how much data you need to process. Steep is an open-source software developed at Fraunhofer IGD. You can download the binaries and the source code of Steep from its GitHub repository.
Further new features
Besides the features mentioned above, the new version contains the following improvements:
- The executable data model has been extended to keep the service ID
- Provisioning templates can now use the new
readFilefunction to read a local file and deploy it to a VM.
- Improve UI responsiveness
- Update UI dependencies (in particular update Next.js to 9.5.1)
- Migrate UI to styled-jsx
- Update Vert.x to 3.9.2
- Update Kotlin to 1.4.0
- Fix an unnecessary page refresh in the web UI
- Render HTTP errors with correct content type (i.e. text/plain)
Posted by Michel Krämer
on 21 August 2020
In our paper, we introduce a system for fast, scalable indexing of arbitrarily sized point clouds based on a task-parallel computation model. We evaluate our system in two experiments processing datasets with several billion points.
I’m thrilled to announce that the workflow management system I’ve been working on for the last couple of years is now open-source! Read more about Steep and its features in this blog post.
The new version of the scientific workflow management system contains many new features including an improved workflow syntax, better parallelization, workflow priorities, and full-text search. It also fixes a few bugs.
I’ve just released a new version of my scientific workflow management system Steep. It introduces live process chain logs, improved VM management, and many other new features. This post summarises all changes.