The modern scientific data processing is not only a collection of powerful algorithms but also a whole infrastructure of facilities for data reading, processing, and outputting the results. As the amount of data grows, so grows the need for automation of the process. Proper automation requires not only to improveme existing frameworks, but also to search for new ways to organize data processing so that it could be automated and parallelized. DataForge experimental framework solves some problems by making the analysis configuration declarative instead of imperative.
A more detailed description is available at the project site.
Repository with the current version: https://github.com/mipt-npm/dataforge-core.
Declarative analysis in “Troitsk nu-mass” experiment
Shape-based event pileup separation in Troitsk nu-mass experiment
DataForge: Modular platform for data storage and analysis