Data archiving
Higher Edu - Research dev card
Development from the higher education and research community
  • Creation or important update: 25/10/10
  • Minor correction: 28/10/10

EDNA : framework for plugin-based applications for online data analysis

This software was developed (or is under development) within the higher education and research community. Its stability can vary (see fields below) and its working state is not guaranteed.
  • Web site
  • System:
  • Current version: 1.0 (Application MXv1 june 2009) - revision SVN 1930
  • License(s): GPL -

    EDNA kernel and many plugins are LGPLv3+

    EDNA applications (like MXv1) and some plugins are GPLv3+

  • Status: stable release, internal use, under development
  • Support: maintained, ongoing development
  • Designer(s):

    Executive Committee Chair: Andrew Leslie (MRC-LMB)

    Project Manager: Olof Svensson (ESRF)

    Project Coordinator: Alun Ashton (Diamond)

    Currently around 10 developers but 40 people have been involved in the project.

  • Contact designer(s):
  • Laboratory, service: DLS, MRC-LMB, CCP4, Bessy (HZB), Max Lab. NSLS, SLS-PSI, Univ Sydney, Univ York, Global Phasing


General software features

The EDNA kernel runs on all platforms which can provide a Python interpreter. The MXv1 application runs on all Unix/Linux platforms that have bash, Python (version 2.5 and later) and which can run the programs MOSFLM, LABELIT, RADDOSE and BEST. The EDNA framework and MXv1 application are installed and regularly used at the Diamond Light Source (Didcot), EMBL Hamburg, ESRF Grenoble and NSLS Brookhaven National Laboratory (New York).

EDNA-based applications are designed and developed with the aim of being easily configurable, extensible and smoothly maintainable. This has been made possible thanks to the technical facilities that the EDNA framework provides, including configuration facilities, a library of re-usable components, datamodel-driven code generation machinery and a testing framework.

The components are organized in a logical class hierarchy that makes it straightforward to develop new functional plugins by deriving them from the appropriate parent. Two families of plugins have been designed: the first branch contains the execution plugins (EDPluginExec classes) that are responsible for the execution of a particular action (e.g. execution of third-party software); the second branch contains the control plugins (EDPluginControl classes) that are responsible for the data flow (propagation of the data), the workflow (sequential or parallel execution of appropriate execution plugins) and the error-tracking mechanism (propagation of the errors).

The EDNA kernel provides a data model tool kit that allows the construction of elaborate data models needed by advanced applications. In addition, it allows the design of unitary data models that can be unit-tested, so that a plugin can be launched and tested independently of any application context. This tool kit consists of generic low-level class definitions including general types (XSDataString, XSDataFloat etc.) and X-ray experiment classes which can be re-used when designing specific components data models. It is available in several standard data formats including XMI (XML metadata interchange) and XSD (XML schema definition) in order to facilitate importing into UML (unified modeling language) data modeling tools and/or XSD files. The framework also provides (external) code generation machinery generateDS that allows automatic code generation from UML diagrams to Python code via XSD format.

Testing framework
A testing framework has been developed and integrated with the kernel in order to test the applications and the components easily and efficiently. To ensure the reliability and robustness of the components, the testing framework provides all the necessary utilities to check a class (EDTestCase) and a plugin (EDTestCasePlugin) either in a unitary manner (EDTestCasePluginUnit) or by testing its execution in an application context (EDTestCasePluginExecute). These families of tests can be automatically launched via test suites. An automatic analysis of the test results is performed by comparing the obtained result with the expected one (assert mechanism), so that a successful test proves that a result conforms to the expectation. This allows a high degree of confidence when implementing new components or re-implementing features of existing components.

Configuration facilities
Each plugin is configured (path of the controlled executable, batch queuing system configuration, ...) by the mean of XML files. The configuration is selected according to the EDNA_SITE environment variable.

Context in which the software is used

EDNA is used to create online data analysis applications on synchrotron beam-lines (but not only), among them:

  • Mxv1 application for fast characterization of protein crystals; in regular use at the ESRF, the DLS and the NSLS.
  • MXv2 application is in preparation to take advantage of kappa goniostats.
  • Dimple application has been written by CCP4 for doing molecular replacement and ligand location in proteins.
  • Diffraction computed tomography application with azimuthal integration and online sinogram generation with dynamic region of interest for Nano-Analysis Beamlines.
  • Saxs pre-processing (azimuthal integration and averaging) and downstream processing for BioSaxs Beamline.
  • A Tutorial for Raw digital camera development that explains how to take advantage of the parallel capabilities of EDNA.
Publications related to the software

EDNA: a framework for plugin-based applications applied to X-ray experiment online data analysis
M.-F. Incardona, G. P. Bourenkov, K. Levik, R. A. Pieritz, A. N. Popov and O. Svensson
J. Synchrotron Rad. (2009). 16, 872-879 [ doi:10.1107/S0909049509036681 ]

Abstract: EDNA is a framework for developing plugin-based applications especially for online data analysis in the X-ray experiments field. This article describes the features provided by the EDNA framework to ease the development of extensible scientific applications. This framework includes a plugins class hierarchy, configuration and application facilities, a mechanism to generate data classes and a testing framework. These utilities allow rapid development and integration in which robustness and quality play a fundamental role. A first prototype, designed for macromolecular crystallography experiments and tested at several synchrotrons, is presented.

Syndicate content