Higher Edu - Research dev card
Development from the higher education and research community
  • Creation or important update: 03/01/12
  • Minor correction: 03/01/12

XtremWeb-HEP : middleware for distributed data processing

This software was developed (or is under development) within the higher education and research community. Its stability can vary (see fields below) and its working state is not guaranteed.
  • Web site
  • System: UNIX-like, Windows, MacOS X
  • Current version: 7.6.4 - 12/12/2011
  • License(s): GPL
  • Status: validated (according to PLUME)
  • Support: maintained, ongoing development
  • Designer(s): Oleg LODYGENSKY
  • Contact designer(s): xtremweb (at) lal.in2p3.fr
  • Laboratory, service: LAL


General software features
  • ’XtremWeb-HEP’ is a middleware for Distributed Data Processing (grids) :
    –  It permits Administrators to :
        - manage various Users and Applications, by providing adequate access rights to them,
        - catalog various Data and Computing Resources :
           Â· PC farms managed by an IT department,
           Â· PC grids contributed by volunteer citizens,
    –  It permits Users to submit Jobs referencing these Applications,
    –  From Job descriptions, it dynamically deploys and executes these Applications on available Computing Resources, then it provides the results to authorized Users,
    –  It protects the Computing Resources running Mac OS X by starting the Application inside MAC OS X Sandbox,
    –  For the access to data, it permits the usage of HTTP, HTTPS, and any URI scheme whose driver is provided by the User.

Secured three tiers Architecture.  Scheduler and data repository managed by a software administrator on a server;  Client installed on the machine of each User (for ex. scientist);  Worker installed on the resource of each contributor.

  • Soon will come in production the version of XtremWeb-HEP additionally managing the submission of complete virtual machines for execution inside VirtualBox.

  • Interoperability with other grid middleware stacks :
    –  XtremWeb-HEP accepts X509 certificates and proxies for user management, in particular those of the DEGISCO international project.
    –  XtremWeb-HEP integrates a bridge permitting suitable XtremWeb-HEP jobs to be accepted by the gLite middleware in order to be executed by the EGI European infrastructure.
    –  On the other way, thanks to the 3G Bridge of the EDGI European project, the resources gathered by XtremWeb-HEP are available for the many users of the EGI infrastructure (gLite, ARC and Unicore middleware stacks).

  • Domain, Infrastructures, Documentation and Maintenance :
    –  In spite of its name, XtremWeb-HEP is used way beyond High Energy Physics :  Biology,  ADN Research,  Mathematics,  Physics of Solids,  Signal Processing.
    –  XtremWeb-HEP is powering at least 2 production grids (For each grid, look at the 'Statistics' page) :
        - http://www.xtremweb-hep.org/lal/xw_lal/
        - http://xw.lri.fr:4330/XWHEP
    –  XtremWeb-HEP has a complete up to date set of user manuals, presented at http://www.xtremweb-hep.org/spip.php?rubrique16
    –  XtremWeb-HEP is maintained by the software team presented at http://www.xtremweb-hep.org/spip.php?rubrique35 and is strongly supported by Institut des Grilles et du Cloud, INRIA, ENS Lyon, GRID5000

Context in which the software is used
  • Distributed Data Processing
  • Distributed Computing
  • Resource Sharing
  • Computing Grid (PC Grid)
  • Job Submission
Publications related to the software
  • Hybrid Distributed Computing Infrastructure Experiments in Grid5000 : Supporting QoS in Desktop Grids with Cloud Resources   http://users.lal.in2p3.fr/lodygens/gc/g5k.pdf
    G. Fedak, S. Delamare, O. Lodygensky.   Grid 5000 School, Reims, France - April 18-21, 2011

  • Extending the EGEE grid with XtremWeb-HEP Desktop Grid   http://users.lal.in2p3.fr/lodygens/gc/PCGrid2010.pdf
    H. He, G. Fedak, P. Kacsuk, Z. Farkas, Z. Balaton, O. Lodygensky, E. Urbah, G. Caillat, F. Aurajo, A. Emmen.   4th Workshop on Desktop Grids and Volunteer Computing Systems, Melbourne, Australia - May 17-20, 2010

  • EDGeS : Bridging EGEE to BOINC and XtremWeb   http://users.lal.in2p3.fr/lodygens/gc/EDGeS-Bridgi...
    E. Urbah, P. Kacsuk, Z. Farkas, G. Fedak, G. Kecskemeti, O. Lodygensky, A. Marosi, Z. Balaton, G. Caillat, G. Gombas, A. Kornafeld, J. Kovacs, H. He, and R. Lovas.   JoGC Journal of Grid Computing, Volume 7, Number 3, 2009.