Navigation Navigation

Blogs Blogs

«Back

XtremWeb-HEP 8.1.2 released

DEGISCO project partner CNRS from the Laboratoire de l'Accélérateur Linéaire in Orsay, France has released the XtremWeb-HEP 8.1.2 version.

Major points in this version include:

  • Correction: inter node communication thanks to Jason Maassen from the Netherlands eScience Center in Amsterdam
  • Correction: REST interface
  • New feature: the server embeds Web service available through HTTPS

A. Corrections:

  • A.1 on server side, the scheduling improved: hosts.incomingconnection is now taken into account to ensure a server like job is not sent to a volunteer resource where incoming connections are not allowed
  • A.2 on server side, the scheduling improved: restoring a feature from XtremWeb 1.8.0 by INRIA: "expected host" so that the end user can specify at submission time the expected volunteer resource to run the job. Note that if the expected host does not request a job or does not match the requirements (OS, CPU, ...), the job is never computed.
  • A.3 the DG to SG bridge is now compatible with shared applications so that one can use VirtualBox on EGI resources
  • A.4 some bugs corrected on client side regarding application registration and task management
  • A.5 a bug corrected on temporary file usage: they are now cleaned at software exit
  • A.6 the REST interface has been completely rewritten. The <xwhep></xwhep> root element MUST be used. Ex:

a/ to send a new application: https://server/sendapp/?XWPARAM=<xwhep version="8.0.2-flyinggrid"><app name="pouet" /></xwhep>

b/ to retrieved registered applications: https://server/getapps

  • A.7 a bug corrected in HTTP data download
  • A.8 SmartSocket usage corrected; inter node communications fully functional now

B. New features:

  • B.1 the creation of new Live CD can be customized
  • this is done by providing several optional files:
  • > a text file 'user.packages' may be provided to install custom packages: this file must contain a space separated packages list
  • > a text file 'user.hostname' may be provided to customize the LiveCD hostname: this file must contain the host name only
  • > any package files are installed on the fly (*.rpm or *.deb, depending on the LiveCD OS)
  • B.2 the client can now be used to create SmartSockets end point, independently of any task
  • this may be useful to create SmartSockets tunnel from a running VM to the client PC
  • > e.g. mount the client FS inside the running VM
  • B.3 a new Mac OS X package "xwhep.vworker" to deploy the middleware inside a VM
  • B.4 the server contains an embedded Web service interface available through HTTPS

C. Known Bugs:

  • C.1 xtremweb.gmond.pl does not scale
  • C.2 the scheduler is not fair
  • C.3 there may be some concurrent access problems leading to some errors

There may be two concurrent access problems

  • on server side, DB access may lead to inconsistency between works and tasks tables

This could certainly be solved using SQL transactions

But we would then need MySQL >=5

There are sime issues with transactions with hsqldb

  • on worker side, file access if a worker is configured to run jobs in parallel (default one thread per CPU core)

CNRS has observed that these concurrent access problems may lead to up to 6% erroneous jobs. Until further notification, CNRS considers this rate acceptable.

More information is available at http://www.xtremweb-hep.org/spip.php?rubrique15