xrootd

Langue: en

Autres versions - même langue

Version: Version 4 (fedora - 01/12/10)

Section: 1 (Commandes utilisateur)

NAME

xrootd - The eXtended ROOT file server daemon

SYNOPSIS

xrootd [options] [exported directories]

DESCRIPTION

This manual page documents briefly the xrootd program.

xrootd is a ROOT remote file server daemon.

STARTING XROOTD

The simplest possible setup is one server and one or more clients:
                        +-- kan001 -----------+ 
 +-----------+        +--------+  +---------+ |      
 | Client    |------->| Server |  | Storage | | 
 | TXNetFile |        | xrootd |--|  /data  |-+ 
 +-----------+        +--------+  +---------+   
                      
 

As with the original rootd(1) server, this is relatively easy to setup. You can start it as:

 kan001> xrootd
 

Unlike the rootdserver(1), this doesn't immediately go into background, but in fact produces some amount of logging information.

Using some xrootd client application, you should be able to access a file on the server using the same syntax as with TNetFile/rootd, for example:

 root://kan001//tmp/mydatafile.root
 

From a client-side application such as a ROOT interactive session, you should be able to open the file and perform operations on it:

 root [1] myfile = TFile::Open("root://kan001//tmp/mydatafile.root");    
 root [2] myfile->ls(); 
 XTNetFile**             root://kan001//tmp/mydatafile.root
  XTNetFile*             root://kan001//tmp/mydatafile.root
   KEY: TF1      fooHist;1       gaus
   KEY: TF1      barHist;1       gaus
 root [3] myfile->Close();
 root [4] .q
 

ACCESSING AREAS OTHER THAN /tmp

One important difference from rootd(1) is that xrootd by default only allows one to access files in /tmp. This is for security reasons so that a running xrootd does not give remote clients access to read (or write to!) everything visible as xrootd.

A client accessing an area which is not explicitly exported by xrootd e.g., root://kan001//data/mydatafile.root will get an error.

You must specify explicitly any directories you wish to export via xrootd. This can be done on the command line when starting xrootd, for example:

 kan001> xrootd /data
 

will export the /data directory.

MULTIPLE SERVERS

Often it is the case that the data that an application wants to access is spread over multiple data servers.

The client application thus needs to know (or determine from some external source) on which server any given data file is located. It can then access those files directly with URL's like:

 root://kan001//data/aaaFile.root
 root://kan002//data/bbbFile.root
 

While this may work for small systems with few users, it has some disadvantages:

If users know statically where files are located it doesn't easily permit load balancing by moving files between servers or by replicating them to spread the load.
If an external catalog is read by the client in order to determine which server has the file, any changes to that catalog must be synchronized with what actually happens to the files on disk.
If the file isn't actually on disk, but is retrieved from somewhere else when the client asks for it (e.g. mass storage, discussed in later examples), there is no easy way to manage the disk space to insure that it is used efficiently and uniformly.

The xrootd system provides a method for solving these problems via its open load balancer component. This provides a method for providing transparent access data servers.

                                    +-- kan001 ----------------+
                                    |  +--------+  +---------+ |
                                  +--->| Server |  | Storage | |
                      + kanrdr +  | |  |  olbd  |  |  /data  | |
 +-----------+    +--------+   |  | +--| xrootd |--|         |-+
 | Client    |--->| Server |------+    +--------+  +---------+  
 | TXNetFile |    | xrootd |   |  | 
 +-----------+    |  olbd  |---+  | +-- kan002 ----------------+
                  +--------+      | |  +--------+  +---------+ |
                                  +--->| Server |  | Storage | |
                                  | |  |  olbd  |  |  /data  | |
                                  | +--| xrootd |--|         |-+
                                  |    +--------+  +---------+  
                                  |
                                  ...
 

In addition to the dataservers an additional machine (the redirector - kanrdr above) is added. It does not need to have any disk space itself as its sole role is to redirect the client dynamically to an actual data server machine which can serve the requested file. Thus client applications open all files via the redirector kanrdr:

 root://kanrdr//data/aaaFile.root
 root://kanrdr//data/bbbFile.root
 

The redirection to the appropriate dataserver will take place automatically (with the proper server configuration, discussed below) and hence the user running the application doesn't need to know anything about the actual layout of the data over the data servers. Note also that after redirection the client is reading directly from the data server to which it was redirected. It does need the redirector any longer (at least until it needs to open a different file).

To accomplish this a second daemon, the olbd, must be started on both the redirector and each of the dataservers as show in the figure above. Configuration of an xrootd/olbd system

OPTIONS

-l logfile
The primary advantage of using this option as opposed to simply redirecting the output to a file is that there is also a feature which does automatic log file rotation. At midnight each night xrootd will move the current logfile to logfile.yyyymmdd and then continue writing to logfile.
-p port
As with rootd(1), xrootd will started automatically on the IANA assigned port number 1094. If you need to start it on another port, that can be done using this option. Clients will need to specify this port when accessing the server, using an URL like root://kan001:2525//tmp/mydatafile.root. In general it is probably simplest to run xrootd on the standard port 1094.
-c configfile
xrootd can be configured via a config file as well as via command line options. There are in fact many more options available via the config file and hence this is the recommended way of starting xrootd is to use this option. The config file configfile can have configuration directives as described below.
-r
Flag this server as a redirector.

CONFIGURATION FILES

A configuration file for xrootd looks like
 # Start the server on port 2525
 xrd.port 2525
 # Export files in /data only
 xrootd.export /data
 

in order to achieve (for example) what was done with command line options in the sections above. Note that lines beginning with # are taken to be comments and ignored. For the moment we just note that all config lines are of the form:

 configclass.option [optval1] [optval2] [...]
 

It is also important to note that spelling errors in either configclass or option will result in the entire line being (silently) ignored.

Read-only data access
An important thing to note about xrootd access to files is that by default it allows any accessed file to be opened in update mode (and/or new files to be created). For many situations this is undesirable: the goal is simply to make data files available for others to read. The easiest thing to do in that case is simply to disallow all write access. To do this you need to add two lines to your config file:

 xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
 oss.readonly
 

in your configuration file. The first of these will cause the shared library for the optional ofs/oss file layer to be loaded (the exact path the lib area will depend on how you installed xrootd). This shared library provides a number of filesystem extensions. The second line disallows all write/update access using one of these options.

More fine-grained configurations are possible, but globally disallowing writes like this should cover a common use case.

Multiple servers
As an example we take a redirector (called kanrdr) and two data servers (called kan001 and kan002) with files aaaFile.root and bbbFile.root as above.

For the redirector we use a config file:

 #
 # redirector.cf 
 #
 # xrootd
 xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
 xrootd.export /data
 odc.manager kanrdr 3121
 odc.trace redirect
 # olbd
 olb.port 3121
 olb.allow host kanrdr.slac.stanford.edu
 olb.allow host kan001.slac.stanford.edu
 olb.allow host kan002.slac.stanford.edu
 

and for the data servers we use:

 #
 # dataserver.cf 
 #
 # xrootd
 xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
 xrootd.export /data
 oss.readonly
 odc.manager kanrdr 3121
 # olbd
 olb.port 3121
 olb.subscribe kanrdr 3121
 

As in the earlier examples, the actual path the to the libXrdOfs.so may depend on your installation. (And you should obviously substitute your machine names for kanrdr/kan001/kan002.)

It is then possible to start the xrootd and olbd servers on each of the machines.

On dataserver kan001:

 xrootd    -l kan001.xrdlog -c dataserver.cf &
 olbd -s   -l kan001.olblog -c dataserver.cf &
 

On dataserver kan002:

 xrootd    -l kan002.xrdlog -c dataserver.cf &
 olbd -s   -l kan002.olblog -c dataserver.cf &
 

And on the redirector kanrdr:

 xrootd -r -l kanrdr.xrdlog -c redirector.cf &
 olbd -m   -l kanrdr.olblog -c redirector.cf &
 

This should give you a working system capable of providing (readonly) access to data files on two servers transparently masqueraded behind a single redirector machine.

Testing the example load balanced system
Once the system is running we can try to open one of the files, for example:

 root [1] myfile = TFile::Open("root://kanrdr//data/aaafile.root");
 root [2] myfile->ls();
 XTNetFile**             root://kanrdr//data/aaafile.root
  XTNetFile*             root://kanrdr//data/aaafile.root
   KEY: TF1      fooHist;1       gaus
   KEY: TF1      barHist;1       gaus
 

Your client will be redirected and connect itself to kan001, the actual location of the requested file.

SEE ALSO

rootd(1), proofd(1), system.rootdaemonrc(1)

See also http://xrootd.slac.stanford.edu for more on the extended ROOT server.

For more information on the ROOT system, please refer to http://root.cern.ch

ORIGINAL AUTHORS

The ROOT team (see web page above):
Rene Brun and Fons Rademakers
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA

AUTHOR

This manual page was originally written by Christian Holm Christensen <cholm@nbi.dk>, for the Debian GNU/Linux system and ROOT version 3.