A configuration file (fcl file) should be developed and verified
with interactive jobs. Using it with the production system imposes
one extra requirement: all output files of the job should satisfy
the Mu2e naming conventions, as explained on the
tapeUpload page.
The values of the owner, configuration,
and sequencer fields used in the prepared fcl file are
not important, because they will be overridden later. The values of
data_tier, description,
and file_format will be used as is and must be set
correctly. Use the "art" extension ("file format") for framework
outputs (written by RootOutput modules) and "root"
for TFileService outputs.
Prepare a shell where you can run the Mu2e Offline job interactively (i.e. source setup.sh in Offine). In that shell
setup mu2etools
The above makes available the generate_fcl command.
There are two invocation modes that require mutually exclusive sets of parameters
EmptyEvent input source one has to specify
--run-number that will be used for all the generated fcl files--events-per-job--njobsRootInput source require--inputs a list of input data files for the set of jobs--merge-factor how many input files should be analyzed by a single jobThe following parameters are used to construct the names of fcl
files produced by the generate_fcl invocation.
--description the "description" field of fcl file names--dsconf the "configuration" field--dsowner the "owner" field. This parameter
defaults to the user who executes generate_fcl, but
can be overridden. For example, --dsowner=mu2e
should be used to generate an official dataset of fcl files.
The --old-seeds parameter can be used
for incremental generation of fcl datasets. For example, one can
generate a test batch of 1000 fcl files and run them through the
grid. If the result is satisfactory, and one wants to increase the
statistics to 10,000 jobs, care should be taken to guarantee the
uniqueness of random seeds across all the 10,000 jobs. Each run
of generate_fcl produces a text file that contains the
values of all random seeds used so far for the current set of jobs.
So when generate_fcl is used the second time to add
9,000 jobs to the dataset, one should use the file with 1000 seeds
from the first run for the --old-seeds parameter to
make sure those seeds are not re-used.
(Also, --first-subrun should be adjusted so that subrun
numbers do not repeat.) The second run will dump a list of 10,000
seeds, which can be used in a subsequent generation if a further
increase in statistics is desired. For the initial run you can
specify --old-seeds=/dev/null.
Run generate_fcl --help to see all the options.
A first stage simulation job with no input files.
We start in a new empty directory
mkdir -p /mu2e/data/users/`whoami`/fclds cd /mu2e/data/users/`whoami`/fclds mkdir 20161121-pions cd 20161121-pions
Prepare a template file. Usually it can be a single line file with just an include directive, in our case
#include "JobConfig/cd3/pions/pions_g4s1.fcl"
but one can also add e.g. geometry file overrides, or even write a
completely new fcl configuration and use it as a template. In this example
we use template.fcl as the template file name.
Note that the include directive should specify include file pathname
relative to the Offline directory. (More precisely, relative to
a directory listed in the FHICL_FILE_PATH.) Absolute filenames
do not work in fhicl #include.
Now generate the files:
generate_fcl --description=my-test-s1 \
--dsconf=v0 \
--run=2700 \
--events=1000 \
--njobs=5 \
template.fcl
After the command completes, we will see something like
mu2egpvm05 /mu2e/data/users/gandr/fclds/20161121-pions$ ls 000 template.fcl seeds.gandr.my-test-s1.v0.Td6j.txt mu2egpvm05 /mu2e/data/users/gandr/fclds/20161121-pions$ ls 000 cnf.gandr.my-test-s1.v0.002700_00000000.fcl cnf.gandr.my-test-s1.v0.002700_00000002.fcl.json cnf.gandr.my-test-s1.v0.002700_00000000.fcl.json cnf.gandr.my-test-s1.v0.002700_00000003.fcl cnf.gandr.my-test-s1.v0.002700_00000001.fcl cnf.gandr.my-test-s1.v0.002700_00000003.fcl.json cnf.gandr.my-test-s1.v0.002700_00000001.fcl.json cnf.gandr.my-test-s1.v0.002700_00000004.fcl cnf.gandr.my-test-s1.v0.002700_00000002.fcl cnf.gandr.my-test-s1.v0.002700_00000004.fcl.json
The generated fcl files and their corresponding json files are written into subdirectories 000, 001, etc, with up to 1000 fcl files per subdirectory. Random number seeds used for all the fcl files have been dumped into the "seeds" file.
Start in a new empty directory
mkdir -p /mu2e/data/users/`whoami`/fclds cd /mu2e/data/users/`whoami`/fclds mkdir 20161121-ce-reco cd 20161121-ce-reco
Prepare a template file. We want to use
JobConfig/cd3/beam/dra_mix_baseline.fcl, but in many
Offline releases this file is, strictly speaking, not a valid fcl
because it references a variable bgHitFiles that is not
defined. (A legacy of the mu2eart way of running grid
jobs.) To fix that, we define the variable before including the
"baseline" file. This example template.fcl also
shows how to set names of output histogram files.
BEGIN_PROLOG bgHitFiles: @nil END_PROLOG #include "JobConfig/cd3/beam/dra_mix_baseline.fcl" services.TFileService.fileName: "nts.owner.my-ce-reco.ver.seq.root"
We also need a list of input files, and a list of background overlay files. We want to run 5 jobs with one input file per job, so we need to shorten the conversion list to have only 5 input files. We only need one background hits file per job, but there is no harm of listing more background overlay files than necessary, so we will use a complete detmix-cut dataset:
setup mu2efiletools mu2eDatasetFileList sim.mu2e.cd3-beam-g4s4-detconversion.v566.art | head -n 5 > inputs.txt mu2eDatasetFileList sim.mu2e.cd3-detmix-cut.v566b.art > backgrounds.txt
We are ready to generate the fcl dataset. Note the '@' symbol in the --aux parameter - it says that bgHitFiles should be devined in a PROLOG, as the included file expects instead of being appended at the end.
generate_fcl --desc=my-reco-test \
--dsconf=v0 \
--inputs=inputs.txt \
--merge=1 \
--aux=1:@bgHitFiles:backgrounds.txt \
template.fcl
Take a look:
mu2egpvm05 /mu2e/data/users/gandr/fclds/20161121-ce-reco$ ls 000 backgrounds.txt template.fcl inputs.txt seeds.gandr.my-reco-test.v0.CRdP.txt mu2egpvm05 /mu2e/data/users/gandr/fclds/20161121-ce-reco$ ls 000 cnf.gandr.my-reco-test.v0.004001_00000000.fcl cnf.gandr.my-reco-test.v0.004001_00000002.fcl.json cnf.gandr.my-reco-test.v0.004001_00000000.fcl.json cnf.gandr.my-reco-test.v0.004001_00000003.fcl cnf.gandr.my-reco-test.v0.004001_00000001.fcl cnf.gandr.my-reco-test.v0.004001_00000003.fcl.json cnf.gandr.my-reco-test.v0.004001_00000001.fcl.json cnf.gandr.my-reco-test.v0.004001_00000004.fcl cnf.gandr.my-reco-test.v0.004001_00000002.fcl cnf.gandr.my-reco-test.v0.004001_00000004.fcl.json
It is highly recommended to test a newly generated fcl file by running a small interactive job. Following up on Example 1 above, one can do
mkdir test cd test /usr/bin/time mu2e -c /mu2e/data/users/`whoami`/fclds/20161121-pions/000/cnf.`whoami`.my-test-s1.v0.002700_00000000.fclto run a full size job, or add a
-n 10 option to the mu2e command line
to quickly make sure that there are no obvious problems with the configuration.
|
|
| Security, Privacy, Legal |
|
