Dear FCCSW experts,
I am trying to produce some samples with the FCC EventProducer for some EWK studies. I am following the instructions in the README there. When generating LHE files with gridpacks, I get the following error about permission denied:
[ltoffoli@lxplus934 MC_prod]$ python3 EventProducer/bin/run.py --FCCee --LHE --send --condor --typelhe gp_mg -p mg_ee_zhh_ecm365 -n 10 -N 10 -q espresso --prodtag winter2023 --detector IDEA
INFO: Importing FCC-ee config…
INFO: Running version: winter2023
sending jobs
send to condor
queue espresso
priority group_u_FCC.local_gen
ncpus 1
preparing to send lhe jobs from madgraph/powheg gridpacks for process mg_ee_zhh_ecm365
Traceback (most recent call last):
File “/eos/home-l/ltoffoli/FCC/MC_prod/EventProducer/bin/run.py”, line 369, in
main()
~~~~^^
File “/eos/home-l/ltoffoli/FCC/MC_prod/EventProducer/bin/run.py”, line 270, in main
sendlhe.send()
~~~~~~~~~~~~^^
File “/eos/user/l/ltoffoli/FCC/MC_prod/EventProducer/bin/send_lhe.py”, line 69, in send
myyaml = my.makeyaml(yamldir, uid)
File “/eos/user/l/ltoffoli/FCC/MC_prod/EventProducer/common/makeyaml.py”, line 24, in makeyaml
with open(outfile, ‘w’) as outyaml:
~~~~^^^^^^^^^^^^^^`
PermissionError: [Errno 13] Permission denied: ‘/afs/cern.ch/work/f/fccsw/public/FCCDicts/yaml/FCCee//lhe/mg_ee_zhh_ecm365/events_089809387.yaml’
I feel that this error is very similar to the issue reported a long time ago in: #115
Could you please advice me how to proceed?
Thanks in advance for your support.
Best,
Leonardo
Hi @ltoffoli,
The EventProducer is primarily intended for the centralized production of common datasets, hence it writes to the “global” location for the FCCDicts (YAML files storing information about the sample/dataset). We are in the process of deprecating it in favour of the DIRAC/iLCDirac workflow.
In order to generate local samples using EventProducer, you will have to alter the hardcoded paths in config/param_FCCee.py. And even after that the scripts are quite brittle. So the better alternative might be to script the generation yourself (for running on HTCondor) or try using FLARE.
If you would like to have the samples generated centrally, and they need to be compatible with the winter2023 campaign please get in touch with @lportale. If you would like to have completely new samples then email to FCC-PED-SoftwareAndComputing-MCProduction@cern.ch. We do not have the procedure how to request new datasets defined yet, but it will be discussed and setup in next few weeks.
It appears that you run the EventProducer scripts from a folder stored on EOS, I would dicourange running from EOS and run instead from AFS, i.e. your LXPlus home or work directories.
In general, it is a good practice to store only the generated files on the EOS shared filesystem, and I’m not sure if one can submit to HTCondor from EOS using EventProducer.
Best,
Juraj
Hi @jsmiesko,
thanks very much for the insights. Indeed, I got in touch with @lportale to get some instructions already before opening this topic. I tried to change the hard-coded paths in param_FCCee.py , spacifically eosbaseoutputdir and pubbasedir (pointing to my local copy of the FCCDicts), but it didn’t help because of:
['', "ERROR: Can't find address of local schedd", '']
Also, doing something like export _CONDOR_SCHEDD_HOST=eoscondor.cern.ch before submitting didn’t change the situation.
Maybe @lportale could have some insight here?
Regarding your comment on launching from EOS, you’re right: I already experienced it in ATLAS (my primary experiment). Unfortunately I have a very limited space on afs on my computing account, 10 GB, which is already saturated by my personal ATLAS work. I have asked for a secondary computing account with associated workspace in the hope to be able to launch from there.
Thank you for the support.
Best,
Leonardo