Attemping to import lhe file into FCC for LAr full simulation

Hello,
I am attempting to do what the subject field says.

I followed some advice from Clement, who adviced me then to go here.

My problem:

a) As a start I try to run:
FCCSW/pythia.py at master · HEP-FCC/FCCSW · GitHub

This does not work with my local FCCSW installation:

 lxplus734:~/public/ECalSimple/FCCSW$ fccrun runPythia.py
Traceback (most recent call last):
File "/cvmfs/[sft.cern.ch/lcg/views/LCG_97a_FCC_4/x86_64-centos7-gcc8-opt/scripts/fccrun](http://sft.cern.ch/lcg/views/LCG_97a_FCC_4/x86_64-centos7-gcc8-opt/scripts/fccrun)", line 33, in <module>
exec(open(f).read())
File "<string>", line 19, in <module>
ImportError: cannot import name k4DataSvc

b) Then Clement advised me to try the lcg installation my executing (in a clean shell)
source /cvmfs/sw.hsf.org/key4hep/setup.sh

 $ which fccrun
   /cvmfs/sw.hsf.org/spackages2/fccsw/1.0pre01/x86_64-centos7-gcc8.3.0-opt/zppwpxxeiu66wwavl75t2tcwx4aqf7rb/bin/fccrun

Now, after this I try:

 $ fccrun runPythia.py --filename ../mytutorial/Pythia_ee_ZH_Zmumu_ecm240.cmd 

which gives me:

 PYTHIA Error in Pythia::readFile: did not find file /cvmfs/sw.hsf.org/spackages2/k4gen/0.1pre02/x86_64-centos7-gcc8.3.0-opt/nyi7pl2fn5zuj65majyzpffuu6pokt6i/share/k4Gen/data/Pythia_standard.cmd

Trying to debug:

$ echo $K4GEN
      /cvmfs/sw.hsf.org/spackages2/k4gen/0.1pre02/x86_64-centos7-gcc8.3.0-opt/nyi7pl2fn5zuj65majyzpffuu6pokt6i/share/k4Gen
$ ls $K4GEN

ls: cannot access /cvmfs/sw.hsf.org/spackages2/k4gen/0.1pre02/x86_64-centos7-gcc8.3.0-opt/nyi7pl2fn5zuj65majyzpffuu6pokt6i/share/k4Gen: No such file or directory

$ ls /cvmfs/sw.hsf.org/spackages2/k4gen/0.1pre02/x86_64-centos7-gcc8.3.0-opt/nyi7pl2fn5zuj65majyzpffuu6pokt6i/
 lib/  python/

Obviously there is not share folder where $K4GEN is pointing.

Those are the problems I see.
Any other advice on the topic of the subject field is very welcome.

Best,
-Mogens

Hi Mogens,

This is a problem with the pythia interface that does not seem to accept to change the command file (.cmd) on the command line. I have discovered a few days ago but did not find yet the time to investigate it in depth.
In the mean time you should edit the pythia.py file and change the lines

# take from $K4GEN if defined, locally if not
path_to_pythiafile = os.environ.get("K4GEN", "")
pythiafilename = "data/Pythia_standard.cmd"
pythiafile = os.path.join(path_to_pythiafile, pythiafilename)

to

pythiafile = ../mytutorial/Pythia_ee_ZH_Zmumu_ecm240.cmd

(or put the full path if you want to run it from elsewhere …).
Please try and let us know.

The variables such as K4GEN are defined automatically to point to the ../share/<package>. Of course in this case it does not make much sense. Another thing to fix.

Thanks for reporting.

Gerri

Thank you for this, Gerri!

Now I believe I have a setup which reads my lhe file after I use the cvfms version of FCCSW
That is my runPythia.py file

lxplus721:~/public/ECalSimple/FCCSW$ vdir -rt *.py
-rw-r–r–. 1 dam zp 5424 May 20 06:44 runCaloSim.py
-rw-r–r–. 1 dam zp 2208 May 20 20:11 runPythia.py

However, my configuration which runs the full simulation of the LAr calorimeter (runCaloSim.py) does not seem to like this verion on FCCSW. That seem to run only with my local FCCSW.

So, how on Earth to merge the two???
I am afraid it could take me ages to figure out…
Everything is in my ~/public/ECalSimple/FCCSW folder.
If anybody has the courage to take a look at the two files and can suggest how to merge them, I would be very grateful.

Best,
-Mogens

Hello @dam ,

if I remember correctly your local FCCSW version does contains changes wrt to the master branch.
If that is the case, then you might need to merge remote changes inside you local version, and recompile.
A way to see the local changes is to run a git status and then a git diff on each file that has some modifications. Before doing a git pull you would need to git commit -a the changes.
But first let’s see if you have many local modifications
Cheers,
Clement

Thank you Clement,

However, I do believe (99% confidence) that my only modification was the one suggested by Jana on how to save the photon truth information from photons from pi0 decays. That worked (I beleive) but we never used it. Point being, that (of course) this information was saved as SimParticles and not as GenParticles (or whatever the two are called). To handle this would require a non-trivial modification of my analysis code. All of this information I hope to get “for free” (in GenParticles), when inputting generated tau decays via lhe.

I believe all other modifications only reside in xml files: detector geometry, etc. But they are extremely valuable. I have both LAr and a LKr detectors with our simplified geometry ready to run.

So, what is the easiest way to proceed?
To check out from git in a new directory, and manually move all modified xml files there?

Sorry, I am really rather ignorant when it comes to modern software (and have very little time to learn). Even something as basic as git makes me nervous.

Best,
-Mogens

@dam, as git is not part of the topic here, do you have a channel that we could use to chat directly (mattermost, skype?), if yes just ping me by email and we can try to merge your local changes with upstream

@dam @ganis I’m just seeing this topic now. The PythiaInterface is, in fact, handling the command line arguments correctly, it is just that the argument names are poorly chosen, the option is --Filename with a capital F. --filename is an argument for PodioOutput and will set the filename for the root file in which the output will be written (The arguments also have a long form in which this is slightly clearer: --PythiaInterface.Filename and --out.filename).

This is objectively terrible, but we didn’t change this yet because it would have meant breaking a lot of workflows, but since we are anyway forced to do some breaking changes due to edm4hep, this is probably a good opportunity to do this. What do you think of --pythiacard?

@dam I’m also happy to help with pushing things upstream. Taking a look at your public folder now

@vavolkl , I spend some time with @dam on this and it turns out the issue was simply him trying to adapt a too recent P8 joboption. Indeed, Mogens is not ready to transition to edm4hep yet, and urgently needed to run his old setup with LHE as inputs. Now we have fixed that and it is running as it should. But I will let him comment if he thinks this project can be conducted with edm4hep dataformat, in which case we could certainly help with the transition. For that he needs to understand the amount of work needed to adapt his downstream analysis code to a new input format

@vavolkl, thanks for clarifying, --pythiacard seems indeed a much better name to me. Thanks!