Is there a way to have a stable version of the stack?

Whenever I open a fresh shell if I want to get the FCC software I have to do

cd FCCeePhysicsPerformance/case-studies/flavour/tools
source /cvmfs/fcc.cern.ch/sw/latest/setup.sh
source ./localSetup.sh $PWD/localPythonTools
cd ../../../../FCCAnalyses/
source ./setup.sh
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install
make install
cd ../../FCCAnalyses/examples/FCCee/flavour/b2snunu/

If there are any changes in the remote files in CVMFS then often this will break and I will have to update my fork of FCCAnalyses, pull, rebase and rebuild. Sometimes that is enough for things to go smoothly but sometimes not and I end up wasting time trying to figure out what has broken. Is there a way I can pick some particular previous version of the stack to build against such that I will never have to pull any changes from master? This way I can finish up my study much quicker and then when I’m finished align all my work with the latest stack.

Thanks for any help!

Hello @aiwieder,

It seems to me that you are sourcing too many setup files. Do you have your analysis somewhere public?

In any case, the /cvmfs/fcc.cern.ch/sw/latest/setup.sh points to the latest version of the stack. When you are sourcing it, it should tell you which version was actually sourced, like this:

source /cvmfs/fcc.cern.ch/sw/latest/setup.sh
Setting up the latest Key4HEP software stack from CVMFS ...
 ...  Key4HEP release: key4hep-stack/2023-04-08
 ... Use the following command to reproduce the current environment: 
 ... 
         source /cvmfs/sw.hsf.org/spackages7/key4hep-stack/2023-04-08/x86_64-centos7-gcc11.2.0-opt/urwcv/setup.sh
 ... 
 ... done.

If you later source the /cvmfs/sw.hsf.org/spackages7/key4hep-stack/2023-04-08/x86_64-centos7-gcc11.2.0-opt/urwcv/setup.sh script, you should end up with the same stack.

Best,
Juraj

Hi @jsmiesko,

Maybe my method is slightly outdated. You can see my FCCAnalysis repository here: GitHub - AidanWiederhold/FCCAnalyses: FCC Physics Analyses using RDataFrame
For FCCeePhysicsPerformance it’s: GitHub - AidanWiederhold/FCCeePhysicsPerformance at aiwieder/b2snunu

For both repositories I’ve been working in the branch aiwieder/b2snunu.

To set up my environment from scratch I use: FCCAnalyses/setup.sh at aiwieder/b2snunu · AidanWiederhold/FCCAnalyses · GitHub

Then whenever I have a fresh shell I use: FCCAnalyses/startup.sh at aiwieder/b2snunu · AidanWiederhold/FCCAnalyses · GitHub

The reason I have this local tools stuff is so I can use Snakemake and a couple other packages as they aren’t in the stack. Is there a cleaner way to do this?

Looking at FCCAnalyses could I do:

source setup.sh
fccanalysis pin
fccanalysis build

and then pip any extra python packages on top (or just after the initial source?)?

Then from a fresh shell would it be enough to do

source ./setup.sh
fccanalysis build

to have the same stack setup each time or would I need to re-pip things each time?

Hello @aiwieder,

Pinning of the analysis can be done at any time after source setup.sh and stays recorded in hidden file until you do unpin.

and then pip any extra python packages on top

Yap. Maybe it is worth trying creating requirements.txt file with

seaborn
uproot
awkward
zfit
xgboost
root-pandas
scikit-learn

and creating a virtual environment:

source ./setup.sh
python -m venv venv
source ./venv/bin/activate
pip install -r requirements.txt

For the regular use it will boil down to two commands:

source ./setup.sh
source ./venv/bin/activate

Do you use anythin defined here:

https://github.com/AidanWiederhold/FCCeePhysicsPerformance/tree/aiwieder/b2snunu/case-studies/flavour/tools/python

Best,
Juraj

Do you use anythin defined here:

https://github.com/AidanWiederhold/FCCeePhysicsPerformance/tree/aiwieder/b2snunu/case-studies/flavour/tools/python

I don’t think so, if I did used to depend on it I probably moved it to FCCAnalyses anyway.

Thanks, I’ll try with the virtual env and see how things go.

@jsmiesko I think this is working, thanks. However after pulling some changes from master, to try stay in step with stack changes, a little while ago I noticed some issues showing in my stage1 logs that hadn’t been there before. So I would like to try rewind my local FCCAnalysis to a time when these weren’t showing. If I ls /cvmfs/sw.hsf.org/spackages7/key4hep-stack/ which is where my current pin is it shows 2023-04-08, how can I find old versions of the stack so I can try pin to one of them (or manually source them if I can’t pin to them)? Then I’ll rewind any changes in my git that aren’t directly for my analysis so I can just worry about aligning with master when I’m finished with my study.

I think I’ve sorted this now, I found /cvmfs/sw.hsf.org/spackages6/key4hep-stack/2023-01-15/x86_64-centos7-gcc11.2.0-opt/csapx/setup.sh. I checked in my working branch aiwieder/b2snunu when one of my commits were from when I last ran my workflow successfully so made a new branch aiwieder/reverted based on my fork of master, reverted it to around when that commit of mine was, cherry-picked my commits from aiwieder/b2snunu to get all my analysis work I needed.

So now in aiwieder/reverted I source /cvmfs/sw.hsf.org/spackages6/key4hep-stack/2023-01-15/x86_64-centos7-gcc11.2.0-opt/csapx/setup.sh, build FCCAnalysis, make the venv and then run my stage1 analysis. In my log now the only issue I see is

/cvmfs/sw.hsf.org/spackages6/py-numpy/1.22.3/x86_64-centos7-gcc11.2.0-opt/r7bmv/lib/python3.9/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/cvmfs/sw.hsf.org/spackages6/py-numpy/1.22.3/x86_64-centos7-gcc11.2.0-opt/r7bmv/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
/cvmfs/sw.hsf.org/spackages6/py-numpy/1.22.3/x86_64-centos7-gcc11.2.0-opt/r7bmv/lib/python3.9/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/cvmfs/sw.hsf.org/spackages6/py-numpy/1.22.3/x86_64-centos7-gcc11.2.0-opt/r7bmv/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  return self._float_to_str(self.smallest_subnormal)

I was getting this in the more up-to-date stack but for now I’m going to assume this won’t be an issue so will move on as is and see how things pan out.

It seems because I reverted so much I can’t use the fccanalysis pin, fccanalysis build etc… commands but can get by just sourcing the particular version of the stack.

Thanks for all the help @jsmiesko!

Hi @jsmiesko,

Actually I am still having some slight trouble, I can no longer edit my local FCCAnalyses package and use what I change. e.g. I am trying to edit the PID function in myUtils.cc but after my edit and building I get

input_line_150:2:254: error: too many arguments to function call, expected 4, have 5
auto lambda35 = [](ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData>& var0, ROOT::VecOps::RVec<Int_t>& var1, ROOT::VecOps::RVec<Int_t>& var2, ROOT::VecOps::RVec<edm4hep::MCParticleData>& var3){return FCCAnalyses::myUtils::PID(var0, var1,var2,var3, 0.)
                                                                                                                                                                                                             ~~~~~~~~~~~~~~~~~~~~~~~~~                       ^~
/cvmfs/sw.hsf.org/spackages6/fccanalyses/0.6.0/x86_64-centos7-gcc11.2.0-opt/qo45d/include/FCCAnalyses/myUtils.h:363:3: note: 'PID' declared here
  ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData> PID(ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData> recop,
  ^
input_line_154:2:254: error: too many arguments to function call, expected 4, have 5
auto lambda35 = [](ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData>& var0, ROOT::VecOps::RVec<Int_t>& var1, ROOT::VecOps::RVec<Int_t>& var2, ROOT::VecOps::RVec<edm4hep::MCParticleData>& var3){return FCCAnalyses::myUtils::PID(var0, var1,var2,var3, 0.)
                                                                                                                                                                                                             ~~~~~~~~~~~~~~~~~~~~~~~~~                       ^~
/cvmfs/sw.hsf.org/spackages6/fccanalyses/0.6.0/x86_64-centos7-gcc11.2.0-opt/qo45d/include/FCCAnalyses/myUtils.h:363:3: note: 'PID' declared here
  ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData> PID(ROOT::VecOps::RVec<edm4hep::ReconstructedParticleData> recop,
  ^

so clearly it’s not happy that I’m trying to use the function with my changes.

Originally to set up my environment I had been doing

source /cvmfs/sw.hsf.org/spackages6/key4hep-stack/2023-01-15/x86_64-centos7-gcc11.2.0-opt/csapx/setup.sh

mkdir build install
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install
make install

cd ..
python3 -m venv FCC_env
source ./FCC_env/bin/activate
cd ./FCCAnalyses/examples/FCCee/flavour/b2snunu/
pip install -r ./envs/venv_requirements.txt

I had a look at what gets done by setup.sh in the master FCCAnalyses so tried to do

source /cvmfs/sw.hsf.org/spackages6/key4hep-stack/2023-01-15/x86_64-centos7-gcc11.2.0-opt/csapx/setup.sh

mkdir build install
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install
make install

export LOCAL_DIR=$pwd

export PYTHONPATH=${LOCAL_DIR}/python:${PYTHONPATH}
export PYTHONPATH=${LOCAL_DIR}/install/python:${PYTHONPATH}
export PATH=${LOCAL_DIR}/bin:${PATH}
export PATH=${LOCAL_DIR}/install/bin:${PATH}
export LD_LIBRARY_PATH=${LOCAL_DIR}/install/lib:${LD_LIBRARY_PATH}
export CMAKE_PREFIX_PATH=${LOCAL_DIR}/install:${CMAKE_PREFIX_PATH}
export ROOT_INCLUDE_PATH=${LOCAL_DIR}/install/include:${ROOT_INCLUDE_PATH}

export ONNXRUNTIME_ROOT_DIR=`python -c "import onnxruntime; print(onnxruntime.__path__[0]+'/../../../..')"`
if [ -z "${ONNXRUNTIME_ROOT_DIR}" ]; then
    echo "----> Warning: ONNX Runtime not found! Related analyzers won't be build..."
else
    export LD_LIBRARY_PATH=${ONNXRUNTIME_ROOT_DIR}/lib:${LD_LIBRARY_PATH}
fi

cd ..
python3 -m venv FCC_env
source ./FCC_env/bin/activate
cd ./FCCAnalyses/examples/FCCee/flavour/b2snunu/
pip install -r ./envs/venv_requirements.txt

But it seems that the export PATH=${LOCAL_DIR}/bin:${PATH} line resets my python version to 3.6.8 so then trying to install Snakemake fails.

Do you have any recommendation for what I should be doing to pick up my local changes to FCCAnalyses without the python version changing?

My current version of FCCAnalyses is at GitHub - AidanWiederhold/FCCAnalyses at aiwieder/reverter

In the meantime going back to my old way of setting things up seems to be working.

Hello @aiwieder,

I’m not sure what is happening in the case of user warnings above, but I tried your alternative way of setting things up, there are few problems:

$pwd -- this variable is empty, do you mean $(pwd)?
venv_requirements.txt -- file does not exists

I believe the reason why the PATH and PYTHONPATH are overwritten to use old system python is that $LOCAL_DIR is empty.

Can you try checking out the latest master from HEP-FCC and cherry pick changes you need on top of it in separate branch?

Also, it seems that the ID function errors are there because your new version of the FCCAnalysis library is not found or build properly.

Best,
Juraj