Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substructure to Shared Library #63

Merged
merged 62 commits into from
May 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
e6796ce
add checkup for fjcontrib
jackaraz Apr 2, 2022
b2f87cb
modify library structure
jackaraz Apr 2, 2022
9cc1658
create shared library for substructure module
jackaraz Apr 2, 2022
9809346
bugfix in execution without fastjet
jackaraz Apr 2, 2022
6d55de3
slight modification in coding style of main.py
jackaraz Apr 2, 2022
c054fd2
add substructure library to normal mode workflow
jackaraz Apr 2, 2022
735fedc
bugfix in RecJetFormat.h
jackaraz Apr 2, 2022
d426641
improve the code structure
jackaraz Apr 3, 2022
03c7dac
move fastjet into shared library
jackaraz Apr 3, 2022
38e2357
move filter into the shared library
jackaraz Apr 3, 2022
8cf6a65
add Pruner.cpp
jackaraz Apr 3, 2022
0e9c061
update the structure of the class: ClusterBase.h
jackaraz Apr 4, 2022
1ae418e
improve Nsubjettiness.h
jackaraz Apr 5, 2022
7cfbc9a
add HEPTopTagger to SampleAnalyzer backend
jackaraz Apr 5, 2022
8a3476e
integration of HTT for compilation libraries
jackaraz Apr 5, 2022
ecae28d
HTT installation and detection interfaces
jackaraz Apr 5, 2022
74cbb28
add fastjet flag
jackaraz Apr 5, 2022
118c378
add HTT to AnalyzerBase.h
jackaraz Apr 5, 2022
ad9903e
move Recluster.hh to the source file
jackaraz Apr 6, 2022
d2f4b2a
update install_heptoptagger.py
jackaraz Apr 6, 2022
8536884
update install_heptoptagger.py
jackaraz Apr 6, 2022
821bf6d
simplify Filter.h
jackaraz Apr 6, 2022
37ad5e7
add empty jet (thanks to @econte-cms)
jackaraz Apr 6, 2022
e09c803
move softdrop to source
jackaraz Apr 6, 2022
7b35f4f
add examples for substructure module
jackaraz Apr 7, 2022
d8a0dae
add table for available normal mode jet_algo definitions
jackaraz Apr 7, 2022
41ee4f4
add installation instructions
jackaraz Apr 7, 2022
62ed504
fix syntax in virtual functions
jackaraz Apr 10, 2022
d7f7995
fix memory leaks and doc
jackaraz Apr 11, 2022
8312fa6
remove pseudojet dependency from JetClusterer.cpp
jackaraz Apr 12, 2022
d01fab3
protection against delphes
jackaraz Apr 12, 2022
575a8f3
implement `exclusive_subjets` accessor to RecJetFormat.h
jackaraz Apr 13, 2022
66bb550
bugfix in malloc for Pruner.cpp and Filter.cpp
jackaraz Apr 19, 2022
3d8da12
bugfix in malloc for SoftDrop.cpp
jackaraz Apr 19, 2022
b54d4a7
add failsafe for `exclusive_subjets`
jackaraz Apr 19, 2022
6b2a4cf
remove unnecessary class declarations in ClusterBase.h
jackaraz Apr 19, 2022
e528f70
fix library name
jackaraz Apr 20, 2022
4efc86d
bugfix in HTT.h and HTT.cpp
jackaraz Apr 20, 2022
5824c69
add test module for HTT
jackaraz Apr 20, 2022
65eaaf6
check compiler version
jackaraz Apr 20, 2022
bbe0afb
raise proper error with necessary information
jackaraz Apr 20, 2022
a943159
optimise codeflow
jackaraz Apr 21, 2022
46d4c4b
optimise codeflow: HTT.h
jackaraz Apr 21, 2022
ebead65
update FastJet and fjcontrib
jackaraz Apr 21, 2022
3a1cbbd
modify behaviour incase fjcontrib is not available
jackaraz Apr 21, 2022
dfbd209
import fjcontrib based interface if fjcontrib is available
jackaraz Apr 21, 2022
f8eb75a
bugfix: fjconfig control requires new flag
jackaraz Apr 21, 2022
2c43e59
move variableR headers to source file
jackaraz Apr 22, 2022
0e95a55
compile fjcontrib as position independent code
jackaraz Apr 22, 2022
6beb13f
Merge branch 'substructure' into sharedlib
jackaraz Apr 26, 2022
f9118f6
removing useless comments in install_fastjet.py
BFuks Apr 27, 2022
4bbb4cc
removing useless comments in install_fastjetcontrib.py
BFuks Apr 27, 2022
7f26ef7
polishing the heptoptagger installation routines + downgrading clang …
BFuks May 5, 2022
84fcb30
polishing install_manager.py
BFuks May 5, 2022
49feb65
polishing detect_fastjet.py
BFuks May 5, 2022
9632393
polishing detect_heptoptagger.py
BFuks May 5, 2022
061200d
polishing RecJetFormat.cpp
BFuks May 17, 2022
2277188
Polishing HEPTopTagger
BFuks May 17, 2022
a534e6d
Removing a useless line in ClusterBase.h
BFuks May 17, 2022
27c5b5d
Polishing the substructure readme
BFuks May 17, 2022
6b3123b
Fixing the substructure readme (markdown)
BFuks May 17, 2022
fbe64c8
Fixing the substructure readme (markdown)
BFuks May 17, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions madanalysis/IOinterface/job_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -708,16 +708,20 @@ def WriteMakefiles(self, option="", **kwargs):
options.has_process = True
# @JACK enable usage of fastjet
# If there are any root files, fastjet clusterer should not run with MA5_FASTJET_MODE Flag
options.ma5_fastjet_mode = (
all([all([('root' not in x) for x in dataset.filenames]) for dataset in self.main.datasets])
and self.main.archi_info.has_fastjet
and kwargs.get('ma5_fastjet_mode',True)
and self.main.fastsim.package == 'fastjet'
)
options.ma5_fastjet_mode = self.main.archi_info.has_fastjet and self.main.archi_info.has_fjcontrib
options.has_fastjet_lib = self.main.archi_info.has_fastjet
options.has_fastjet_ma5lib = self.main.archi_info.has_fastjet
# @JACK: fastjet_inc is required to be able to use the FJ files when FJ mode is on
options.has_fastjet_inc = self.main.archi_info.has_fastjet
# @Jack: add substructure library
options.has_substructure = self.main.archi_info.has_fastjet and self.main.archi_info.has_fjcontrib
# @Jack: add HTT library
options.has_heptoptagger = (
self.main.archi_info.has_fastjet and \
self.main.archi_info.has_fjcontrib and \
self.main.archi_info.has_heptoptagger
)

options.has_root_inc = self.main.archi_info.has_root
options.has_root_lib = self.main.archi_info.has_root
#options.has_userpackage = True
Expand Down
76 changes: 63 additions & 13 deletions madanalysis/IOinterface/library_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,10 @@


from __future__ import absolute_import
from madanalysis.selection.instance_name import InstanceName
from madanalysis.IOinterface.folder_writer import FolderWriter
from madanalysis.IOinterface.job_writer import JobWriter
from string_tools import StringTools
from shell_command import ShellCommand
import logging
import shutil
import os
import subprocess
from six.moves import input


Expand Down Expand Up @@ -126,6 +121,8 @@ def WriteMakefileForInterfaces(self,package):
filename = self.path+"/SampleAnalyzer/Test/Makefile_zlib"
elif package=='test_fastjet':
filename = self.path+"/SampleAnalyzer/Test/Makefile_fastjet"
elif package=='test_htt':
filename = self.path+"/SampleAnalyzer/Test/Makefile_htt"
elif package=='test_delphes':
filename = self.path+"/SampleAnalyzer/Test/Makefile_delphes"
elif package=='test_delphesMA5tune':
Expand All @@ -149,6 +146,8 @@ def WriteMakefileForInterfaces(self,package):
title='*zlib-interface* test'
elif package=='test_fastjet':
title='*fastjet-interface* test'
elif package=='test_htt':
title='*htt-interface* test'
elif package=='test_delphes':
title='*delphes-interface* test'
elif package=='test_delphesMA5tune':
Expand All @@ -173,6 +172,49 @@ def WriteMakefileForInterfaces(self,package):
options.has_fastjet_inc=True
# options.has_fastjet_lib=True
toRemove.extend(['compilation_fastjet.log','linking_fastjet.log','cleanup_fastjet.log','mrproper_fastjet.log','../Bin/TestFastjet.log'])
elif package == "substructure":
options.has_commons=True
options.has_fastjet_inc=True
options.has_fastjet_lib=True
# @JACK: To be able to use fastjet in Ma5 data structure
options.ma5_fastjet_mode=True
options.has_fjcontrib = True
options.has_nsubjettiness = True
toRemove.extend(
['compilation_substructure.log',
'linking_substructure.log',
'cleanup_substructure.log',
'mrproper_substructure.log']
)
elif package == "HEPTopTagger":
options.has_commons=True
options.has_fastjet_inc=True
options.has_fastjet_lib=True
# @JACK: To be able to use fastjet in Ma5 data structure
options.ma5_fastjet_mode=True
options.has_nsubjettiness = True
options.has_substructure = True
toRemove.extend(
['compilation_heptoptagger.log',
'linking_heptoptagger.log',
'cleanup_heptoptagger.log',
'mrproper_heptoptagger.log']
)
elif package == "test_htt":
options.has_commons=True
options.has_fastjet_inc=True
# options.has_fastjet_lib=True
# @JACK: To be able to use fastjet in Ma5 data structure
options.ma5_fastjet_mode=True
options.has_fastjet_ma5lib = True
options.has_substructure = True
options.has_heptoptagger = True
toRemove.extend(
['compilation_test_heptoptagger.log',
'linking_test_heptoptagger.log',
'cleanup_test_heptoptagger.log',
'mrproper_test_heptoptagger.log']
)
elif package=='configuration':
toRemove.extend(['compilation.log','linking.log','cleanup.log','mrproper.log'])
elif package=='commons':
Expand Down Expand Up @@ -264,6 +306,7 @@ def WriteMakefileForInterfaces(self,package):
options.has_fastjet_inc = self.main.archi_info.has_fastjet
options.has_fastjet_lib = self.main.archi_info.has_fastjet
options.ma5_fastjet_mode = self.main.archi_info.has_fastjet
options.has_substructure = self.main.archi_info.has_fjcontrib and self.main.archi_info.has_fastjet

toRemove.extend(['compilation.log','linking.log','cleanup.log','mrproper.log'])
elif package=='test_process':
Expand Down Expand Up @@ -299,6 +342,9 @@ def WriteMakefileForInterfaces(self,package):
elif package=='test_fastjet':
cppfiles = ['Fastjet/*.cpp']
hfiles = ['Fastjet/*.h']
elif package=='test_htt':
cppfiles = ['HEPTopTagger/*.cpp']
hfiles = ['HEPTopTagger/*.h']
elif package=='test_delphes':
cppfiles = ['Delphes/*.cpp']
hfiles = ['Delphes/*.h']
Expand Down Expand Up @@ -333,6 +379,10 @@ def WriteMakefileForInterfaces(self,package):
isLibrary=False
ProductName='TestFastjet'
ProductPath='../Bin/'
elif package=='test_htt':
isLibrary=False
ProductName='TestHEPTopTagger'
ProductPath='../Bin/'
elif package=='test_root':
isLibrary=False
ProductName='TestRoot'
Expand Down Expand Up @@ -366,15 +416,15 @@ def Compile(self,ncores,package,folder):
# log file name
if package in ['process','commons','test','configuration']:
logfile = folder+'/compilation.log'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet', "test_htt", 'test_root','test_delphes','test_delphesMA5tune']:
logfile = folder+'/compilation_'+package[5:]+'.log'
else:
logfile = folder+'/compilation_'+package+'.log'

# makefile
if package in ['process','commons','test','configuration']:
makefile = 'Makefile'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet', "test_htt", 'test_root','test_delphes','test_delphesMA5tune']:
makefile = 'Makefile_'+package[5:]
else:
makefile = 'Makefile_'+package
Expand All @@ -401,15 +451,15 @@ def Link(self,package,folder):
# log file name
if package in ['process','commons','test','configuration']:
logfile = folder+'/linking.log'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet',"test_htt",'test_root','test_delphes','test_delphesMA5tune']:
logfile = folder+'/linking_'+package[5:]+'.log'
else:
logfile = folder+'/linking_'+package+'.log'

# makefile
if package in ['process','commons','test','configuration']:
makefile = 'Makefile'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet',"test_htt",'test_root','test_delphes','test_delphesMA5tune']:
makefile = 'Makefile_'+package[5:]
else:
makefile = 'Makefile_'+package
Expand All @@ -433,15 +483,15 @@ def Clean(self,package,folder):
# log file name
if package in ['process','commons','configuration','test']:
logfile = folder+'/cleanup.log'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet',"test_htt",'test_root','test_delphes','test_delphesMA5tune']:
logfile = folder+'/cleanup_'+package[5:]+'.log'
else:
logfile = folder+'/cleanup_'+package+'.log'

# makefile
if package in ['process','commons','test','configuration']:
makefile = 'Makefile'
elif package in ['test_process','test_commons','test_zlib','test_fastjet','test_root','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_fastjet',"test_htt",'test_root','test_delphes','test_delphesMA5tune']:
makefile = 'Makefile_'+package[5:]
else:
makefile = 'Makefile_'+package
Expand All @@ -465,7 +515,7 @@ def MrProper(self,package,folder):
# log file name
if package in ['process','commons','configuration']:
logfile = folder+'/mrproper.log'
elif package in ['test_process','test_commons','test_zlib','test_root','test_fastjet','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_root','test_fastjet',"test_htt",'test_delphes','test_delphesMA5tune']:
logfile = folder+'/mrproper_'+package[5:]+'.log'
else:
logfile = folder+'/mrproper_'+package+'.log'
Expand All @@ -475,7 +525,7 @@ def MrProper(self,package,folder):
# makefile
if package in ['process','commons','test','configuration']:
makefile = 'Makefile'
elif package in ['test_process','test_commons','test_zlib','test_root','test_fastjet','test_delphes','test_delphesMA5tune']:
elif package in ['test_process','test_commons','test_zlib','test_root','test_fastjet',"test_htt",'test_delphes','test_delphesMA5tune']:
makefile = 'Makefile_'+package[5:]
else:
makefile = 'Makefile_'+package
Expand Down
32 changes: 24 additions & 8 deletions madanalysis/build/makefile_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ def __init__(self):
self.has_fastjet_tag = False
# @JACK: cant use FASTJET_USE tag, it clashes with ROOT
self.ma5_fastjet_mode = False
self.has_fjcontrib = False
self.has_substructure = False
self.has_heptoptagger = False
self.has_nsubjettiness = False

self.has_delphes_tag = False
self.has_delphesMA5tune_tag = False
self.has_root_tag = False
Expand Down Expand Up @@ -363,7 +368,6 @@ def Makefile(
if options.has_zlib_ma5lib:
libs.extend(['-lzlib_for_ma5'])
if options.has_zlib_lib:
# libs.extend(['-L'+archi_info.zlib_lib_path,'-lz'])
libs.extend(['-lz'])
file.write('LIBFLAGS += '+' '.join(libs)+'\n')

Expand All @@ -386,22 +390,25 @@ def Makefile(
if options.has_fastjet_ma5lib:
libs.extend(['-lfastjet_for_ma5'])
if options.has_fastjet_lib:
# libs.extend(['$(shell fastjet-config --libs --plugins)']) # --rpath=no)'])
libs.extend(['$(shell $(MA5_BASE)/tools/SampleAnalyzer/ExternalSymLink/Bin/fastjet-config --libs --plugins)'])
file.write('LIBFLAGS += '+' '.join(libs)+'\n')
libs=[]
if options.has_fjcontrib:
# Add fjcontrib libraries
libs.extend(["-lRecursiveTools"]) # SoftDrop
libs.extend(["-lNsubjettiness"]) # Nsubjettiness
libs.extend(["-lVariableR"]) # VariableR
libs.extend(["-lEnergyCorrelator"]) # -lEnergyCorrelator
file.write('LIBFLAGS += '+' '.join(libs)+'\n')
if options.has_nsubjettiness:
libs.extend(["-lNsubjettiness"]) # Nsubjettiness
if options.has_nsubjettiness or options.has_fjcontrib:
file.write('LIBFLAGS += '+' '.join(libs)+'\n')

# - delphes
if options.has_delphes_ma5lib or options.has_delphes_lib:
libs=[]
if options.has_delphes_ma5lib:
libs.extend(['-ldelphes_for_ma5'])
if options.has_delphes_lib:
# libs.extend(['-L'+archi_info.delphes_lib_paths[0],'-lDelphes'])
libs.extend(['-lDelphes'])
file.write('LIBFLAGS += '+' '.join(libs)+'\n')

Expand All @@ -411,10 +418,17 @@ def Makefile(
if options.has_delphesMA5tune_ma5lib:
libs.extend(['-ldelphesMA5tune_for_ma5'])
if options.has_delphesMA5tune_lib:
# libs.extend(['-L'+archi_info.delphesMA5tune_lib_paths[0],'-lDelphesMA5tune'])
libs.extend(['-lDelphesMA5tune'])
file.write('LIBFLAGS += '+' '.join(libs)+'\n')

# Substructure module
if options.has_substructure:
file.write('LIBFLAGS += -lsubstructure_for_ma5\n')

# HEPTopTagger module
if options.has_heptoptagger:
file.write('LIBFLAGS += -lHEPTopTagger_for_ma5\n')

# - Commons
if options.has_commons:
libs=[]
Expand All @@ -423,8 +437,6 @@ def Makefile(

# - Root
libs=[]
#libs.extend(['-L'+archi_info.root_lib_path, \
# '-lCore -lCint -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lTree -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -pthread -lm -ldl -rdynamic -lEG'])
# becareful: to not forget -lEG
if options.has_root:
libs.extend(['$(shell $(MA5_BASE)/tools/SampleAnalyzer/ExternalSymLink/Bin/root-config --libs)','-lEG'])
Expand All @@ -448,6 +460,10 @@ def Makefile(
libs.append('$(MA5_BASE)/tools/SampleAnalyzer/Lib/libdelphesMA5tune_for_ma5.so')
if options.has_fastjet_ma5lib:
libs.append('$(MA5_BASE)/tools/SampleAnalyzer/Lib/libfastjet_for_ma5.so')
if options.has_substructure:
libs.append('$(MA5_BASE)/tools/SampleAnalyzer/Lib/libsubstructure_for_ma5.so')
if options.has_heptoptagger:
libs.append('$(MA5_BASE)/tools/SampleAnalyzer/Lib/libHEPTopTagger_for_ma5.so')
if len(libs)!=0:
file.write('# Requirements to check before building\n')
for ind in range(0,len(libs)):
Expand Down
Loading