Skip to content

Handle CAF making when Pandora runs after the NuGraph2 filter #616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rtriozzi wants to merge 13 commits into
base: develop
Choose a base branch
from
+264 −83
Open

Handle CAF making when Pandora runs after the NuGraph2 filter #616

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
351d379
Updated CAFMaker to work with NuGraph inference on multiple slices.
leonardo-lena Oct 6, 2025
762e6ed
Fixed memory leak issues.
leonardo-lena Oct 14, 2025
78a0667
FEATURE: implemented CAFMaker compatibility with PandoraAfterNuGraph.
leonardo-lena Oct 23, 2025
d5d32a0
fill nugraph hip tagger
rtriozzi Dec 24, 2025
cf283cb
change to int to allow for negative default
rtriozzi Jan 6, 2026
98a4654
more shower vars
rtriozzi Jan 7, 2026
612ae6a
move HIP tagging params to fcl
rtriozzi Jan 12, 2026
55812bc
move more params to fcl, address Giuseppe's comments
rtriozzi Jan 12, 2026
d483222
removed spare comments
rtriozzi Jan 12, 2026
deb9088
use NuGraphCategory enum and add more vars
rtriozzi Jan 16, 2026
6aa4a4c
added guard for tpcID
rtriozzi Mar 16, 2026
074258e
add post-NG2-filter filtered hits count
rtriozzi Jun 4, 2026
0800060
correct typo
rtriozzi Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions sbncode/CAFMaker/CAFMakerParams.h
View file
Open in desktop
Comment thread
rtriozzi marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -350,10 +350,10 @@ namespace caf
"" //Empty by default, configured in icaruscode cafmaker_defs
};

Atom<art::InputTag> NuGraphSliceHitLabel {
Name("NuGraphSliceHitLabel"),
Comment("Label of NuGraph slice hit map."),
"" //Empty by default, please set to e.g. art::InputTag("nuslhits")
Atom<art::InputTag> NuGraphSlicesLabel {
Name("NuGraphSlicesLabel"),
Comment("Label of slices that have NuGraph inference."),
"" //Empty by default, please set to e.g. art::InputTag("NCCSlices")
};

Atom<art::InputTag> NuGraphFilterLabel {
Expand All @@ -368,6 +368,12 @@ namespace caf
"" //Empty by default, please set to e.g. art::InputTag("NuGraph","semantic")
};

Atom<bool> UsePandoraAfterNuGraph {
Name("UsePandoraAfterNuGraph"),
Comment("Whether to use the second pass Pandora outputs for NuGraph reco."),
false
};

Atom<string> OpFlashLabel {
Name("OpFlashLabel"),
Comment("Label of PMT flash."),
Expand Down
91 changes: 68 additions & 23 deletions sbncode/CAFMaker/CAFMaker_module.cc
View file
Open in desktop
Comment thread
rtriozzi marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -1717,6 +1717,7 @@ void CAFMaker::produce(art::Event& evt) noexcept {

// collect the TPC slices
std::vector<art::Ptr<recob::Slice>> slices;
std::vector<art::Ptr<recob::Slice>> nuGraphSlices;
std::vector<std::string> slice_tag_suffixes;
std::vector<unsigned> slice_tag_indices;
for (unsigned i_tag = 0; i_tag < pandora_tag_suffixes.size(); i_tag++) {
Expand All @@ -1726,24 +1727,17 @@ void CAFMaker::produce(art::Event& evt) noexcept {
GetByLabelStrict(evt, fParams.PFParticleLabel() + pandora_tag_suffix, thisSlices);
if (thisSlices.isValid()) {
art::fill_ptr_vector(slices, thisSlices);
nuGraphSlices = evt.getProduct<std::vector<art::Ptr<recob::Slice>>>(fParams.NuGraphSlicesLabel().label() + pandora_tag_suffix);
if (fParams.UsePandoraAfterNuGraph()) {
nuGraphSlices = slices;
}

@PetrilloAtWork PetrilloAtWork Jul 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this logic.
If fParams.UsePandoraAfterNuGraph() is true, nugraphSlices will be the same as slices (so that case could easily live outside the loop); in the other case, nuGraphSlices is overwritten with the content of each tag until the last one, hence holding only slices from the data product with the last tag.
I suspect what was meant to add slices, something like

      if (!fParams.UsePandoraAfterNuGraph()) {
        auto const& moreSlices = evt.getProduct<std::vector<art::Ptr<recob::Slice>>>(fParams.NuGraphSlicesLabel().label() + pandora_tag_suffix);
        nuGraphSlices.insert(nuGraphSlices.end(), moreSlices.begin(), moreSlices.end());
      }

for (unsigned i = 0; i < thisSlices->size(); i++) {
slice_tag_suffixes.push_back(pandora_tag_suffix);
slice_tag_indices.push_back(i_tag);
}
}
}

// nu graph
std::vector< art::Handle<std::vector<unsigned int>> > ng2_slice_hit_map_handle(pandora_tag_suffixes.size());
std::vector< art::Handle<std::vector<anab::FeatureVector<1>>> > ng2_filter_handle(pandora_tag_suffixes.size());
std::vector< art::Handle<std::vector<anab::FeatureVector<5>>> > ng2_semantic_handle(pandora_tag_suffixes.size());
for (unsigned i_tag = 0; i_tag < pandora_tag_suffixes.size(); i_tag++) {
const std::string &pandora_tag_suffix = pandora_tag_suffixes[i_tag];
GetByLabelIfExists(evt, fParams.NuGraphSliceHitLabel().encode() + pandora_tag_suffix, ng2_slice_hit_map_handle[i_tag]);
GetByLabelIfExists(evt, fParams.NuGraphFilterLabel().label() + pandora_tag_suffix + ":" + fParams.NuGraphFilterLabel().instance(), ng2_filter_handle[i_tag]);
GetByLabelIfExists(evt, fParams.NuGraphSemanticLabel().label() + pandora_tag_suffix + ":" + fParams.NuGraphSemanticLabel().instance(), ng2_semantic_handle[i_tag]);
}

// The Standard Record
// Branch entry definition -- contains list of slices, CRT information, and truth information
StandardRecord rec;
Expand Down Expand Up @@ -1799,17 +1793,38 @@ void CAFMaker::produce(art::Event& evt) noexcept {
fmatch_assn_map.emplace(std::make_pair(fname_opdet, sfm_assn));
}
}

if (std::find(nuGraphSlices.begin(), nuGraphSlices.end(), slice) != nuGraphSlices.end()) {
std::vector<art::Ptr<anab::FeatureVector<1>>> ng2_filter_vec;
std::vector<art::Ptr<anab::FeatureVector<5>>> ng2_semantic_vec;
art::FindOneP<anab::FeatureVector<1>> findOneFilter(slcHits, evt, fParams.NuGraphFilterLabel().label() + slice_tag_suff + ":" + fParams.NuGraphFilterLabel().instance());
art::FindOneP<anab::FeatureVector<5>> findOneSemantic(slcHits, evt, fParams.NuGraphSemanticLabel().label() + slice_tag_suff + ":" + fParams.NuGraphSemanticLabel().instance());
// iteration is the only way to get something out of a FindOne or FindMany...
if (findOneFilter.isValid()) {
for (size_t hitIdx = 0; hitIdx < slcHits.size(); ++hitIdx) {
if (findOneFilter.at(hitIdx).isNull()) {
slcHits.erase(slcHits.begin()+hitIdx);
hitIdx--;
continue;
}
ng2_filter_vec.emplace_back(findOneFilter.at(hitIdx));
}
}

std::vector<art::Ptr<anab::FeatureVector<1>>> ng2_filter_vec;
std::vector<art::Ptr<anab::FeatureVector<5>>> ng2_semantic_vec;
if (ng2_filter_handle[producer].isValid()) {
art::fill_ptr_vector(ng2_filter_vec,ng2_filter_handle[producer]);
}
if (ng2_semantic_handle[producer].isValid()) {
art::fill_ptr_vector(ng2_semantic_vec,ng2_semantic_handle[producer]);
}
if (ng2_slice_hit_map_handle[producer].isValid()) {
FillSliceNuGraph(slcHits,*ng2_slice_hit_map_handle[producer],ng2_filter_vec,ng2_semantic_vec,recslc);
if (findOneSemantic.isValid()) {
for (size_t hitIdx = 0; hitIdx < slcHits.size(); ++hitIdx) {
if (findOneSemantic.at(hitIdx).isNull()) {
slcHits.erase(slcHits.begin()+hitIdx);
hitIdx--;
continue;
}
ng2_semantic_vec.emplace_back(findOneSemantic.at(hitIdx));
}
}

if (ng2_filter_vec.size() > 0 || ng2_semantic_vec.size() > 0) {
FillSliceNuGraph(slcHits, ng2_filter_vec, ng2_semantic_vec, recslc);
}
}

art::FindManyP<sbn::OpT0Finder> fmOpT0 =
Expand Down Expand Up @@ -2189,8 +2204,38 @@ void CAFMaker::produce(art::Event& evt) noexcept {
FillCNNScores(thisParticle, cnnScores, pfp);
}

if (ng2_slice_hit_map_handle[producer].isValid()) {
FillPFPNuGraph(*ng2_slice_hit_map_handle[producer], ng2_filter_vec, ng2_semantic_vec, fmPFPartHits.at(iPart), pfp);
if (std::find(nuGraphSlices.begin(), nuGraphSlices.end(), slice) != nuGraphSlices.end()) {
std::vector<art::Ptr<recob::Hit>>& PFPHits = fmPFPartHits.at(iPart);
art::FindOneP<anab::FeatureVector<1>> findOneFilter(PFPHits, evt, fParams.NuGraphFilterLabel().label() + slice_tag_suff + ":" + fParams.NuGraphFilterLabel().instance());
art::FindOneP<anab::FeatureVector<5>> findOneSemantic(PFPHits, evt, fParams.NuGraphSemanticLabel().label() + slice_tag_suff + ":" + fParams.NuGraphSemanticLabel().instance());
std::vector<art::Ptr<anab::FeatureVector<1>>> ng2_filter_vec;
std::vector<art::Ptr<anab::FeatureVector<5>>> ng2_semantic_vec;

if (findOneFilter.isValid()) {
for (size_t hitIdx = 0; hitIdx < PFPHits.size(); ++hitIdx) {
if (findOneFilter.at(hitIdx).isNull()) {
PFPHits.erase(PFPHits.begin() + hitIdx);
hitIdx--;
continue;
}
ng2_filter_vec.emplace_back(findOneFilter.at(hitIdx));
}
}

if (findOneSemantic.isValid()) {
for (size_t hitIdx = 0; hitIdx < PFPHits.size(); ++hitIdx) {
if (findOneSemantic.at(hitIdx).isNull()) {
PFPHits.erase(PFPHits.begin() + hitIdx);
hitIdx--;
continue;
}
ng2_semantic_vec.emplace_back(findOneSemantic.at(hitIdx));
}
}
Comment on lines +2238 to +2258

@PetrilloAtWork PetrilloAtWork Jul 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels quite familiar... same comments as for twin code above.
Also, consider moving into a member function

std::tuple<
  std::vector<art::Ptr<anab::FeatureVector<1>>>,
  std::vector<art::Ptr<anab::FeatureVector<5>>>
  >
CAFMaker::readNuGraphHitFeatures(
  std::vector<art::Ptr<recob::Hit>> const& hits,
  art::Event const& event, std::string const& suffix
) const;


if (ng2_filter_vec.size() > 0 || ng2_semantic_vec.size() > 0) {
FillPFPNuGraph(PFPHits, ng2_filter_vec, ng2_semantic_vec, pfp);
}
}

if (!thisTrack.empty()) { // it has a track!
Expand Down
44 changes: 16 additions & 28 deletions sbncode/CAFMaker/FillReco.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -581,16 +581,16 @@ namespace caf
}

void FillSliceNuGraph(const std::vector<art::Ptr<recob::Hit>> &inputHits,
const std::vector<unsigned int> &sliceHitsMap,
const std::vector<art::Ptr<anab::FeatureVector<1>>> &ngFilterResult,
const std::vector<art::Ptr<anab::FeatureVector<5>>> &ngSemanticResult,
caf::SRSlice &slice)
{

//need to double check that the slice processed by NuGraph is the same under consideration
//std::cout << "sizes=" << inputHits.size() << " " << sliceHitsMap.size() << " " << ngFilterResult.size() << " " << ngSemanticResult.size() << std::endl;
// std::cout << "sizes=" << inputHits.size() << " " << " " << ngFilterResult.size() << " " << ngSemanticResult.size() << std::endl;
unsigned int nHits = inputHits.size();
if (nHits==0 || nHits!=sliceHitsMap.size() || inputHits[0].key()!=sliceHitsMap[0]) return;//not the same slice!
// implementing this same-slice check with the updated architecture would be a major headache and I believe is not required given how inputHits is retrieved.
// if (nHits==0 || nHits!=sliceHitsMap.size() || inputHits[0].key()!=sliceHitsMap[0]) return;//not the same slice!

unsigned int npass = 0;
for ( unsigned int i = 0; i < nHits; i++ ) {
Expand Down Expand Up @@ -1024,38 +1024,26 @@ namespace caf
srpfp.cnnscore.nclusters = cnnscore->nClusters;
}

void FillPFPNuGraph(const std::vector<unsigned int> &sliceHitsMap,
void FillPFPNuGraph(const std::vector<art::Ptr<recob::Hit>> &pfpHits,
const std::vector<art::Ptr<anab::FeatureVector<1>>> &ngFilterResult,
const std::vector<art::Ptr<anab::FeatureVector<5>>> &ngSemanticResult,
const std::vector<art::Ptr<recob::Hit>> &pfpHits,
caf::SRPFP& srpfp,
bool allowEmpty)
{

// the nugraph elements are ordered the same as the sliceHitsMap
std::vector<size_t> mappedhits;
for (auto& hit : pfpHits) {
auto it = std::find(sliceHitsMap.begin(), sliceHitsMap.end(), hit.key());
if (it != sliceHitsMap.end()) {
size_t index = std::distance(sliceHitsMap.begin(), it);
mappedhits.push_back(index);
}
}

if (mappedhits.size()>0) {
if (pfpHits.size()>0) {
std::vector<float> ng2sempfpcounts(5,0);
size_t ng2bkgpfpcount = 0;
for (size_t pos : mappedhits) {
auto const& bkgscore = ngFilterResult.at(pos);
if (bkgscore->at(0)<ng_filter_cut) {
ng2bkgpfpcount++;
} else {
auto const& scores = ngSemanticResult.at(pos);
std::vector<float> ng2semscores;
for (size_t i=0;i<scores->size();i++) ng2semscores.push_back(scores->at(i));
size_t sem_label = std::distance(ng2semscores.begin(), std::max_element(ng2semscores.begin(), ng2semscores.end()));//arg_max(ng2semscores);
ng2sempfpcounts[sem_label]++;
}
for (size_t pos = 0; pos < pfpHits.size(); pos++) {
auto const& bkgscore = ngFilterResult.at(pos);
if (bkgscore->at(0)<ng_filter_cut) {
ng2bkgpfpcount++;
} else {
auto const& scores = ngSemanticResult.at(pos);
std::vector<float> ng2semscores;
for (size_t i=0;i<scores->size();i++) ng2semscores.push_back(scores->at(i));
Comment on lines +1131 to +1132

@PetrilloAtWork PetrilloAtWork Jul 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::vector<float> ng2semscores;
for (size_t i=0;i<scores->size();i++) ng2semscores.push_back(scores->at(i));
std::vector<float> ng2semscores{ scores.begin(), scores.end() };

size_t sem_label = std::distance(ng2semscores.begin(), std::max_element(ng2semscores.begin(), ng2semscores.end()));//arg_max(ng2semscores);
ng2sempfpcounts[sem_label]++;
}
}
srpfp.ngscore.sem_cat = SRNuGraphScore::NuGraphCategory(std::distance(ng2sempfpcounts.begin(), std::max_element(ng2sempfpcounts.begin(), ng2sempfpcounts.end())));//arg_max(ng2sempfpcounts);
size_t nonBkgHits = (pfpHits.size() > ng2bkgpfpcount ? pfpHits.size()-ng2bkgpfpcount : 0);
Expand Down
4 changes: 1 addition & 3 deletions sbncode/CAFMaker/FillReco.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,6 @@ namespace caf
* Hits with filter value (`ngFilterResult`) lower than `ng_filter_cut` are counted as background.
*/
void FillSliceNuGraph(const std::vector<art::Ptr<recob::Hit>> &inputHits,
const std::vector<unsigned int> &sliceHitsMap,
const std::vector<art::Ptr<anab::FeatureVector<1>>> &ngFilterResult,
const std::vector<art::Ptr<anab::FeatureVector<5>>> &ngSemanticResult,
caf::SRSlice &slice);
Expand Down Expand Up @@ -158,10 +157,9 @@ namespace caf
*
* Hits with filter value (`ngFilterResult`) lower than `ng_filter_cut` are counted as background.
*/
void FillPFPNuGraph(const std::vector<unsigned int> &sliceHitsMap,
void FillPFPNuGraph(const std::vector<art::Ptr<recob::Hit>> &pfpHits,
const std::vector<art::Ptr<anab::FeatureVector<1>>> &ngFilterResult,
const std::vector<art::Ptr<anab::FeatureVector<5>>> &ngSemanticResult,
const std::vector<art::Ptr<recob::Hit>> &pfpHits,
caf::SRPFP& srpfp,
bool allowEmpty= false);

Expand Down