MINIFICPP-2840 - Refactor provenance part 1#2195
Conversation
There was a problem hiding this comment.
Pull request overview
This PR is the first part of a provenance refactor, focusing on modernizing the provenance/flowfile APIs (naming and types) and simplifying repository record retrieval to return a vector of deserialized components.
Changes:
- Renames
getlineageStartDate()togetLineageStartDate()across FlowFile/Provenance APIs and call sites. - Refactors
core::Repository::getElementsfrom an out-parameter/bool API tostd::vector<...> getElements(size_t max_size). - Updates provenance event/reporting plumbing (component id now uses processor UUID; JSON report builder returns a
std::string).
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| minifi-api/include/minifi-cpp/provenance/Provenance.h | API rename for lineage start date; removes parent/child “remove” APIs. |
| minifi-api/include/minifi-cpp/core/Repository.h | Changes getElements API to return a vector. |
| minifi-api/include/minifi-cpp/core/FlowFile.h | Renames getlineageStartDate to getLineageStartDate in the API. |
| libminifi/test/libtest/unit/ProvenanceTestHelper.h | Updates test repository to new getElements signature and provenance record creation. |
| libminifi/src/provenance/Provenance.cpp | Refactors ProvenanceEventRecordImpl fields/ctor usage; updates serialization/deserialization and reporter commit loop. |
| libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp | Refactors JSON report generator to return string; adapts to new getElements API. |
| libminifi/src/core/ProcessSession.cpp | Passes processor UUID as provenance component id; updates lineage start date call. |
| libminifi/src/core/FlowFile.cpp | Renames FlowFileImpl method to getLineageStartDate. |
| libminifi/src/core/flow/FlowSchema.cpp | Adds NiFi flow JSON keys for provenance reporting section/fields. |
| libminifi/include/provenance/Provenance.h | Refactors provenance impl types/ctors and member naming; removes default ctor. |
| libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h | Updates getJsonReport signature (returns string); removes onSchedule override declaration. |
| libminifi/include/core/FlowFile.h | Renames FlowFileImpl virtual override to getLineageStartDate. |
| extensions/standard-processors/tests/unit/ProcessorTests.cpp | Updates tests for provenance record creation and new reporting task API. |
| extensions/standard-processors/processors/LogAttribute.cpp | Updates lineage start date getter name. |
| extensions/rocksdb-repos/tests/ProvenanceTests.cpp | Updates record loading creation to use ProvenanceEventRecord::create(); removes child UUID removal assertions. |
| extensions/rocksdb-repos/ProvenanceRepository.h | Updates provenance repository interface to new getElements signature. |
| extensions/rocksdb-repos/ProvenanceRepository.cpp | Implements new getElements signature returning a vector. |
| extensions/grafana-loki/PushGrafanaLokiREST.cpp | Updates lineage start date getter name. |
| extensions/grafana-loki/PushGrafanaLokiGrpc.cpp | Updates lineage start date getter name. |
| core-framework/include/core/Repository.h | Updates default repository implementation to new getElements signature. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
c1f65b2 to
e4d93ad
Compare
|
|
||
| std::shared_ptr<ProvenanceEventRecord> ProvenanceEventRecord::create() { | ||
| return std::make_shared<ProvenanceEventRecordImpl>(); | ||
| return std::make_shared<ProvenanceEventRecordImpl>(ProvenanceEventType::CLONE, utils::Identifier{}, ""); |
There was a problem hiding this comment.
why do we default to a clone event now? clone only happens when multiple connections stem from the same relationship
There was a problem hiding this comment.
this create is only used to generate an "empty" instance before reading into it from the repository, I removed the default ctor, why I picked CLONE I don't remember, should we go for CREATE or add a new value UNKNOWN?
There was a problem hiding this comment.
it's fine, it just wasn't obvious when reading the code
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.