feat(kafka): add MessageBroker/Kafka/Cluster/{id}/Topic/{topic} metrics#1747
Draft
shashank-reddy-nr wants to merge 4 commits into
Draft
feat(kafka): add MessageBroker/Kafka/Cluster/{id}/Topic/{topic} metrics#1747shashank-reddy-nr wants to merge 4 commits into
shashank-reddy-nr wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1747 +/- ##
==========================================
- Coverage 82.08% 81.96% -0.12%
==========================================
Files 215 215
Lines 26309 26792 +483
Branches 4150 4260 +110
==========================================
+ Hits 21596 21961 +365
- Misses 3301 3352 +51
- Partials 1412 1479 +67 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mergify
Bot
added
tests-failing
Records per-cluster Kafka metrics (MessageBroker/Kafka/Cluster/{clusterId}/Topic/{topic}/Produce
and MessageBroker/Kafka/Cluster/{clusterId}/Topic/{topic}/Consume) to let customers track
throughput broken out by Kafka cluster, not just by topic. The cluster ID is fetched
once per unique broker set via a background AdminClient call; the hot produce/consume path
has no additional overhead. The feature is always-on and best-effort — it does not inject
anything into Kafka wire headers and does not add span or custom attributes.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
shashank-reddy-nr
force-pushed
the
feat/kafka-cluster-id
branch
from
767faa5 to
455ec7b
Compare
…aches Both hooks now store (cluster_id, time.monotonic()) tuples and trigger a background re-fetch on expiry, returning the stale value in the meantime. Also handles describe_cluster() returning a dict (newer kafka-python-ng). Assisted-by: Claude Sonnet 4.6
…unit Ruff E702 (multiple statements on one line) and I001 (unsorted imports) flagged the inline `import time; time.sleep(...)` patterns. Moving the import to the module-level import block is the correct fix. Assisted-by: Claude Sonnet 4.6
The hook stores cluster IDs as (str, float) tuples for stale-while-revalidate TTL tracking. Three test files seeded _kafka_cluster_id_cache with plain strings, so isinstance(_cached, tuple) was False and metrics were never recorded — causing test_cluster_produce_metric, test_cluster_consume_metric, and test_cluster_id_written_to_cache_on_success to fail. - test_producer.py / test_consumer.py: seed with (id, time.monotonic()) tuple; add missing import time - test_cluster_metrics_unit.py: same for key-preservation and resolved-entry fixtures; fix assertion to check cached[0] instead of comparing full tuple to a bare string Assisted-by: Claude Sonnet 4.6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds per-cluster, per-topic throughput metrics using the Kafka cluster UUID.
New metrics:
NRQL
SELECT sum(newrelic.timeslice.value) FROM Metric
WHERE metricTimesliceName LIKE 'MessageBroker/Kafka/Cluster%'
FACET appName, metricTimesliceName SINCE 30 minutes ago
Before contributing, please read our contributing guidelines and code of conduct.
Overview
Describe the changes present in the pull request
Related Github Issue
Include a link to the related GitHub issue, if applicable
Testing
The agent includes a suite of tests which should be used to
verify your changes don't break existing functionality. These tests will run with
Github Actions when a pull request is made. More details on running the tests locally can be found in our
testing guidelines,
For most contributions it is strongly recommended to add additional tests which
exercise your changes.