Fallback for failed exports

[adriangb]

Is there any way to configure the collector to write somewhere "safe" if the downstream service that I'm exporting to becomes unavailable? E.g. retry an export 5 times then write it to disk or object storage.

Or is there even a way I could write an exporter or processor to handle this?

[ms-jcorley]

This might be the place to start: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md

If your needs go beyond what that provides, you can definitely write your own processor or exporter to do these kinds of tasks.

[tailg8nj]

Here is a use case:

A pipeline with a kafka exporter. The kafka pipe has a max message size and a single Trace object, serialized and compressed, might be too big. If it fails to write to kafka, a secondary export, say to s3 would ensure even large objects can get passed along.

I am exploring functionality of trying to create an exporter that delegates to n others in order. However, it raises some questions:

• will I be able to define the exporters at the top level exporters yaml section or will the collector complain / not start them thinking that they are not used directly in a pipeline
• how will I reference the delegates in the overarching exporter, and how will I control their lifecycle. In other words, can I use Service.Settings.ExporterFactories? and then can I call them directly, and subsequent Start/Stop methods?