Amazon Data Firehose

This guide configures logs, metrics. Replace YOUR_INGEST_KEY with your key on your server, never in a public example.

1. Configure an HTTP destination

Firehose places the access key in X-Amz-Firehose-Access-Key, without Bearer. Do not add Authorization or api_key as well. The envelope must use protocol version 1.0, a matching requestId and base64 records. Choose small buffers so encoded envelope bytes stay below 4,000,000; base64 adds roughly one third.

Endpoint URL: https://otlp.siteqwality.com/v1/firehose
Access key: YOUR_INGEST_KEY
Content encoding: GZIP
Retry duration: choose for your delivery requirements
S3 backup: enable failed-data backup

2. Select the source payload

The adapter decodes CloudWatch log events and length-delimited OTLP metric messages. Configure the source transform/output accordingly. The receiver returns the Firehose requestId/timestamp contract only after durable delivery. Any rejected records return a failure response so Firehose can retry or back up the batch.

CloudWatch Logs: preserve the supported gzip/base64 CloudWatch log envelope.
CloudWatch Metric Streams: choose OpenTelemetry 1.0.0, not JSON or OpenTelemetry 0.7.

3. Test delivery

Firehose buffers can add latency. Platform retries may duplicate records across requests. Never disable backup to hide failed delivery.

Use the Firehose destination test, then generate a real CloudWatch event or metric. Inspect DeliveryToHttpEndpoint failures and the S3 error output, and confirm the relevant explorer.

See your data

Open Logs, Traces or Metrics. New data usually shows up within a minute, and a page that says it is waiting for data switches to your data on its own. Nothing arriving? See what each status code means in failure handling.

Production checklist

Store the key in your platform secret mechanism, rotate it when exposed and remove test keys. Set a stable service.name and deployment.environment.name; avoid user IDs or unbounded values in metric labels. Prefer a persistent queue, bounded retries and exporter failure monitoring. Review redaction and processors before increasing traffic.