Skip to content

Fluentd

Fluentd is a log collector, parser, transformer, and router for log pipelines. It is often used to move logs from applications and nodes into a central storage backend.

What It Does

  • Collects logs from files, sockets, containers, and services.
  • Parses and normalizes log records.
  • Adds metadata such as host, namespace, or application name.
  • Routes records to one or more destinations.

Pipeline Model

Fluentd is built around a simple flow:

  1. Input sources read records.
  2. Filters enrich or transform them.
  3. Match blocks send them to outputs.

That model makes it useful for pipelines where the same log stream needs to go to multiple systems.

Common Use Cases

  • Shipping container logs from Kubernetes nodes.
  • Transforming unstructured logs before they are indexed.
  • Forwarding application logs into Loki, Elasticsearch, S3, or a SIEM.

Example Configuration

<source>
  @type tail
  path /var/log/app.log
  pos_file /var/log/fluentd-app.pos
  tag app.log
  <parse>
    @type none
  </parse>
</source>

<filter app.**>
  @type record_transformer
  <record>
    environment production
    service my-app
  </record>
</filter>

<match app.**>
  @type stdout
</match>

Practical Notes

  • Keep parsing rules close to the source format.
  • Add consistent metadata before shipping logs downstream.
  • Avoid turning Fluentd into a logic layer; keep transformations simple.