Fluentd is an open source data collector that supports different formats, protocols, and customizable plugins for reading and writing log streams.

  • If you have data in Fluentd, we recommend using the Unomaly plugin to forward that data directly to a Unomaly instance for analysis.
  • As a fallback option for data ingestion, Unomaly also runs with Fluentd pre-installed on the instance. This topic helps you to configure Fluentd on the instance to receive data from other Fluentd, Docker, or syslog.

The Unomaly plugin for Fluentd

Send fluent records to the Unomaly ingestion API with the Unomaly plugin for Fluentd: https://github.com/unomaly/fluent-plugin-unomaly

Sending data to Fluentd on Unomaly

1. Fluentd listens on port 24224, which is the default fluentd forwarder port. This means that the port needs to be accessible through the firewall. See "Edit network and communication settings".

2. The main Fluentd configuration file on your Unomaly instance is located at /DATA/fluentd/etc/fluent.conf.

Do not make any changes to this file, because it will be overwritten on upgrades. Instead, add your configurations to /DATA/fluentd/etc/conf.d as separate files.

Fluentd configuration options
Directive Description
source Decides which interface fluentd should use to read data.
filter Allows you to define the custom filters that modify event streams.
match Tells fluentd what to do.
label Used to define different sources.

Refer to Fluentd's documentation for more descriptions of their configuration options: https://docs.fluentd.org/v0.12/articles/config-file.

3. After you make any changes to the configuration file, you need to restart Fluentd:

unomaly restart fluentd

You can view the stdout/log of Fluentd by running:

unomaly logs fluentd

Or, you can run the following for tail-mode viewing:

unomaly logs fluentd -f

Receive data from other Fluentd or Docker

The following configuration example receives data forwarded from other Fluentd installations or Docker.

@type forward
port 24224
<filter **.docker.**>
@type record_transformer
hostname "${tag_parts[2]}"
<match **.docker.**>
@type unomaly
flush_interval 1s
source_key hostname
message_key log
accept_self_signed_certs true

Specifically in this case, the following statements are declared:


  • @type forward means that this plugin is mainly used to receive event logs from other Fluentd instances, the fluent-cat command, or client libraries. This is by far the most efficient way to retrieve the records.
  • port 24224 designates the port on which fluentd should listen for data.
  • bind means that it will listen to any network interface.


  • filter **.docker.** means that messages are transformed.
  • In this case we re-write the hostname in the <record>-block.
  • tag_parts[2] means to against the second index of the tag in the sender’s message.


  • match **.docker.** means that messages from docker matched.
  • @type unomaly means that fluentd will look for a file called out_unomaly.rb in /DATA/fluentd/plugins and pass log data into it. The out_unomaly.rb plugin will ingest data into Unomaly.

Receive standard syslog data

The following configuration example receives standard syslog data and ingests them into Unomaly.

@type syslog
@label @mystream
port 51400
tag system
<label @mystream>
<match system.**>
@type unomaly
flush_interval 1s
source_key hostname
message_key log
accept_self_signed_certs true

You may add the following source section to switch protocols to TCP syslogs (by default, UDP is used):

protocol_type tcp

If you have problems receiving syslog messages, it may be the format doesn't match. You can change the format in the source section using message_format:

message_format auto
message_format rfc3164
message_format rfc5424

Installing Fluentd plugins

You can install Fluentd plugins by using unomaly_fluent_gem:

unomaly restart fluentd

For a list of available commands run:

unomaly logs fluentd

After installing new plugins you need to restart the fluentd service to be able to use them in a configuration:

unomaly restart fluentd

You can write your own plugins or find existing ones for Fluentd and save them into /DATA/fluentd/plugins:

  • Make sure that they are registers with the name you use in the fluentd.conf
  • Make sure that the source-code registers them in the same name
  • Make sure that the file-name matches and has the correct type (such as out_ for output type plugins)


To see whether data comes into fluentd at all, you can use for example:

<match **>
@type stdout

This will print the message on the stdout of the running fluentd process.

You can use for instance fluent-cat (a fluentd tool) or simply logger (a standard Linux syslog tool) to produce log message input for fluentd.

1. Example of using fluent-cat:

echo '{"message":"hello world"}' | fluent-cat --host --port 24224 --format json hello

2. Example of using logger:

logger --server --port 51400 '<6>Jan 13 12:34:56 sunworld myapp[31802]: 
[info] test logline in rfc3164'