
[[ga-telemetryd]]
= Telemetry Daemon
:description: Learn about the telemetry daemon in {page-component-title} to support applications that use different protocols to transfer metrics data.

The telemetry daemon (telemetryd) provides an extensible framework that you can use to handle sensor data pushed to {page-component-title}.
You can use the framework to support applications that use different protocols to transfer metrics data.

With telemetryd, operators can define listeners that support different protocols to receive the telemetry data, and adapters that transfer the received data into generic formats like flows or performance data.

.Generic overview of components in telemetryd
image::telemetryd/telemetryd-overview.png["Network overview diagram showing relationships among telemetryd components"]

The configuration is split in two parts: on one side, listeners and their attached parsers or connectors receive telemetry data that is transported over a specific protocol, and parse the data according to the protocol's specification.
On the other side, adapters pick up the parsed data and enrich it before persisting it.

Queues transport the parsed telemetry data from parsers to adapters by binding a specific parser to a specific adapter.

== Listeners and parsers

Listeners receive sensor data from an external source and hand the data off to parsers, which handle the transport protocol format and extract the transported data.
Listeners do the transport protocol handling, like opening sockets and accepting incoming connections.
The received messages are then passed to the parsers that are configured to the listeners.
They parse the concrete protocol implementation and extract all available information from the messages.
The extracted data is then handed over to the configured queue for further processing.

For example, a listener may open a UDP socket to receive packets, and an assigned parser parses the incoming IPFIX packets.

Parsers and listeners must work closely together; for that reason, they cannot be combined randomly.
The supported parsers are described in the documentation for each listener.
For more information on listeners, their supported parsers, and how to configure them, see xref:reference:telemetryd/listeners/introduction.adoc[Telemetryd listeners reference].

== Connectors

Connectors initiate connections and maintain sessions with agents.
A new connector is created for every agent that it is responsible for connecting to.
Here, "agent" refers to a named IP service attached to an IP interface on a specific node.

Connectors do not use parsers; they dispatch messages directly to queues.

.Telemetryd connectors
image::telemetryd/telemetryd-connectors.png["Network overview diagram showing relationships among telemetryd connectors"]

== Adapters

Adapters enrich and process data that is dispatched over a queue.
They receive the data from the queue and transform it into a format that can be used by {page-component-title}.
They can also enrich incoming data to attach node or interface information, depending on the information that is already available.
For example, an adapter may be used to transform telemetry data that is received via sFlow into performance data and persist it to the applicable node.

Since adapters must handle data dependent on a transport protocol, not all of them can be used in combination with every possible parser.
The supported parsers are described in the documentation for each adapter.

NOTE: If you have multiple adapters, the order in which they are run is the same as defined in `telemetryd-configuration.xml`.

=== Working with Minions

Listeners and parsers run on Minions, but adapters run on Sentinel.
Your main {page-component-title} instance can run listeners, parsers, and adapters.

When using a Minion as a listener, you must use adapters on your {page-component-title} instance or a Sentinel instance to transform the data.

== Queues

Queues transfer data between parsers and adapters, and are represented by a channel in the messaging system.

=== Configuring queues

The following options can help fine-tune queue behavior:

[cols="2,2,3,2"]
|===
| Queue Attribute ({page-component-title})  | Key (Minion/Sentinel)     | Description   | Default Value

| threads
| queue.threads
| Number of threads used for consuming and dispatching messages.
| (2 * _number of cores_)

| queue-size
| queue.size
| Maximum number of messages to keep in memory while waiting to be dispatched.
| 10000

| use-routing-key
| queue.use-routing-key
| Whether to use the routing key when forwarding messages to the broker.
This enforces message ordering.
| true

| batch-size
| batch.size
| Maximum batch size.
When the batch reaches this size, it is dispatched.
| 1000

| batch-interval
| batch.interval
| Maximum batch interval, in milliseconds.
When the batch has been created for longer than this interval, it is dispatched, regardless of the current size.
| 500
|===

TIP: When using Kafka as a message broker, setting `use-routing-key` to `false` balances messages across all partitions.
This can be done safely for flows, but is not supported for metrics when using thresholding (order is required).

In {page-component-title}, you can add these options to the `<queue>` element as attributes:

[source, xml]
----
<queue name="IPFIX" use-routing-key="false">
    ...
</queue>
----

On a Minion, you can add these options as parser properties, and as adapter properties on Sentinel:

[source, properties]
----
name=IPFIX-Listener
class-name=org.opennms.netmgt.telemetry.listeners.UdpListener
parameters.host=0.0.0.0
parameters.port=4738
parsers.0.name=IPFIX
parsers.0.class-name=org.opennms.netmgt.telemetry.protocols.netflow.parser.IpfixUdpParser
parsers.0.queue.use-routing-key=false
----

== Push sensor data through Minion

Listeners and their parsers may run either on {page-component-title} or a Minion, whereas adapters may run either on {page-component-title} or Sentinel.
If a listener and its parsers are running on a Minion, the received messages are dispatched automatically via a queue to associated adapters running in {page-component-title} or Sentinel.
Minions can receive data, but require {page-component-title} or Sentinel to process it.

.Running listener on a Minion forwarding packets using the messaging system
image::telemetryd/telemetryd-minion.png["Network overview diagram showing a listener running on a Minion, and the listener's related components"]

== Reverse hostname resolution

Some parsers support reverse hostname resolution to annotate IP addresses using the actual hostname.
The Minion performs the reverse lookup while it parses the protocol data.
The resolved hostname, if found, is attached to the address information.
Both are then sent to the adapter for further processing.
For more information see xref:deep-dive/dnsresolver/introduction.adoc[].

The following parsers currently support reverse hostname resolution:

* Netflow v5
* Netflow v9
* IPFIX
* sFlow
