OpenTelemetry Advanced Settings
Hardware Sentry OpenTelemetry Collector comes with a default configuration file (
config/otel-config-example.yaml) which is intended to work for most situations and only requires minor changes for Hardware Sentry OpenTelemetry Collector to operate properly (refer to the Integration pages for more information).
This page is therefore intended for advanced users who have a deep knowledge of OpenTelemetry and wish to learn more about the properties available in
As a regular OpenTelemetry Collector, Hardware Sentry OpenTelemetry Collector consists of:
- and several extensions.
This version of Hardware Sentry OpenTelemetry Collector leverages version 0.55.0 of OpenTelemetry.
OTLP Receiver is configured by default with the self-signed certificate
security/otel.crt and the private key
security/otel.key to enable the TLS protocol. If you wish to set your own certificate file, configure the Hardware Sentry Agent with the correct Trusted Certificates File. Because the
OTLP Exporter of the Hardware Sentry Agent performs hostname verification, you will also have to add the
localhost entry (
DNS:localhost,IP:127.0.0.1) to the
Subject Alternative Name (SAN) extension of the new generated certificate.
Clients requests are authenticated with the Basic Authenticator extension.
otlp: protocols: grpc: endpoint: localhost:4317 tls: cert_file: security/otel.crt key_file: security/otel.key auth: authenticator: basicauth
OpenTelemetry Collector Internal Exporter for Prometheus
The OpenTelemetry Collector's internal Exporter for Prometheus is an optional source of data. It provides information about the collector activity (see Health Check). It's referred to as
prometheus/internal in the pipeline and leverages the standard
prometheus/internal: config: scrape_configs: - job_name: otel-collector-internal scrape_interval: 60s static_configs: - targets: [ localhost:8888 ]
service:telemetry:metrics section, you can set the metrics
level or the
address of the OpenTelemetry Collector Internal Exporter (by default: localhost:8888).
service: telemetry: metrics: address: localhost:8888 level: basic
By default, the collected metrics go through 5 processors:
metricstransformto enrich the collected metrics, typically with labels required by the observability platforms. The
metricstransformprocessor has many options to add, rename, delete labels and metrics. Note that Hardware Sentry Agent can also be configured to enrich the collected metrics with extra labels.
memory_limiterto limit the memory consumed by the OpenTelemetry Collector process (configurable)
resourcedetectionto find out the actual host name of the system monitored
filterto include or exclude metrics
batchto process data in batches of 10 seconds (configurable).
exporters section defines the destination of the collected metrics. Hardware Sentry OpenTelemetry Collector version 2.0.00 includes support for the below exporters:
- OLTP/gRPC Exporter
- Prometheus Remote Write Exporter
- Prometheus Exporter
- Datadog Exporter
- Logging Exporter
- Splunk SignalFx Exporter
You can configure several exporters in the same instance of the OpenTelemetry Collector to send the collected metrics to multiple platforms.
Use the above links to learn how to configure these exporters. Specific integration scenarios are also described for:
Refer to Check the collector is up and running for more details.
The zpages extension provides debug information about all the different components. It notably provides:
- general information about Hardware Sentry OpenTelemetry Collector
- details about the active pipeline
- activity details of each receiver and exporter configured in the pipeline.
Refer to Check the pipelines status for more details.
Hardware Sentry Agent (
The Hardware Sentry Agent is the internal component which scrapes hosts, collects metrics and pushes OTLP data to the OTLP receiver of the OpenTelemetry Collector. The
hws_agent extension starts the Hardware Sentry Agent as a child process of the OpenTelemetry Collector, checks that this child process is up and running and restarts it if needed.
hws_agent extension as follows:
hws_agent: grpc: <http|https>://<host>:<port> # Default: https://localhost:4317 extra_args: [ <string> ... ] # Example: [ --config=config/alternate-configuration-file.yaml ] restart_delay: <duration> # Default: 10s retries: <int> # Default: -1 (Means no limit)
grpcis the endpoint to which the Hardware Sentry Agent will push OpenTelemetry data. By default, the Hardware Sentry Agent pushes metrics to the local OTLP receiver using gRPC on port TCP/4317 (By default:
extra_argsspecifies a list of additional arguments to be used by the Hardware Sentry Agent. By default, the Hardware Sentry Agent's configuration file is
./config/hws-config.yamlbut you can provide an alternate configuration file by adding a new extra argument. Example:
restart_delayspecifies the period of time after which the Hardware Sentry Agent is restarted when a problem has been detected. If not set, the Hardware Sentry Agent will be restarted after 10 seconds.
retriesspecifies the number of restarts to be triggered until the Hardware Sentry Agent is up and running again. If not set, the extension will try restarting the Hardware Sentry Agent until it is up and running.
Refer to Configure the Hardware Sentry Agent for more details.
Basic Authenticator extension authenticates the
OTLP Exporter requests by comparing the Authorization header sent by the
OTLP Exporter and the credentials provided in the
security/.htpasswd file. Refer to the Apache htpasswd documentation to know how to manage user files for basic authentication.
basicauth: htpasswd: file: security/.htpasswd
.htpasswd file is stored in the
Warning: If a different password is specified in the
.htpasswdfile, update the Basic Authentication Header of the Hardware Sentry Agent.
Configured extensions, receivers, processors and exporters are taken into account if and only if they are declared in the pipeline:
service: telemetry: logs: level: info # Change to debug for more details metrics: address: localhost:8888 level: basic extensions: [health_check, basicauth, hws_agent] pipelines: metrics: receivers: [otlp, prometheus/internal] processors: [memory_limiter, batch, resourcedetection, metricstransform] exporters: [prometheusremotewrite/your-server] # List here the platform of your choice # Uncomment the section below to enable logging of hardware alerts. # logs: # receivers: [otlp] # processors: [memory_limiter, batch, resourcedetection] # exporters: [logging] # List here the platform of your choice