Syslog Integration

fluent-bit.conf

Complete configuration with Docker logs AND syslog support:

# Fluent Bit Configuration for LogTide
# Supports Docker container logs AND syslog from network devices

[SERVICE]
  Flush        5
  Daemon       Off
  Log_Level    info
  Parsers_File /fluent-bit/etc/parsers.conf

# =============================================================================
# INPUT - Docker Container Logs
# =============================================================================
[INPUT]
  Name              tail
  Path              /var/lib/docker/containers/*/*.log
  Parser            docker
  Tag               docker.*
  Refresh_Interval  5
  Mem_Buf_Limit     5MB
  Skip_Long_Lines   On
  Path_Key          filepath

# =============================================================================
# INPUT - Syslog (UDP) - Port 514
# =============================================================================
# Receive syslog messages on UDP (Proxmox, ESXi, firewalls, etc.)
[INPUT]
  Name        syslog
  Parser      syslog-rfc3164
  Listen      0.0.0.0
  Port        514
  Mode        udp
  Tag         syslog.udp

# =============================================================================
# INPUT - Syslog (TCP) - Port 514
# =============================================================================
# TCP is more reliable for important logs
[INPUT]
  Name        syslog
  Parser      syslog-rfc3164
  Listen      0.0.0.0
  Port        514
  Mode        tcp
  Tag         syslog.tcp

# =============================================================================
# FILTER - Docker Logs Processing
# =============================================================================
[FILTER]
  Name                parser
  Match               docker.*
  Key_Name            log
  Parser              docker_json
  Reserve_Data        On
  Preserve_Key        On

[FILTER]
  Name                lua
  Match               docker.*
  script              /fluent-bit/etc/extract_container_id.lua
  call                extract_container_id

[FILTER]
  Name                modify
  Match               docker.*
  Add                 level info
  Rename              log message
  Copy                container_name service

[FILTER]
  Name                record_modifier
  Match               docker.*
  Remove_key          stream
  Remove_key          filepath
  Remove_key          container_name

# =============================================================================
# FILTER - Syslog Processing
# =============================================================================
[FILTER]
  Name                modify
  Match               syslog.*
  Copy                ident service
  Add                 level info

# Map syslog severity to log level
[FILTER]
  Name                lua
  Match               syslog.*
  script              /fluent-bit/etc/map_syslog_level.lua
  call                map_syslog_level

[FILTER]
  Name                record_modifier
  Match               syslog.*
  Remove_key          pri
  Remove_key          ident
  Remove_key          pid

# =============================================================================
# OUTPUT - Send to LogTide API
# =============================================================================
[OUTPUT]
  Name                http
  Match               docker.*
  Host                ${LOGTIDE_API_HOST}
  Port                8080
  URI                 /api/v1/ingest/single
  Format              json_lines
  Header              X-API-Key ${LOGTIDE_API_KEY}
  Header              Content-Type application/json
  Json_date_key       time
  Json_date_format    iso8601
  Retry_Limit         3
  tls                 Off

[OUTPUT]
  Name                http
  Match               syslog.*
  Host                ${LOGTIDE_API_HOST}
  Port                8080
  URI                 /api/v1/ingest/single
  Format              json_lines
  Header              X-API-Key ${LOGTIDE_API_KEY}
  Header              Content-Type application/json
  Json_date_key       time
  Json_date_format    iso8601
  Retry_Limit         3
  tls                 Off

parsers.conf

Parser definitions for Docker and both syslog formats:

# Fluent Bit Parsers Configuration

# =============================================================================
# PARSERS - Docker
# =============================================================================
[PARSER]
  Name        docker_json
  Format      json
  Time_Key    time
  Time_Format %Y-%m-%dT%H:%M:%S.%L%z
  Time_Keep   On

[PARSER]
  Name        docker
  Format      json
  Time_Key    time
  Time_Format %Y-%m-%dT%H:%M:%S.%L%z
  Time_Keep   On

# =============================================================================
# PARSERS - Syslog
# =============================================================================
# RFC 3164 - Traditional syslog format
# Used by: Proxmox, many Linux systems, older network devices
# IMPORTANT: RFC 3164 does NOT include timezone info in timestamps!
# Set Time_Offset to your local timezone offset (e.g., +0100 for CET, +0200 for CEST)
[PARSER]
  Name        syslog-rfc3164
  Format      regex
  Regex       /^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
  Time_Key    time
  Time_Format %b %d %H:%M:%S
  Time_Offset +0100
  Time_Keep   On

# RFC 5424 - Modern syslog format
# Used by: Modern Linux systems, VMware ESXi, some firewalls
[PARSER]
  Name        syslog-rfc5424
  Format      regex
  Regex       /^\<(?<pri>[0-9]{1,5})\>1 (?<time>[^ ]+) (?<host>[^ ]+) (?<ident>[^ ]+) (?<pid>[-0-9]+) (?<msgid>[^ ]+) (?<extradata>(\[.*]|-)) (?<message>.+)$/
  Time_Key    time
  Time_Format %Y-%m-%dT%H:%M:%S.%L%z
  Time_Keep   On

map_syslog_level.lua

Lua script to convert syslog severity (0-7) to log levels. Note: LogTide automatically normalizes syslog levels server-side, so you can send levels like "notice", "emergency", "warning" etc. and they will be mapped to LogTide's 5 levels (debug, info, warn, error, critical).

-- Map syslog priority/severity to log level
-- Syslog severity levels (from RFC 3164/5424):
-- 0 = Emergency, 1 = Alert, 2 = Critical, 3 = Error
-- 4 = Warning, 5 = Notice, 6 = Informational, 7 = Debug
--
-- LogTide automatically maps these to its 5 levels:
-- emergency/alert/critical -> critical
-- error -> error
-- warning -> warn
-- notice/info -> info
-- debug -> debug
--
-- TIMEZONE FIX for RFC 3164:
-- RFC 3164 timestamps don't include timezone info.
-- Set SYSLOG_TZ_OFFSET env var to your timezone offset in hours.
-- Examples: SYSLOG_TZ_OFFSET=1 (CET), SYSLOG_TZ_OFFSET=2 (CEST), SYSLOG_TZ_OFFSET=-5 (EST)

-- Read timezone offset from environment (in hours, e.g., 1 for CET, 2 for CEST)
local tz_offset_hours = tonumber(os.getenv("SYSLOG_TZ_OFFSET")) or 0
local tz_offset_seconds = tz_offset_hours * 3600

function map_syslog_level(tag, timestamp, record)
  local pri = tonumber(record["pri"])

  if pri then
      -- Extract severity from priority (severity = pri % 8)
      local severity = pri % 8

      -- Map severity number to log level string
      -- LogTide will normalize these to its 5 levels
      local level_map = {
          [0] = "emergency",
          [1] = "alert",
          [2] = "critical",
          [3] = "error",
          [4] = "warning",
          [5] = "notice",
          [6] = "info",
          [7] = "debug"
      }

      record["level"] = level_map[severity] or "info"
  else
      record["level"] = "info"
  end

  -- Apply timezone offset for RFC 3164 (which lacks timezone info)
  -- The offset converts local time to UTC
  local adjusted_timestamp = timestamp - tz_offset_seconds

  -- Set 'time' field in ISO8601 format (UTC)
  record["time"] = os.date("!%Y-%m-%dT%H:%M:%SZ", adjusted_timestamp)

  -- Keep hostname from syslog
  if record["host"] and record["host"] ~= "-" then
      record["hostname"] = record["host"]
  end

  -- Use ident (program name) as service if available
  if not record["service"] and record["ident"] and record["ident"] ~= "-" then
      record["service"] = record["ident"]
  elseif not record["service"] and record["hostname"] then
      record["service"] = record["hostname"]
  else
      record["service"] = "syslog"
  end

  -- Clean up fields
  record["pri"] = nil
  record["ident"] = nil
  record["host"] = nil

  return 1, adjusted_timestamp, record
end

extract_container_id.lua

Lua script to extract container ID from Docker log paths:

-- Extract container ID from Docker log filepath
-- Path format: /var/lib/docker/containers/CONTAINER_ID/CONTAINER_ID-json.log

function extract_container_id(tag, timestamp, record)
  local filepath = record["filepath"]

  if filepath then
      local container_id = filepath:match("/var/lib/docker/containers/([^/]+)/", 1)

      if container_id then
          record["container_id"] = container_id
          record["container_short_id"] = container_id:sub(1, 12)
      end
  end

  return 1, timestamp, record
end

Fluent Bit Configuration in docker-compose.yml

Here's the Fluent Bit service configuration (already included in the default compose file):

fluent-bit:
  image: fluent/fluent-bit:4.2.2  # For ARM64: cr.fluentbit.io/fluent/fluent-bit:4.2.2
  container_name: logtide-fluent-bit
  profiles:
    - logging  # Only starts with --profile logging or COMPOSE_PROFILES=logging
  ports:
    - "514:514/udp"  # Syslog UDP
    - "514:514/tcp"  # Syslog TCP
  volumes:
    # Configuration files
    - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
    - ./parsers.conf:/fluent-bit/etc/parsers.conf:ro
    # Lua scripts
    - ./extract_container_id.lua:/fluent-bit/etc/extract_container_id.lua:ro
    - ./map_syslog_level.lua:/fluent-bit/etc/map_syslog_level.lua:ro
    - ./wrap_logs.lua:/fluent-bit/etc/wrap_logs.lua:ro
    # Docker logs access
    - /var/lib/docker/containers:/var/lib/docker/containers:ro
    - /var/run/docker.sock:/var/run/docker.sock:ro
  environment:
    LOGTIDE_API_KEY: ${FLUENT_BIT_API_KEY}
    LOGTIDE_API_HOST: backend
    SYSLOG_TZ_OFFSET: ${SYSLOG_TZ_OFFSET:-0}  # Timezone offset for RFC3164
  depends_on:
    - backend
  restart: unless-stopped

1. Install rsyslog (if not installed)

apt install rsyslog -y

2. Create rsyslog configuration

Create /etc/rsyslog.d/50-logtide.conf:

# Replace with your LogTide server IP
*.* @@10.0.0.100:514

Use @@ for TCP (recommended) or @ for UDP.

3. Restart rsyslog

systemctl restart rsyslog
systemctl status rsyslog

4. Test the connection

logger -t proxmox-test "Test log from Proxmox to LogTide"

Check LogTide - you should see the test message with service "proxmox-test".

Via Web UI (vSphere Client)

1. Navigate to Host → Manage → System → Advanced Settings
2. Find Syslog.global.logHost
3. Set value to: udp://YOUR_LOGTIDE_IP:514 or tcp://YOUR_LOGTIDE_IP:514
4. Click Save

Via SSH/CLI

# Enable SSH on ESXi first, then:
esxcli system syslog config set --loghost='tcp://10.0.0.100:514'
esxcli system syslog reload

UniFi (UDM/USG)

1. Go to Settings → System → Remote Logging
2. Enable Remote Syslog Server
3. Enter your LogTide server IP and port 514
4. Select log level (Info recommended)

pfSense / OPNsense

1. Go to Status → System Logs → Settings
2. Check Enable Remote Logging
3. Enter your LogTide server IP in Remote log servers
4. Select which logs to send (firewall, system, etc.)

Linux Servers (rsyslog)

Same as Proxmox - create /etc/rsyslog.d/50-logtide.conf:

# Forward all logs to LogTide (TCP)
*.* @@YOUR_LOGTIDE_IP:514

# Or forward only specific facilities/severities:
# auth,authpriv.* @@YOUR_LOGTIDE_IP:514    # Auth logs only
# *.err @@YOUR_LOGTIDE_IP:514              # Errors and above only

Synology NAS

1. Go to Control Panel → Log Center → Log Sending
2. Check Send logs to a syslog server
3. Enter your LogTide server IP
4. Select format: BSD (RFC 3164) or IETF (RFC 5424)

Separating Logs by Server

By default, syslog logs use the ident field (program name) as the service name, with hostname as fallback. You have two options to separate servers:

-- Change the service assignment logic in map_syslog_level.lua:
if record["host"] and record["host"] ~= "-" then
    record["service"] = record["host"]  -- Use hostname as service
elseif record["ident"] and record["ident"] ~= "-" then
    record["service"] = record["ident"]
else
    record["service"] = "syslog"
end

Per-Server Custom Parsing

Some devices (like Unraid) may not follow standard RFC3164 format exactly. You can route logs from different servers through different parsers.

# fluent-bit.conf - Unraid on port 5514
[INPUT]
    Name        syslog
    Parser      syslog-unraid    # Custom parser
    Listen      0.0.0.0
    Port        5514
    Mode        udp
    Tag         syslog.unraid

# Proxmox on port 5515 (standard format)
[INPUT]
    Name        syslog
    Parser      syslog-rfc3164
    Listen      0.0.0.0
    Port        5515
    Mode        udp
    Tag         syslog.proxmox

# Rewrite tag based on hostname pattern
[FILTER]
    Name          rewrite_tag
    Match         syslog.*
    Rule          $host ^unraid.* syslog.unraid false
    Rule          $host ^proxmox.* syslog.proxmox false
    Rule          $host ^pfsense.* syslog.firewall false

# Apply Unraid-specific processing
[FILTER]
    Name          lua
    Match         syslog.unraid
    script        /fluent-bit/etc/parse_unraid.lua
    call          parse_unraid

# Apply standard processing to Proxmox
[FILTER]
    Name          lua
    Match         syslog.proxmox
    script        /fluent-bit/etc/map_syslog_level.lua
    call          map_syslog_level

# Custom parser for Unraid syslog format
[PARSER]
    Name        syslog-unraid
    Format      regex
    Regex       ^(?<time>[A-Z][a-z]{2} [ ]?[0-9]{1,2} [0-9:]+) (?<host>[^ ]+) (?<ident>[^:]+): (?<message>.*)$
    Time_Key    time
    Time_Format %b %d %H:%M:%S
    Time_Keep   On

Option 1: Use Fluent Bit (Recommended)

The simplest solution is to use Fluent Bit as described in this guide. Fluent Bit receives syslog messages on port 514 and forwards them to LogTide's HTTP API. This is the recommended approach.

Option 2: Vector as HTTP Bridge

Vector is a high-performance observability data pipeline that can act as a syslog-to-HTTP bridge:

# Install Vector (Debian/Ubuntu)
curl --proto '=https' --tlsv1.2 -sSf https://sh.vector.dev | bash

# Or via apt
curl -1sLf 'https://repositories.timber.io/public/vector/cfg/setup/bash.deb.sh' | sudo bash
sudo apt install vector

Create /etc/vector/vector.toml:

# Vector configuration for rsyslog → LogTide

[sources.rsyslog]
type = "syslog"
address = "0.0.0.0:514"
mode = "tcp"  # or "udp"

[transforms.format_logs]
type = "remap"
inputs = ["rsyslog"]
source = '''
# Transform syslog fields to LogTide format
.message = string!(.message)
.level = downcase(string(.severity) ?? "info")
.service = string(.appname) ?? string(.hostname) ?? "syslog"
.time = now()

# Add metadata
.metadata = {}
.metadata.hostname = string(.hostname) ?? null
.metadata.facility = string(.facility) ?? null
.metadata.procid = string(.procid) ?? null

# Remove original syslog fields
del(.severity)
del(.appname)
del(.hostname)
del(.facility)
del(.procid)
'''

[sinks.logtide]
type = "http"
inputs = ["format_logs"]
uri = "http://YOUR_LOGTIDE_HOST:8080/api/v1/ingest/single"
encoding.codec = "json"

[sinks.logtide.request]
headers.X-API-Key = "lp_your_api_key_here"
headers.Content-Type = "application/json"

# Start Vector
sudo systemctl enable vector
sudo systemctl start vector

# Check status
sudo systemctl status vector
vector --version

Option 3: Build rsyslog with omhttp

If you need native rsyslog HTTP support, you can compile rsyslog from source with the omhttp module enabled. This requires development libraries and is more complex to maintain:

# Install build dependencies (Debian/Ubuntu)
sudo apt install build-essential pkg-config libestr-dev libfastjson-dev zlib1g-dev libcurl4-openssl-dev uuid-dev libgcrypt20-dev

# Download and compile rsyslog
wget https://www.rsyslog.com/files/download/rsyslog/rsyslog-8.2312.0.tar.gz
tar xzf rsyslog-8.2312.0.tar.gz
cd rsyslog-8.2312.0

./configure --enable-omhttp
make
sudo make install

Note: Building from source means you're responsible for security updates and compatibility. The Fluent Bit or Vector approaches are generally preferred.

Option 4: Use omfwd + Local HTTP Forwarder

Forward syslog to a local Fluent Bit or Vector instance using standard rsyslog forwarding (omfwd), then let that service handle HTTP:

# /etc/rsyslog.d/50-logtide.conf
# Forward all logs to local Fluent Bit syslog input
*.* @@127.0.0.1:5514

This keeps your rsyslog configuration simple and delegates HTTP complexity to a dedicated tool.

Syslog messages not appearing?

1. Verify Fluent Bit is running: docker compose ps fluent-bit
2. Check Fluent Bit logs: docker compose logs fluent-bit
3. Test connectivity from source: nc -zv YOUR_LOGTIDE_IP 514
4. Check firewall allows port 514 (TCP/UDP)
5. Verify API key is set in .env: FLUENT_BIT_API_KEY=lp_...

Test syslog manually

Send a test syslog message using netcat:

# UDP test
echo "<14>Test syslog message from terminal" | nc -u -w1 YOUR_LOGTIDE_IP 514

# TCP test
echo "<14>Test syslog message from terminal" | nc -w1 YOUR_LOGTIDE_IP 514

Service name shows as "syslog"?

The service name comes from the syslog "ident" field (program name). If your device doesn't send an ident, the hostname will be used. You can customize the Lua script to set service names based on hostname patterns.

RFC 3164 timestamps are wrong (off by hours)?

RFC 3164 syslog format does NOT include timezone information in timestamps. If your logs show incorrect times, set the SYSLOG_TZ_OFFSET environment variable on your Fluent Bit container.

Quick fix: Add this to your docker-compose.yml:

services:
fluent-bit:
  environment:
    - SYSLOG_TZ_OFFSET=1    # CET (Central European Time)
    # - SYSLOG_TZ_OFFSET=2  # CEST (Summer time)
    # - SYSLOG_TZ_OFFSET=-5 # EST (Eastern Standard Time)

Alternative: You can also set Time_Offset directly in the parser configuration (parsers.conf):

[PARSER]
  Name        syslog-rfc3164
  ...
  Time_Format %b %d %H:%M:%S
  Time_Offset +0100    # For CET
  Time_Keep   On

Note: RFC 5424 includes timezone in the timestamp (e.g., 2025-12-23T16:51:02+01:00) and doesn't need this fix. Consider configuring your devices to use RFC 5424 format if possible.