1.0 Beta: Tenant Isolation, Metering & Capabilities, Hooks & Webhook Delivery
First beta of the 1.0 line. Tenant data isolation audit, per-org metering with a capability/quota system, typed lifecycle hooks, a hardened outbound webhook dispatcher, a structured audit log primitive, and fail-closed PII masking.
- Tenant data isolation audit across every backend data-access path, with a CI tripwire and isolation test suite (#228, #219)
- Metering + capability system: per-org usage measurement, feature gates, and enforceable limits/quotas (#212, #214)
- Typed lifecycle hooks for ingestion, query, alert evaluation and webhook dispatch (#216)
- Outbound webhook delivery: HMAC signing, retry with backoff, dead-letter queue and SSRF protection (#218)
- Audit log primitive: typed actions, actor types, outcomes and per-org retention (#217)
- PII masking is now fail-closed at ingestion — records that fail masking are rejected, never stored unmasked
First beta of the 1.0 line. The headline work since 0.9.7 is the tenant data isolation audit that hardens every backend data-access path before the stable cut, and the metering + capability system pair that gives every organization usage measurement, feature gates and enforceable limits/quotas (ingestion, spans, storage) — the foundation for plan tiers without changing OSS behavior. A typed lifecycle hooks surface lands alongside it, letting operators observe, mutate or reject ingestion, query, alert-evaluation and webhook-dispatch without forking. A reusable outbound webhook delivery system also lands — HMAC signing, retry with backoff, a dead-letter queue and centralized SSRF protection — onto which every existing webhook sender is migrated, closing three unguarded fetch paths in the process.
Security
- PII masking is now fail-closed at ingestion: previously, if the masking step threw, the batch was stored unmasked with only a
console.warn.maskLogBatchnow reports per-record failures and the ingestion service rejects exactly those records before the reservoir write (the whole batch if rule compilation itself fails). Rejected records are reported back to the client in a new optionalrejected: [{ index, reason: 'pii_masking_failed' }]field on the ingest response (omitted when empty, so existing clients are unaffected) and aspartialSuccess.rejectedLogRecordson the OTLP logs endpoint. No unmasked data can reach any storage engine throughingestLogs - Cross-tenant log reads via unvalidated
?projectIdon the logs query API (#228, audit #219): all 10 endpoints in the query module accepted a?projectId=parameter and queried with it directly, without verifying it belonged to the authenticated API key’s bound project. Fixed with a sharedresolveQueryProjectIdguard that returns403when the requested projectId does not match the key’s project, locked by a dedicated isolation test - Cross-tenant trace/span reads via the same pattern on the traces query API (#228): the identical flaw existed on 8 trace/span read endpoints, fixed with the same guard and locked by an isolation test. The metrics query API was already safe
- Application-layer scoping gaps on tenant-table queries (defense in depth, #228): a sweep of every tenant-table access path closed several queries missing an
organization_id/project_idfilter (PII-masking, SIEM, exceptions, correlation). Session-auth routes now Zod-validateorganizationIdas a uuid instead of a raw cast - Dependency security updates ahead of the 1.0 cut:
vitest/@vitest/coverage-v8to3.2.6, and transitiveesbuild(≥0.28.1) andshell-quote(≥1.8.4) pins bumped via the root pnpmoverrides. No vulnerable version remains in the lockfile
Added
- Capability system: per-organization feature gates, static limits and usage quotas (#214): a typed capability registry is the single source of truth for 12 initial capabilities across three kinds — boolean gates (
auth.sso,detection.advanced,audit.enabled,isolation.dedicated), static numeric limits (alerts.max_rules,notifications.max_channels,apikeys.max,audit.retention_days) and metered usage quotas (ingestion.max_bytes_monthly,ingestion.max_events_monthly,storage.max_bytes,tracing.max_spans_monthly). All defaults are OSS-permissive (booleans enabled, limits/quotas unlimited), so self-hosted behavior is unchanged until an operator sets a cap. Per-org overrides live in a neworganization_entitlementstable; a cache-backed resolver merges rows over registry defaults, fails open on DB errors, and is swappable so a hosted distribution can source entitlements from subscription state without patching core. Enforcement:assertCapability(403),assertWithinLimit(403) andassertWithinUsageQuota(429, hard-blocks log and OTLP span ingestion). Read APIGET /api/v1/capabilitiesreturns the merged set for UI gating; admin endpoints manage per-org overrides, surfaced in an “Entitlements” card on the admin organization page - Resource usage metering per organization and project (#212): a storage-agnostic way to record and aggregate consumption, surfaced as a “Usage” section in the dashboard. A new
metering_eventsTimescaleDB hypertable is fed by a non-blocking, loss-tolerant in-process recorder that buffersmetering.record(...)calls and batch-inserts on a size threshold, flush interval, and graceful shutdown — dropping under back-pressure rather than blocking the request path. Log ingestion recordslogs.ingested.bytesandlogs.ingested.eventsfire-and-forget after the reservoir write (OTLP ingestion is metered too). Read APIsGET /api/v1/usageandGET /api/v1/usage/breakdownanswer “what is being ingested” by type, project, service and log level. Tunable viaMETERING_ENABLED,METERING_FLUSH_INTERVAL_MSandMETERING_FLUSH_MAX_BUFFER - Span and storage metering recording sites (#212 follow-up): span ingestion now records
spans.ingested, activating thetracing.max_spans_monthlyquota end-to-end. A dailyStorageSnapshotJobrecords a per-(organization, project)storage.snapshotestimating stored bytes as the logical bytes ingested within the org’s retention window — engine-agnostic by design — which activatesstorage.max_bytes. Usage pages gain a “Current storage (estimated)” stat and a daily storage trend - Capability usage vs plan limits on the usage page: a “Plan limits” section shows current consumption against each configured capability limit as a color-coded progress bar (green < 80%, amber 80–99%, red ≥ 100%), backed by a new
GET /api/v1/usage/capabilitiesendpoint that joins live usage to the configured cap for every measurable capability. A null limit renders as “Unlimited”, so OSS defaults read correctly - Audit log primitive: typed actions, actor types, outcomes and per-org retention (#217): the audit log grows from a free-string event sink into a structured security primitive. A canonical action registry (70 actions across 13 families like
org.*,apikey.*,rule.*,auth.*,data.*) is the single source of truth — actions are a TypeScript string-literal union, so typos fail to compile. A newauditLogService.record()API reads organization, actor (user/apiKey/system), IP and user agent from the request context, so callsites only state the action and target; it supports atomic recording inside a transaction, awaited fire-and-safe inserts that never fail the request, and a flush buffer for high-volume access logging. All 57 legacy callsites were converted; new coverage includes failed local logins (auth.login_failed, previously not recorded at all). Per-orgaudit_retention_days(1–3650, NULL = keep forever) is editable from the admin org page and enforced by the daily retention job. The audit page gains actor-type, outcome and time-range filters - Lifecycle hooks at ingestion, query, alert evaluation and webhook dispatch (#216): a small, typed set of named extension points (
beforeIngest,beforeQuery,beforeAlertEvaluation,beforeWebhookDispatch), no-op in OSS (nothing registered by default, ahasHandlersguard keeps the hot paths at zero overhead). Handlers run sequentially, receive a typed per-phase context with documented mutable fields, and may abort by throwingHookRejectionError(code, message, statusCode); unexpected hook errors fail closed. Operators register handlers without forking viaHOOKS_MODULES. Contract documented in the new Lifecycle Hooks guide - after- lifecycle hook phases* (#216 follow-up):
afterIngest(batch counts and rejection reasons),afterAlertTriggered(post-persist trigger facts) andafterWebhookDispatch(per delivery-attempt result). Fire-and-forget: handler errors are logged and never block or mutate the operation; contexts are frozen read-only snapshots - Generic outbound webhook delivery infrastructure (#218): a reusable dispatcher that centralizes every outbound HTTP delivery behind one well-tested module. Capabilities: optional HMAC-SHA256 signing (
X-Logtide-Signature: t=<unix>,v1=<hex>, timing-safe to verify); retry with exponential backoff (1s, 5s, 25s, 2m, 10m, transient-only); a dead-letter queue modeled aswebhook_deliveries.status='dead', listable and replayable from the dashboard; SSRF protection by reusing thesafeFetchguard; per-organization concurrency limiting; and a bounded delivery log exposed via an org-scoped API and a “Webhook Deliveries” page under Settings. All five existing webhook senders were migrated onto it — the error/monitor/incident paths previously used a barefetchwith no SSRF guard, closing three sibling gaps. Documented in the new Webhooks guide - Request context propagation across HTTP, jobs and the DB layer (#222, closes #213): a new AsyncLocalStorage-backed
RequestContextprimitive in@logtide/shared/context. A Fastify plugin establishes the context right after auth resolves; BullMQ producers piggy-back_ctxon payloads and consumers wrap processors incontext.run/runAsSystem; cron callbacks run underrunAsSystem. Thepg.Poolis patched to prepend a/* req=... */SQL comment so requests are traceable in slow-query logs, and each reservoir engine injects the same correlation in its native form (Timescale SQL comment, ClickHousequery_id+log_comment, MongoDB$comment) - Tenant isolation audit, test suite and CI tripwires (#228, #219): a living audit document, a dedicated isolation test suite (the
createIsolatedTenantsfixture plus per-area tests for query, traces, crud, api-key auth, audit-log and metering) that runs in CI, a static tripwire (check:tenant-scoping) that flags Kysely queries on tenant tables lacking org/project scoping and fails the CI typecheck on new unscoped sites, an opt-in runtime Kysely guard (TENANT_GUARD=1), and a PR template carrying a tenant-safety checklist. The codified model: organization is a hard boundary for everyone, API keys are project-scoped, session users are org-wide by current policy - Capability enforcement completed across resource creation (#214 follow-up): the previously defined
notifications.max_channelsandapikeys.maxlimits are now enforced at channel/key creation (API keys counted org-wide), and two new limit keys land with enforcement:sigma.max_active_rulesanddashboards.max_custom. All defaults stay unlimited for self-hosted OSS - Ingestion health visibility for admins: a new
GET /api/v1/admin/stats/ingestion-healthendpoint aggregates the last 24h of new ingestion health counters (ingestion.pii_rejected,ingestion.detection_enqueue_failed, and more) plus SIEM enrichment availability, surfaced as an “Ingestion health (24h)” card that highlights in red when failures occurred. Sigma-detection and exception-parsing job enqueues now retry once and record a counter on failure instead of dropping silently - Scheduled email digest reports — groundwork, disabled (#209, part of #154): the foundation for periodic email summaries of log activity. Intentionally disabled in this beta — the worker leaves the scheduler unregistered and no digests are sent. The service, schema and tests are merged so the remaining scope can be finished. What landed:
digest_configs/digest_recipientstables, anICronRegistryabstraction with native-cron implementations for both BullMQ and Graphile, and a generator that computes log volume from the hourly-stats continuous aggregate - Test debt paid down: six backend coverage exclusions are gone, the 12 previously untested
@logtide/sharedZod schemas gain dedicated tests, CI now runs the shared and frontend unit suites alongside the backend, and frontend component testing is enabled (Svelte 5 + testing-library). A dedicated Playwright journey covers the full traces flow (list, filters, waterfall, span panel and trace/log correlation) - Migrations 044–048:
digest_email_reports(044),metering_events(045),organization_entitlements(046),webhook_deliveries+webhook_delivery_attempts(047), and additiveactor_type/actor_id/outcomecolumns on theaudit_loghypertable (048)
Changed
- BREAKING: unified webhook event envelope (#218 follow-up): every outbound webhook delivery now serializes to one envelope
{ id: "evt_<uuid>", type, version: 1, occurredAt, organizationId, projectId, data }instead of four bespoke payload shapes. Event types arealert.triggered,incident.created,error.detected,monitor.status_changedandchannel.test; the per-typedatapayloads keep their previous snake_case fields minusevent_type/timestamp. Envelope and per-type Zod schemas ship in@logtide/shared(webhookEnvelopeSchema,parseWebhookEvent); deliveries carry a newX-Logtide-Event-Version: 1header. See the Webhooks guide for the full envelope reference - OTLP log metadata shape: resource attributes now land under
metadata.resourceinstead of being spread flat (log-record attributes keep the flat namespace and win collisions there; nothing is lost). Structured log bodies (kvlist/array) keep their decoded structure undermetadata['otel.body']alongside the stringifiedmessage. Logs ingested before this change keep their old shape - Reservoir log query params now require
projectId(#228):projectIdwas optional on the reservoir query/count/aggregate params, which is what allowed an unscoped log read to compile. It is now required, with an explicitGLOBAL_SCOPEsentinel for the handful of intentional platform-wide reads
Fixed
- Sigma search by MITRE technique/tactic/tag always 500ed: the queries compared
TEXT[]columns with a::jsonbcast and failed on every request since they shipped; they now use proper text-array containment and are covered by regression tests - Migration prefix collision that could break production migrate (#229): two migration pairs shared a numeric prefix, which Kysely’s position-by-position validation could reject. Fixed by renumbering the files plus an idempotent pre-migration repair that renames the matching
kysely_migrationrows on already-applied databases; a new CI guard rejects future duplicate prefixes. Verified on both a fresh schema and a database seeded into the pre-fix state DELETE /api/v1/sigma/rules/:idand custom-dashboard update/delete returned 500 instead of 404 for a missing or cross-org id (#229): these now map to a proper404. None leaked or mutated another tenant’s data — the scoping already prevented the operation; this only corrects the HTTP status- Silent failures made visible across admin and monitoring UI: the admin version check no longer claims “up to date” when GitHub is unreachable; the monitoring page no longer swallows incidents/maintenance load errors; GeoLite2/IPsum unavailability warns once per outage instead of spamming every lookup
- API-key
last_usedwrite amplification on hot keys (#222): every authenticated request updatedapi_keys.last_used, producing ~100 updates/sec on a single row under load. The update is now debounced to at most once per 60s per key