AppProfileSafe routes every auditable operation through a structured event pipeline with typed payloads, per-sink delivery tracking, persistent queue storage, and classified failure handling. This page documents the internal systems for architects and integration engineers. The local audit trail is part of Community Edition. The Event Pipeline with SIEM, webhook, and syslog sinks requires an Enterprise license.
Every export, import, and simulation is recorded in a tamper-evident local audit log before any event reaches the pipeline. The hash-chained CSV is the single source of truth — pipeline delivery is additive, never a prerequisite.
Each audit entry contains an HMAC-SHA256 hash linking it to the previous entry. Tampering with any entry breaks the chain. Integrity is verified on every application startup and available on demand via the Audit Log Viewer.
What auditors see: WHO performed the operation, WHAT changed, WHEN it happened, WHETHER it was validated first, and HOW integrity was maintained.
The GUI includes a built-in viewer that displays all audit entries with filtering by date range, user, and action type. Integrity verification runs inline — broken chain links are highlighted immediately.
Community Edition: Full viewer access with integrity verification. Enterprise adds Compliance Report generation (automated PDF) from the same audit data.
Every dry run and live import produces a structured diff CSV. The CSV’s SHA256 hash is recorded in the audit entry, creating a verifiable link between the audit trail and the detailed change record.
Tamper detection: If the diff CSV is modified after the audit entry was written, the hash mismatch is detected during integrity verification.
Every auditable operation — export, import, simulation, integrity check, license validation — emits a structured event. Events are wrapped in a typed envelope, classified by category and severity, and dispatched to all configured sinks in parallel. The Event Pipeline requires an Enterprise license.
Every event is wrapped in a ProfileSafeEvent envelope that provides a consistent structure across all event types. The envelope carries identity, context, timing, and routing metadata independent of the payload content.
Deduplication: Each event receives a deterministic GUID derived from the audit sequence number and timestamp. Sinks can use this EventId to detect and discard duplicate deliveries after retry.
| Field | Purpose |
|---|---|
| EventId | Deterministic GUID for deduplication |
| Timestamp | UTC creation time (ISO 8601) |
| Source | Emitting subsystem (CLI, GUI) |
| Category | Routing category (Export, Import, Security, …) |
| Severity | Info, Warning, Error, Critical |
| Context | UserName, MachineName, Domain |
| PayloadType | Discriminator for typed deserialization |
| Payload | Typed payload object (see below) |
The envelope’s Payload field carries a strongly-typed object determined by PayloadType. Each payload type has a dedicated schema with fields specific to its operation domain. The PayloadTypeMap resolves type names to .NET types for deserialization.
Extensibility: New payload types are added by registering a type mapping. Existing sinks serialize any payload type to their output format without modification.
| PayloadType | Domain |
|---|---|
| AuditEntry | Audit log operations |
| ExportEvent | Export operations |
| ImportEvent | Import operations |
| SimulationEvent | Dry run results |
| LicenseEvent | License validation |
| PreflightEvent | Preflight checks |
Events are classified into categories that determine routing behavior. Sinks can subscribe to specific categories or receive all events. Category assignment is derived from the originating AuditAction.
Severity mapping: Categories also influence severity classification. IntegrityCheckFailed and UnauthorizedAccess are always Critical. Operation failures are Error. Partial successes are Warning.
| Category | Events |
|---|---|
| Export | Started, Completed, Failed |
| Import | Started, Completed, Failed |
| Simulation | DryRun, SimulateMappingExecuted |
| Security | IntegrityCheckFailed, UnauthorizedAccess |
| Privacy | PersonalDataExported/Imported |
| System | LogArchived, SettingsChanged |
Each sink receives events independently through the pipeline. A failure in one sink does not affect delivery to others. Sinks are configured in XML with per-sink authentication, format selection, and redaction policies.
Delivers events to SIEM platforms via HTTP POST. Supports CEF, JSON, and LEEF message formats with configurable Content-Type headers. Authentication is resolved automatically from the endpoint configuration.
Authentication: Bearer token, API Key (custom header), Basic auth, Splunk HEC token. Auto-detection based on configured credentials. Per-request timeout via CancellationToken (no shared HttpClient.Timeout mutation).
| Format | Target |
|---|---|
| CEF | ArcSight, QRadar, generic SIEM |
| JSON | Splunk HEC, Elastic, Azure Sentinel |
| LEEF | IBM QRadar |
Sends JSON event payloads to arbitrary HTTP endpoints. Each request includes an HMAC-SHA256 signature header computed over the request body, allowing receivers to verify payload integrity and authenticity.
Signature headers: X-Signature-256 (HMAC hex digest), X-Timestamp (request time), X-Event-Id (deterministic GUID). Signing key stored in Windows Credential Manager.
Use cases: Slack notifications, Azure Logic Apps, ServiceNow incident creation, custom automation endpoints.
Writes events to the Windows Application Event Log with source name AppProfileSafe. Severity is mapped to EventLogEntryType (Information, Warning, Error). Integrates with existing Windows Event Forwarding (WEF) and Azure Monitor Agent (AMA) infrastructure.
Setup: Enable <WindowsEventLogEnabled>true</WindowsEventLogEnabled> in settings.xml. Run once as administrator to register the event source. Existing WEF subscriptions collect events automatically.
Sends events via UDP to a syslog collector. Severity is mapped to syslog severity levels (0–7). Message format follows RFC 5424 with structured data elements for machine-readable parsing.
Mapping: Critical → 2 (Critical), Error → 3 (Error), Warning → 4 (Warning), Info → 6 (Informational). Facility configurable, defaults to local0.
Events are persisted to an NDJSON file before dispatch. Each event tracks delivery state per sink independently. The queue survives application restarts, crashes, and network outages — pending events are retried on next startup.
The event queue is stored as a newline-delimited JSON file. Each line contains a complete event with its delivery state map. The file is written using the AtomicFileWriter pattern — write to temp, then atomic rename — ensuring no partial writes on crash.
Why NDJSON: Append-friendly, human-readable, and parseable line-by-line without loading the entire file. Each line is self-contained — a corrupted line does not invalidate the rest of the queue.
Compaction: After successful delivery to all sinks, completed events are removed during the next queue compaction cycle. The queue file never grows unbounded.
Each event maintains a delivery state per configured sink. States are tracked independently — an event can be Delivered to SIEM, Pending for webhook, and Failed for syslog simultaneously. This per-sink granularity prevents one failing sink from blocking others.
| State | Meaning |
|---|---|
| Pending | Not yet attempted |
| Delivered | Successfully sent and acknowledged |
| Retrying | Failed, scheduled for retry |
| Failed | Permanent failure, moved to dead letter |
Dispatch failures are classified to determine whether retry is appropriate. Network timeouts and HTTP 5xx responses are retryable. Authentication failures (401/403) and malformed payloads (400) are non-retryable and immediately move to dead letter.
Retryable: Network timeout, DNS resolution failure, HTTP 500/502/503/504, connection refused.
Non-retryable: HTTP 400 (bad request), HTTP 401/403 (auth failure), serialization errors, invalid sink configuration.
Retryable failures use exponential backoff with configurable base delay and maximum retry count. Each retry increments the attempt counter in the delivery state. After the maximum retry count is exceeded, the event is reclassified as permanently failed.
Backoff: Base delay doubles per attempt. Default: 5s, 10s, 20s, 40s, 80s. Maximum 5 attempts before dead letter.
Persistence: Retry state survives application restart. On next startup, the queue processor resumes pending retries with the correct attempt count and backoff delay.
The health subsystem runs checks on startup and on demand. Each check reports a status (Healthy, Degraded, Unhealthy) with diagnostic detail. Results are displayed on the GUI dashboard. Community Edition runs disk space and audit integrity checks. Enterprise additionally monitors SIEM, webhook, and queue health.
Monitors available disk space on the volumes used by audit logs, event queue, and export archives. Reports Degraded when space falls below a configurable warning threshold and Unhealthy below a critical threshold.
Checked volumes: Audit folder volume, queue store volume, export root volume (if configured). Thresholds configurable in settings.xml.
Inspects the event queue for stuck or aging events. Reports Degraded when events older than a configurable age threshold remain undelivered. Reports Unhealthy when the queue file is inaccessible or corrupted.
Diagnostics: Reports pending event count, oldest pending event age, and per-sink delivery summary. Corrupted NDJSON lines are reported with line number.
Tests connectivity to the configured SIEM HTTP endpoint. Sends a lightweight probe request and validates the response status code and timing. Reports Unhealthy if the endpoint is unreachable or returns authentication errors.
Preflight integration: The CLI preflight mode includes the SIEM health check. A failing SIEM check with exit code 4 prevents operations that would generate undeliverable audit events.
Tests connectivity to configured webhook endpoints. Validates that the endpoint is reachable, responds within the timeout, and accepts the expected Content-Type. Reports per-endpoint status independently.
HMAC verification: The health check sends a test payload with a valid HMAC signature. Endpoints that reject the signature are reported as Unhealthy with an authentication detail message.
The event pipeline is designed so that sink failures never block core operations. An export or import completes even if SIEM is down. Events are queued, retried, and eventually dead-lettered — but the local audit trail is always written first.
Every dispatch failure is classified before determining the next action. Transient infrastructure failures (network, DNS, server errors) are retried with exponential backoff. Permanent failures (authentication, malformed payload, invalid configuration) skip retry and move directly to dead letter.
Why classification matters: Retrying a 401 wastes resources and delays dead letter processing. Not retrying a 503 loses events that would have been delivered seconds later. Correct classification maximizes delivery rate while minimizing latency.
Pipeline isolation: Classification and retry happen per sink. A webhook returning 503 retries independently while SIEM delivery continues unaffected.
| Failure Type | Examples | Action |
|---|---|---|
| Retryable | Timeout, DNS failure, HTTP 500/502/503/504 | Exponential backoff, max 5 attempts |
| Non-retryable | HTTP 400, 401, 403, serialization error | Immediate dead letter |
| Configuration | Invalid endpoint URL, missing auth | Sink disabled, health check Unhealthy |
An event can be successfully delivered to some sinks and fail for others. The per-sink delivery state model means that a webhook failure does not cause re-delivery to SIEM. Each sink’s state advances independently — Delivered, Retrying, or Failed — without affecting the others.
Queue compaction: An event is only removed from the queue when all sinks report Delivered or Failed. Partial delivery keeps the event in the queue with mixed states until every sink has reached a terminal state.
Audit guarantee: The local audit log is written synchronously before event dispatch. Even if all sinks fail permanently, the local audit trail is complete and intact — in both Community and Enterprise editions.
Events that exhaust all retry attempts or encounter non-retryable failures are moved to the dead letter store. Dead-lettered events retain the full event envelope, all delivery attempt metadata, and the failure reason for each failed sink.
Diagnostics: The dead letter store is inspectable via the queue health check. Accumulated dead letters trigger a Degraded health status, alerting operators that events are not reaching their configured destinations.
Recovery: After resolving the underlying issue (e.g. rotating an expired API key), dead-lettered events can be replayed. The deterministic EventId ensures idempotent processing on the receiving side.
Download Community Edition to inspect the local audit trail, run simulations, and review diff reports. Contact us for an Enterprise license to configure SIEM forwarding, webhooks, and compliance reporting.