# Observability and privacy

Investigate failure without turning telemetry into a leak.

Components that organize evidence, check integrity and sanitize telemetry. The focus is reconstructing what happened while keeping explicit boundaries on which information may travel.

OTLP · Observability · Privacy · Integrity · Testing

Module in development · local tests

## The problem with an unexplained operation

A failure leaves signals in different places: states, events, responses and records. Without a way to relate them, an investigation can become an impression rather than an evidence-based explanation. Copying entire payloads into indexes or telemetry can also spread sensitive information. I led component development that addresses both requirements together: event reconstruction and exposure boundaries.

The goal is not indiscriminately increasing log volume. It is preserving enough information to examine a hypothesis, identify the origin of evidence and recognize when an instrumentation gap prevents a reliable conclusion about the operation.

## What this case makes assessable

This briefing describes Python components for forensic analysis and OTLP telemetry projection. Available evidence comes from local tests. There is no public dashboard connected to real incidents and no disclosure of operational records. Examples are generalized behavioral descriptions and diagrams, allowing engineering decisions to be discussed without exposing private material.

I led implementation and validation of integrity and sanitization rules. The slice was examined as component work rather than assuming that code availability represents activation of an entire observability platform. The difference determines both what can be claimed and what further validation would be required.

## Evidence as a verifiable object

The forensic core organizes evidence references with identity and a hash. A reference connects the item being examined to its origin, while the hash supports an integrity check of the corresponding content. Changed content needs to be distinguishable from another occurrence of the same reference. This structure makes an investigation more reproducible than a collection of messages without contracts.

Integrity is not the same as truth: a hash can establish that content remained unchanged but cannot establish that its interpretation is correct. Identity, origin, consistency and context therefore remain part of the reasoning, alongside the mechanical integrity check.

## Telemetry with permitted fields

The OTLP projection validates permitted fields and types before producing telemetry information. It does not assume every incoming attribute is safe to forward. Sanitization reduces the possibility of secrets or sensitive payloads becoming searchable attributes in secondary destinations. Incompatible values must be rejected or represented as explicit gaps according to the applicable contract.

This places privacy inside the transformation path. Removing a secret after it has circulated is not equivalent to preventing propagation. The control also makes expected structure testable instead of relying on occasional visual inspection of logs. New attributes require a deliberate decision about their type, purpose and exposure.

## Reconstruction and explicit uncertainty

During incident reconstruction, the central question is what each piece of evidence actually supports. An absent response can mean interruption, an unavailable dependency or simply missing instrumentation. It is incorrect to infer health merely because no error was observed. A system should preserve the distinction between a positive signal, a known failure and inability to establish the state.

This is particularly relevant to agent workflows: a success message may precede a persisted effect, and an interrupted execution may already have completed a step. Investigation needs to follow states and evidence rather than relying only on the wording of messages.

## Failures the contract must reveal

Changed content, inconsistent references, incompatible attributes and disallowed fields are examples requiring defined handling. Protection needs to hold for malformed and incomplete inputs as well as expected ones. Negative tests help establish that an error path does not produce an apparently normal, silently empty or more permissive output than the intended path.

The public boundary of this case is deliberate: incident contents, internal paths, private service names and original payloads are not disclosed. A technical demonstration can use synthetic examples without weakening the privacy principles that the components themselves are intended to protect.

## Why I chose validation and minimization

I preferred projection through permitted fields over forwarding the received payload. This gives a smaller information surface and a testable contract, while requiring deliberate expansion when a legitimate new need appears. I preferred integrity-checkable references over mixing evidence into free text. This improves traceability while preserving the need to interpret content carefully.

I also separated component validation from operational certification. That prevents a unit test from becoming justification for claiming that every collector, exporter and destination in a platform works together. Each boundary should have evidence proportional to the claim being made about it.

## Executed evidence and its boundaries

The October 4, 2026 audit passed 12 forensic-core tests and 18 OTLP privacy tests. These 30 tests exercise local integrity, reference and sanitization behavior in the analyzed slice. They make it possible to examine concrete rules and explain how the implementation handles an input that does not satisfy its contract.

The work was analyzed on a supplementary development line. I do not claim complete integration into the main version, production activation, coverage of every execution path or equivalence to end-to-end LLM tracing. These components are also not presented as a complete MLOps lifecycle or monitoring of trained models.

## Advancing through a measurable hypothesis

A next step would select a small workflow, define expected events and verify whether its execution can be reconstructed end to end. Then it would simulate signal loss and establish that gaps become explicit. Certification should use synthetic data and inspect the telemetry destination to confirm that minimization remains valid after export, not merely at the first transformation point.

That future work is not included in the capabilities proven here. This case demonstrates an approach to implementing observability: make evidence verifiable, constrain information exposure and preserve uncertainty when the available signals cannot support a conclusion about the full system.

---
César Machado · https://cesarmachado.com