Debug a media API workflow with a stable job identifier, a named stage and a small set of observed outcomes. Keep source URLs, signed delivery links, authorization headers and raw response bodies out of routine diagnostic events. You should be able to identify where a workflow stopped without giving every log reader access to its media.
This guide designs application logs for the path from job submission to downstream consumption. It does not describe a logging feature that Tornado enables for your application. Use it alongside the first media workflow when adding your own observability.
Correlate the workflow without copying the response
Keep an application attempt ID before submission, then associate the returned Tornado job ID with that attempt in your operational store. A submission whose response was lost may have no acknowledged job ID; retain that uncertainty instead of inventing one or immediately submitting again.
The job-status endpoint provides the existing job's status and delivery information. That response is useful operational data, but serializing the whole object into a log makes every present and future field part of your telemetry policy.
Instead, emit an event at each meaningful boundary: submission acknowledged, remote outcome observed, delivery read attempted, media validated and downstream outcome recorded. Use the same identifier to connect the events while preserving their distinct stages. A successful status lookup, a completed media job and a usable transcript are different observations.
For example, an event saying delivery_read, failed, HTTP 403 identifies an access failure at the consumer boundary. It does not establish that processing failed or that the object was deleted. The expired-link diagnostic guide explains how to investigate those cases.
Treat a signed URL as access, not as a label
AWS documents presigned URLs as temporary access credentials and identifies URI logging as a way their signatures can spread to clients and intermediaries. Its logging guidance for presigned URLs describes redaction options. An application's ordinary job event rarely needs the complete URI at all.
Removing only a query string is not a complete content-privacy policy: paths can still contain object names or other identifying data. Prefer an internal job or artifact reference when that is sufficient for diagnosis. Store the destination details in the authorized operational record where they are needed for retrieval.
Also check exceptions and HTTP instrumentation. A carefully written event can coexist with an HTTP client's debug log that prints the request URL. Browser errors, tracing exporters and support-ticket attachments can become separate copies. Review each actual output path rather than assuming one formatter governs the entire application.
Build events from an explicit field list
The example below creates a new object instead of removing suspicious keys from a copied API response. Its stages and outcomes are application-defined, not Tornado response values. The caller must map observations deliberately.
It accepts a UUID-shaped job ID, fixed labels and optional integer HTTP status and elapsed time. Unknown extra fields are ignored. It does not accept a free-text error message or a URL field.
import json
from uuid import UUID
STAGES = {"status_lookup", "delivery_read", "media_validation", "downstream"}
OUTCOMES = {"succeeded", "failed", "unresolved"}
def diagnostic_event(observation):
job_id = observation.get("job_id")
if not isinstance(job_id, str):
raise ValueError("Invalid job identifier")
try:
normalized_id = str(UUID(job_id))
except ValueError:
raise ValueError("Invalid job identifier") from None
stage = observation.get("stage")
outcome = observation.get("outcome")
if not isinstance(stage, str) or stage not in STAGES:
raise ValueError("Invalid stage")
if not isinstance(outcome, str) or outcome not in OUTCOMES:
raise ValueError("Invalid outcome")
event = {
"event": "media_stage_observed",
"schema_version": 1,
"job_id": normalized_id,
"stage": stage,
"outcome": outcome,
}
for key, low, high in (("http_status", 100, 599),
("elapsed_ms", 0, 86_400_000)):
value = observation.get(key)
if value is not None:
if type(value) is not int or not low <= value <= high:
raise ValueError("Invalid numeric observation")
event[key] = value
return event
sample = {
"job_id": "00000000-0000-4000-8000-000000000001",
"stage": "delivery_read",
"outcome": "failed",
"http_status": 403,
"elapsed_ms": 120,
"delivery_url": "https://example.invalid/file?token=SYNTHETIC_SECRET",
"error": "Request failed at SYNTHETIC_SECRET",
}
encoded = json.dumps(diagnostic_event(sample), sort_keys=True)
assert "SYNTHETIC_SECRET" not in encoded
assert "delivery_url" not in encoded
assert json.loads(encoded)["http_status"] == 403
The numeric bounds are example schema choices, not API limits or a one-day processing promise. Change them deliberately if your measurements need a different range. Add the event timestamp and approved application context in your logging envelope; measure durations with a suitable monotonic clock. See the latency measurement guide for timing boundaries.
The exact snippet and additional invalid-label, numeric-boundary and extra-field checks were run locally using synthetic values. They verify this event constructor, not the surrounding logger, collector or production deployment. UUID syntax does not establish that the ID belongs to the current user; obtain it from an authorized record. Internal identifiers still warrant controlled access.
The constructor raises on invalid required fields. If your telemetry path catches that error, emit a fixed schema-error counter or fixed message without attaching the rejected object. Logging observation inside the exception handler would defeat the design.
Keep troubleshooting detail in the right place
A small event should answer which stage ran, what it observed and which operational record to inspect next. It need not contain every detail required to resolve the incident. An authorized operator can retrieve the relevant job and storage evidence separately.
Use fixed error categories derived from documented observations when they materially improve diagnosis. Preserve an unresolved outcome for missing evidence rather than converting every timeout into failure. Do not interpolate a provider's free-text response into a category field.
The OWASP logging guidance recommends excluding sensitive values, validating event data and testing logging behavior, including failures. Apply those principles to the receiving system as well: decide who can search events, how long they remain available and how operational exports are handled. A restrictive event schema does not make the log store public data.
Test the path that actually emits logs
Use clearly synthetic marker strings in a controlled test for URL parameters, headers and response bodies. Exercise success, denied access, malformed responses and timeouts. Inspect the emitted application log, exception capture and trace output for those markers. Do not use real credentials to test whether your redaction works.
Check that the sanitized record is still useful: can a developer distinguish acquisition failure from a denied delivery read, and locate the authorized job record? Then verify that rejected telemetry data does not cause an unrelated media operation to be retried.
Start with one existing completed job in the Tornado dashboard, add stage events in your application, and inspect the full consumer handoff. If you need help, contact support with the job ID and a sanitized description of the failing stage. Share only the details needed through the appropriate support path, rather than pasting an entire response into a public issue.