If a subtitle file produces a UnicodeDecodeError or displays broken accents, preserve the original bytes before changing anything. Decode with the encoding your source contract specifies, reject invalid input, and keep text decoding separate from subtitle parsing. Removing undecodable bytes can make an import succeed while changing what a speaker said.
This guide implements a small UTF-8 input gate in Python. It accepts an optional UTF-8 byte order mark (BOM), applies an application-defined size limit, and stops on invalid bytes. It does not guess the encoding or repair already corrupted text.
Identify which stage changed the text
Start with the downloaded file, not a copied preview. Compare a known short caption with the original in a trusted editor and with the text your application stores.
A decoding exception means the chosen decoder could not interpret the bytes. Text such as café can instead be valid Unicode that was decoded incorrectly earlier and then saved again. Successfully reading that file as UTF-8 will not restore café automatically. A replacement character � may already be present in the source; its presence alone does not identify where information was lost.
Keep an unchanged source copy and record a checksum before conversion. The checksum guide explains how to identify a particular artifact without treating its filename as proof of identity.
Read UTF-8 subtitles with an explicit policy
The following function operates on a local regular file that your application has already downloaded. The 2 MiB default is an example application limit, not a Tornado API limit. Choose a limit appropriate to your caption workflow.
from pathlib import Path
def read_utf8_subtitles(path: Path, max_bytes: int = 2 * 1024 * 1024) -> str:
if type(max_bytes) is not int or max_bytes <= 0:
raise ValueError("max_bytes must be a positive integer")
with path.open("rb") as source:
raw = source.read(max_bytes + 1)
if len(raw) > max_bytes:
raise ValueError("subtitle input exceeds the byte limit")
text = raw.decode("utf-8-sig", errors="strict")
if not text.strip():
raise ValueError("subtitle input is empty")
if "\x00" in text:
raise ValueError("NUL characters require input review")
return text
Use read_utf8_subtitles(Path("captions.srt")) and pass the returned string to a format-aware subtitle parser. Catch decoding and validation errors at the workflow boundary, mark the artifact for review, and retain the original. A decoding failure is not a reason to submit the same media job repeatedly.
Python's utf-8-sig codec accepts ordinary UTF-8 and skips its signature when present at the beginning. Strict decoding raises an error for invalid byte sequences. Avoid adding errors="ignore" or errors="replace" to get past an import failure: those policies discard or substitute input rather than prove it is correct.
The function preserves line endings and text content after decoding, apart from the initial signature. It does not remove accents, normalize Unicode, rewrite timestamps, or strip subtitle markup. The NUL check is a conservative application rule: a NUL-containing file needs investigation, not an automatic claim that it is UTF-16.
Handle an unknown encoding without guessing silently
If UTF-8 decoding fails, check the export settings or documented encoding of the source. Ask for a new UTF-8 export when possible. If the producer confirms another encoding, convert a copy using that specific decoder with strict error handling, then compare representative captions before accepting it.
Do not cycle through encodings until one succeeds and treat that as detection. Python's codec documentation explains why an encoding cannot always be reliably inferred from bytes alone. In a multilingual catalog, keep the encoding decision and its source with the artifact record so that a later retry follows the same policy.
An encoding change is not a translation. It should preserve the intended characters. Review names, punctuation, accents and non-Latin text, rather than checking only a timestamp line made of ASCII digits.
Validate the subtitle format after decoding
Readable text can still be an HTML error page, an incomplete download, or malformed captions. Decoding success does not establish that a file is SRT or WebVTT. Check download status and content before parsing, then validate cue structure and timestamps separately.
For SRT, continue with subtitle timing validation. For browser tracks, use a proper conversion when necessary; renaming .srt to .vtt is not conversion. The WebVTT display guide covers the browser handoff. Render imported captions through an appropriate text or subtitle component rather than treating arbitrary source text as trusted HTML.
In a Tornado workflow, inspect the documented job status response for an available subtitle result. A subtitle URL is optional; this example neither guarantees subtitles for every source nor specifies the encoding of every delivered file. Apply the gate to the artifact actually received.
What the example proves
Local synthetic tests cover multilingual UTF-8, a leading BOM, preserved CRLF line endings, invalid and truncated byte sequences, empty input, NUL rejection, size boundaries and invalid limit configuration. They do not demonstrate live source delivery, encoding detection, subtitle timing correctness, or recovery of text that was already corrupted.
The function reads at most the configured limit plus one byte before decoding. It still holds both bytes and decoded text in memory; it is not a streaming subtitle parser. Its local-file assumption also excludes network timeouts and concurrent replacement of a file. Download completely into application-controlled storage before running it.
To connect delivery, validation and downstream processing, build your first Tornado media workflow, then add this text check before importing captions into your application.