Prepare a CSV video batch before creating API jobs

Prepare a CSV video batch in two steps: validate the entire input into a reviewable manifest, then submit approved items through a separate worker. Do not create API jobs while you are still discovering malformed rows, duplicate identifiers or unexpectedly broad source URLs.

This guide starts with a spreadsheet export containing one intended video per row. Its Python example performs an offline preview only. It creates no jobs and does not claim that a syntactically accepted video is available to download.

Decide what one row means

Give each business item a stable item_id and a source_url. Keep identifiers as text in the spreadsheet so leading zeros are not accidentally lost. Record output settings separately as a versioned configuration for this batch, or add explicitly validated columns if settings vary by item.

An item ID should survive a corrected URL or a reordered spreadsheet. A row number is useful for an error message, but becomes a poor durable identity after someone sorts the sheet. Scope identifiers to your project or tenant in the database; the same local label in another customer's import must not identify the same work.

Keep the original CSV and the reviewed manifest as separate revisions. Your importer can trim accidental surrounding whitespace, but should not silently reinterpret unfamiliar columns or discard rows it cannot understand.

Prevent a single-video import from becoming a collection import

Tornado's create-job reference documents POST /jobs and a url field. It also describes collection detection: qualifying playlist, channel and show URLs can create a batch. One spreadsheet row therefore does not always mean one resulting media job.

If the import is intended to contain individual YouTube videos, use an explicit input profile and flag collection links for review. The example accepts only HTTPS watch URLs with a single v parameter, or short youtu.be URLs, with an eleven-character video identifier. It rejects all extra query parameters, fragments, channel paths and playlist paths. That is this example's conservative import policy, not a complete list of Tornado's supported URL forms.

Do not automatically remove a playlist parameter and pretend that the user's intent was unambiguous. Have the input owner choose the individual video or explicitly approve a collection workflow. For a collection, track the returned batch and child jobs instead of forcing the result into a one-row/one-job assumption.

Parse CSV with a CSV reader

A spreadsheet cell can contain a quoted comma or newline. Splitting each line on commas will not reliably preserve fields. The Python CSV documentation describes the reader and its dialect options; when reading a file, open it with newline="" and an explicit encoding such as utf-8-sig for this workflow.

The example requires the exact two-column header shown. It rejects empty data, duplicate item IDs and blank records. Repeated video IDs produce a review list while preserving both items: two distinct business items may intentionally use the same video. Resolve those repetitions before submission.

import csv
import io
import re

VIDEO = re.compile(
    r"https://(?:www\.)?youtube\.com/watch\?v=([A-Za-z0-9_-]{11})"
    r"|https://youtu\.be/([A-Za-z0-9_-]{11})"
)
ITEM = re.compile(r"[A-Za-z0-9_-]{1,64}")


def prepare_csv(text, max_rows=1000):
    # These are application limits, not Tornado API limits.
    if type(max_rows) is not int or max_rows <= 0:
        raise ValueError("max_rows must be a positive integer")
    if len(text) > 1_000_000:
        raise ValueError("CSV exceeds this example's character limit")
    reader = csv.reader(io.StringIO(text.removeprefix("\ufeff"), newline=""), strict=True)
    if next(reader, None) != ["item_id", "source_url"]:
        raise ValueError("Expected item_id,source_url header")
    rows, repeats, item_ids, first_video = [], [], set(), {}
    for record, fields in enumerate(reader, 1):
        if record > max_rows or len(fields) != 2:
            raise ValueError(f"Record {record}: row limit or column count")
        item_id, source_url = (field.strip() for field in fields)
        match = VIDEO.fullmatch(source_url)
        if not ITEM.fullmatch(item_id) or match is None:
            raise ValueError(f"Record {record}: invalid ID or unsupported URL profile")
        if item_id in item_ids:
            raise ValueError(f"Record {record}: duplicate item_id")
        item_ids.add(item_id)
        video_id = match.group(1) or match.group(2)
        if video_id in first_video:
            repeats.append((item_id, first_video[video_id]))
        else:
            first_video[video_id] = item_id
        rows.append({"item_id": item_id, "source_url": source_url})
    if not rows:
        raise ValueError("No data records")
    return rows, repeats


sample = (
    "item_id,source_url\n"
    "clip_a,https://www.youtube.com/watch?v=aaaaaaaaaaa\n"
    "clip_b,https://youtu.be/aaaaaaaaaaa\n"
)
rows, repeats = prepare_csv(sample)
assert len(rows) == 2
assert rows[1]["item_id"] == "clip_b"
assert repeats == [("clip_b", "clip_a")]

The three inline assertions and seventeen additional checks were executed locally. They covered BOM and CRLF handling, empty input, headers, duplicate IDs, unsupported URLs, extra columns, malformed quoting, blank records and application limits. The video identifiers are synthetic; no availability check, download or paid operation was performed.

The helper either returns the complete preview or raises ValueError or csv.Error. Callers should catch both and show a correction message before any submission begins. record refers to a parsed data record, not necessarily a physical line, because CSV can contain quoted newlines. The character and row limits are illustrative application choices, not API quotas. An upload service should also cap bytes before decoding the file.

Review duplicates without losing intent

The preview recognizes the same video ID in the two accepted URL forms. It does not deduplicate arbitrary links, follow redirects or compare downloaded content. If you extend the supported URL profile, test its identity rules rather than deleting all query strings as a shortcut.

Where repeated items are intentional, preserve both identities. Where they are accidental, remove the redundant item in a new reviewed manifest. Do not merge items solely because their titles or source URLs resemble one another.

Freeze the preview before sending requests

Persist the approved manifest, its configuration version and a run ID before distributing work. Then record each submission attempt and any returned job or batch identifier. The CSV's item_id and source_url are application fields: map the source into the documented API request rather than posting this entire manifest as though it were a Tornado schema.

A second import of the same file should consult your persisted records. An in-memory duplicate check only covers this call; it does not protect against another worker, a process restart or an uncertain network response. The worker restart guide explains how to preserve known job IDs and ambiguous submissions.

Keep submission concurrency and account limits separate from the number of rows in the file. A validated thousand-row manifest is not permission to send a thousand simultaneous requests. Use the bounded-queue pattern to control admission into downstream stages.

Test the import and the running batch separately

Use synthetic CSV fixtures to test input decisions without network access. Then test persistence and submission orchestration with an injected adapter. Confirm that no job is submitted when validation fails near the end of the file, and that restarting the importer does not discard its recorded submission state. The offline testing guide explains the limits of that evidence.

Input validation does not establish source accessibility, processing rights, output compatibility or actual usage. Check those in the appropriate stage using a small authorized workload. If some submitted items later fail, use the partial-batch recovery guide to resume individual outcomes instead of reimporting the complete CSV.

Start with one Tornado media workflow, then connect its submission adapter to a reviewed manifest. The useful milestone is a traceable path from an imported item to its recorded result, not merely a spreadsheet that parsed successfully.