Machine Learning Data Extraction from YouTube Videos

Rules work when the text is regular. Speech is not: a reviewer says “the X2”, “Aurora’s new one” and “this thing” for the same product, gives a price as “two forty-nine”, and compares three products in one sentence. A model reads the whole passage, so it can tell which product a price belongs to, whether a sentence is praise or criticism, and which brand is the one under review.

The fields are not fixed in advance either. Classic named entity recognition learns a set of labels such as person, organization and location at training time. Schema-conditioned extraction takes the labels at request time, so the same model returns brands and prices for one request and speakers and claims for the next. For the background, see named entity recognition vs structured extraction.

1. The words

A video’s spoken words come from its caption track, written by the creator or generated by speech recognition. Caption segments are short and break mid-sentence, so they are merged into one document first; a sentence split across two segments is whole again before the model sees it.

2. Chunks

A long video runs to tens of thousands of characters. The document is split into windows on sentence boundaries, each sized so the model has the schema, the text and room to answer. Cutting at a sentence end, never mid-name, keeps a product name and its price in the same window. Chunks are processed in parallel, so a long video takes seconds rather than minutes.

3. A specialized extraction model, conditioned on your schema

Each chunk goes to a specialized extraction model with your schema as its template. It is a model trained for this task: given a template and a text, it answers with the template filled in, and nothing else. The column type is part of what the model reads. A verbatim-string column asks for the words exactly as spoken, an enum asks it to choose from your options, and a number asks for a value, not a phrase. The description tells it what the field means.

The model answers in the template’s own JSON structure, and every answer is read back against your schema before it becomes a row: keys map to your column names and each value is checked against its type. Templates with list columns are also decoded under a grammar built from the schema, so the output cannot drift from its shape.

4. Types and grounding

Every value is converted to its column type: “two forty-nine” becomes 249, a date becomes a date, and a value that cannot be one becomes empty rather than wrong. Values the model should have copied from the video are checked against the text it read: a verbatim-string or a number has to appear in the transcript (a spoken number counts), so nothing is invented. What the checks removed is reported in usage.droppedValues.

5. Rows, one per mention

The result is a row for every mention, in the order the video says it, with what was said at that point.

The Aurora X2 appears twice because the reviewer came back to it with a different opinion. Keeping every mention keeps that signal: how often a brand comes up, what it is compared with, and how the tone shifts over a video.

A list of mentions across many videos is the raw material for the questions teams actually ask: which brands a category talks about most, which one is the benchmark, and how each is received. A few lines of pandas turn the rows from a whole batch into share of voice and overall sentiment per brand:

import pandas as pd
 
# rows from every finished video: job["videos"][i]["rows"]
rows = [dict(r, video_id=v["video_id"])
        for v in job["videos"] if v["status"] == "done"
        for r in v["rows"]]
df = pd.DataFrame(rows)
 
# share of voice: how often each brand comes up, and in how many videos
voice = df.groupby("brand").agg(mentions=("video_id", "size"),
                                videos=("video_id", "nunique"))
 
# the overall mood per brand: its most common sentiment
mood = df.groupby("brand")["sentiment"].agg(lambda s: s.mode().iat[0])
 
print(voice.join(mood).sort_values("mentions", ascending=False))

Because each row carries its video, the same table also answers per-channel and over-time questions: join it with the video’s publish date and channel, and count by week. For a full walk-through of collecting videos for a niche, see how to analyze YouTube trends with the API, and for a ready schema for this exact job, the competitor analysis use case.

• Name what you want in plain words. “The price the speaker gives for it” beats “price”.
• Use verbatim-string for names. They come back exactly as spoken and are checked against the transcript.
• Use an enum for labels you will count. The model picks from your options, so the same label means the same thing in every video.
• Use number for anything you will add up. A value that is not said stays empty instead of guessed.
• Make a column a list when one mention has several values. “Every pro the reviewer names” returns an array.

The schema guide covers every type with examples.

TubeExtract runs this pipeline as an API: send up to 200 YouTube URLs with a schema in one request and get typed rows per video back, at 10 credits per started minute of video. Every account starts with 3,000 free credits. Start with the quickstart.