How the structured extraction API works
How the structured extraction API works: it reads a video's captions and returns JSON rows matching your schema. How jobs run, and what it is not good at.
What this is
You give us a video and a list of columns, each with a name and a plain-English description of what it holds. We read the video’s caption track and return JSON matching those columns, so every value comes from what is said in the video.
A schema either describes one record per video or many. Ask for many and you get an array, one entry per thing the narration talks about.
Captions only, and why
We read the captions a video already has, manual or auto-generated, and nothing else. There is no speech-to-text and no on-screen text recognition, which is what keeps a job fast and cheap. So the rule is simple: if a video has a caption track, we can read it. If it does not, the job fails and costs you nothing.
no_captions and is never charged, so you can send whole playlists without checking first — see videos without captions.The shape of a job
Every call to POST /v1 is a job, holding one video or up to 200. Each video is queued, a worker picks it up, and once every video has finished you read the result back. A video moves through these states and never goes backwards; a job whose videos ended both ways is partial.
On our own measurements a 26-minute video extracts in about ten seconds, and the whole job, caption fetch included, takes under a minute. The videos in a batch run independently, so one slow video never blocks the rest.
Getting the best results
Three habits that make the difference between good rows and great ones.
A column described as "a restaurant the host eats at" returns the places visited; "a restaurant" returns every one named. The description is what the model follows, so the more precise it is, the cleaner the rows.
Auto-generated captions spell the same name in different ways. Near spellings are merged into one row for you, and a verbatim-string column keeps names exactly as spoken so they match your own records.
When a video covers several subjects, word each column relative to the row ("the price of this product") so every detail lands on the right entry.