Guides/Writing a schema

Writing a schema

The schema is the whole interface. Getting the descriptions right is the difference between usable rows and empty ones.

Columns

A schema is a list of columns. Each has a name, which becomes the JSON key, and a description, which is what the extraction actually reads.

{
"multiple": true,
"columns": [
{
"name": "restaurant_name",
"description": "Name of the restaurant, diner or food spot the host visited"
},
{
"name": "food_ordered",
"description": "What food or dish the host said he ate or ordered there"
}
]
}

Descriptions do the work

Write the description as you would explain the column to a person who has not seen the video. Name the thing, then say how it is likely to be spoken about. A bare description costs a great deal of accuracy; in our own testing, going from terse labels to full sentences moved field accuracy from roughly a third to most of the way.

Weak

"price"

Better

"The price the host says the item cost, including the currency if they mention it"

For a field with a loose, general name — a category rather than a specific fact — the same rule needs one more piece: say what should not match, since the video will usually mention other things in a similar way. This matters most for repeated-row fields, where a vague description gets tested against the whole video rather than one clear spot.

Weak

"gas stations mentioned"

Better

"The name of a gas station or convenience store brand the host visits — not snacks, drinks, or fuel types mentioned along the way, just the store's own name"

Column types

A column can also declare a type. It changes two things: what the extraction is allowed to answer with, and what you get back in the JSON. Leave it off and the first column is verbatim-string and the rest are string, which is what most schemas want.

These are NuExtract 3.0's own type names, not ours. The extraction model reads a typed template, and the names you write here are the names that go into it — so what you set and what the model is told are the same word.

verbatim-string

Copied from the transcript word for word. The default for the first column, and what a name, a title or a quote wants.

string

Answered in the model's own words. The default for every column after the first — summaries, explanations, anything paraphrased.

number / integer

Comes back as a number rather than a string. A figure the video never says is left null.

boolean

true or false, for a yes-or-no question about the video.

date

A date the narration mentions.

enum

The answer has to be one of the options you list. Use it for a fixed vocabulary — a sentiment, a verdict, a category — so rows stay comparable across videos.

{
"multiple": true,
"max_rows": 20,
"columns": [
{
"name": "product",
"description": "Name of the product being reviewed",
"type": "verbatim-string"
},
{
"name": "verdict",
"description": "What the host concluded about it",
"type": "string"
},
{
"name": "price",
"description": "Price the host says it costs",
"type": "number"
},
{
"name": "recommendation",
"description": "Whether the host tells viewers to buy it",
"type": "enum",
"options": [
"buy",
"wait",
"skip"
]
}
]
}

verbatim-string and the number types are checked against the transcript: a value that does not appear there is dropped rather than returned. That is the point of them — it is how a name or a figure comes back as something the video actually said, instead of something that sounds right.

Sending a NuExtract template instead

Those columns compile to a NuExtract 3.0 template, and that template is what the model is given. If you already work in that format you can send it directly instead of writing columns — with a descriptions object alongside it, because a template carries types and not meanings. Both forms are stored the same way and run the same job.

{
"schema": {
"template": {
"restaurants": [
{
"restaurant_name": "verbatim-string",
"food_ordered": "string"
}
]
},
"descriptions": {
"restaurant_name": "Name of the restaurant, diner or food spot the host visited",
"food_ordered": "What food or dish the host said he ate or ordered there"
}
}
}
NOTE
A description per field is required in both forms. It is the instructions block the model reads, and a field without one is the single largest accuracy loss available here. Nested objects and nested lists are refused rather than flattened — results are merged row by row across the video, and a nested row cannot be merged.

The schema editor shows the compiled template live while you type, and will take a pasted one and turn it back into columns.

One row or many

Leave multiple off and you get one flat object describing the video as a whole. Set it to true and you get an array, one record per subject the narration covers. Use it when the video walks through several of something and each needs its own row.

NOTE
A saved schema is versioned, and a job records the version it ran. Editing publishes a new version rather than changing the old one — otherwise a finished job would start describing work it never did.
NOTE
When the same real name is spelled two different ways in one video — a caption typo, usually — a repeated-row field already collapses that into a single entry rather than reporting it twice. You do not need to account for that in the description. Two genuinely different names stay two rows, even when they are similar: a video that visits both Lucy's and Mel's gets both.

A wide ask on a long video can return more rows than you want — "every reaction the host gives" can be a row a sentence. Set max_rows alongside multiple to keep the first rows found and drop the rest.

Pitfalls

Asking for what is shown, not said

Anything only visible on screen is invisible to us. If the narration never says it, no description will retrieve it.

Too many columns at once

Six or seven columns per schema is a comfortable ceiling. Beyond that, fields start borrowing from each other. Run two narrower schemas instead.

Several values crammed into one field

With a single-column array schema we split comma-separated runs back apart automatically. With two or more columns we cannot, because there is no reliable way to know which piece of one column matches which piece of another.

Broad categories pull in near matches

A field described only as "gas stations mentioned" also catches snack brands, fuel types, and other nearby nouns the narration happens to use the same way. Say what does not count, not just what does — one short sentence naming the kind of thing to exclude is usually enough to fix it.

Piling on qualifications makes it worse, not better

A description is not something to keep adding clauses to. Past a certain point, in our own testing, a long list of definitions and exceptions crowded out the instruction that actually mattered and the field got messier, not cleaner. State the one or two things that matter and stop.

Was this page useful?Tell us what was missing