Recipes/Node fetch

Extract video data with Node.js

Extract video data with Node.js: a fetch example that submits a video with a schema, polls within the rate limit and saves the structured rows.

The code

const API = "https://api.tubeextract.dev/v1";
const headers = {
"X-API-Key": process.env.TUBEEXTRACT_KEY,
"Content-Type": "application/json"
};
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
 
const schema = [
{ name: "restaurant_name", type: "verbatim-string",
description: "Name of the restaurant the host visited" },
{ name: "food_ordered", type: "string[]",
description: "What the host said he ate there" }
];
 
const res = await fetch(API, {
method: "POST", headers,
// always a list, even for one video
body: JSON.stringify({ videos: [url], multiple: true, schema })
});
if (!res.ok) throw new Error((await res.json()).error.message);
const { job_id } = await res.json();
 
for (let attempt = 0; attempt < 60; attempt++) {
// every poll counts against the per-minute limit (10 on Free),
// so wait long enough to stay under it
await sleep(8000);
 
const poll = await fetch(`${API}/${job_id}`, { headers });
if (poll.status === 429) {
await sleep(Number(poll.headers.get("Retry-After") ?? 60) * 1000);
continue;
}
if (!poll.ok) throw new Error((await poll.json()).error.message);
 
const job = await poll.json();
// rows stay null until every video has settled, so polling costs nothing
if (job.status !== "queued" && job.status !== "processing") {
await save(job.videos); // handed over once: persist before anything else
break;
}
}

Notes

The save call sits inside the loop deliberately. The rows are handed over once, so writing them has to happen in the same turn as reading them. The loop gives up after 60 polls, about eight minutes, so a stuck job cannot spin forever. On a paid plan, a webhook replaces the loop entirely.

Was this page useful?Tell us what was missing