Collect it once, or keep it fresh: Datasets

A single API call is a snapshot of one page. Sometimes that is all you want. More often you want the whole set, once, as a file you can open: every hotel in a city, every listing in a category, every event this quarter. And sometimes you want it kept current. Datasets does all three: collect it, keep it, download it.

A call is a moment. A dataset is a record.

A dataset is a saved result. You run any endpoint in the catalog, like you always do, and instead of the JSON scrolling past in your terminal, you keep it: named, stored, and yours to come back to. Same clean, structured response, now with a memory attached.

That alone is useful. The moment data has a home, you can point a dashboard, a sheet, or a teammate at it without anyone re-running anything or wondering whether they are looking at today’s numbers or last week’s.

Every page, once, no code

Open any API in the catalog and choose Get all the data. We follow the site’s pages until there are none left and store every row, while a small ledger counts pages, rows, and credits as they land. When it finishes, the CSV is one click away. You never touch a cursor or write a loop.

The price is the same as calling the API yourself: one credit per page fetched, and the download is free. Before you commit, an estimate tells you roughly how many rows, pages, and credits the whole set will take, and whether your plan would stop it early. If a run does stop early, for a cap or a site error, it says so and keeps what arrived. Nothing is silently cut off.

Or kept fresh, on your schedule

The part that turns a saved result into a living one is the schedule. Give a dataset a refresh cadence and it re-runs itself and keeps the latest, so the thing you saved is never stale. You do not host a cron, stand up a worker, or wire any of it together. You choose how often, and it stays current between visits.

Because every refresh is a real run, you always know how fresh the data is. When it last updated is right there next to it, so “is this current?” stops being a question you have to investigate.

Yours to take anywhere

A record you cannot get out of the building is not much of a record. Every dataset exports on demand, in full: pull the whole thing as JSON to feed a service, JSONL for a pipeline, or CSV to drop straight into a spreadsheet or a notebook. The clean shape you get from the API is the clean shape you export, no reshaping in between. Scripts can fetch the same file with an API key.

Who this is for

Two kinds of people. The one who needs a source once, as a file, and would rather not write a scraper for a Tuesday afternoon: the analyst pulling a market list, the researcher building a corpus, the founder sizing a category. And the one whose relationship to a source is “check on it regularly”: the pricing analyst, the ops person watching for changes, anyone who would rather open a table that is already current than remember to refresh it. It is one of the things people quietly build on us the most.

Pick a source in the catalog, choose Get all the data, and download the result from your dashboard. Set a cadence only if you want it kept up to date.