Data freshness
Data freshness is how up to date the results of a search engine or search API are compared with the live web at the moment of the query. It depends on how quickly new pages are discovered, how often known pages are re-crawled and how fast the index is updated. For questions about recent events, it can matter more than the size of the index.
It matters most for news, prices, schedules and anything else that changes faster than an index is rebuilt. Search APIs expose it through publication-date filters, such as published_after in the Keenable Search API, and through the published date they return with each result.
Example
An agent is asked about a court ruling issued this morning. If the newest page the search API returns on the case is from last week, the agent can report only the hearing.
How it works in NEEDLE
NEEDLE’s News vertical tests data freshness directly: every hour it turns the newest items from about 124 curated RSS feeds and Google Trends into queries, and its LLM judge caps the rating of results that give stale coverage of a dated event. See the live NEEDLE results.
FAQ
How do you get recent results from a search API?
To get recent results, set a lower bound on the publication date, such as published_after, instead of relying on the ranking. A point-in-time parameter such as query_time works the other way: it sets an upper bound and hides newer pages.
Does a bigger index mean fresher results?
Not necessarily. Data freshness depends on how quickly an index adds new pages and re-crawls changing ones, so a smaller index updated within minutes can return fresher results than a larger one updated weekly.
Related terms
Written by Keenable.
Published
