> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cortex-labs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Understand streaming data and retention

> Distinguish temporary source windows, processing coverage and evidence retained for later verification.

A continuing source produces bounded input. Retention determines how long that input remains available. Processing coverage describes what the platform actually analysed.

These boundaries can differ. A source can be collected successfully while enrichment is paused or limited to a shorter processing horizon.

## Three periods to inspect

| Period | Meaning | Check |
| - | - | - |
| Source coverage | Intervals actually collected | Connection gaps, dropped messages, late arrivals and provider limits. |
| Dataset retention | Available rolling source history | Time basis, time limit, byte limit and whole-file expiry. |
| Processing horizon | Inputs eligible for embedding or enrichment | Skipped older inputs and the selected binding's horizon. |

The dataset can use observed time or ingestion time for retention. Observed dates describe source events; ingestion time describes arrival. A missing asserted date uses arrival time.

Files expire as units. One recent record can therefore keep a file containing older records. Retention is not an exact per-record time filter.

## Windows can change

Rows windows group observations by source group and time bucket. Open windows can be visible before their final summary is ready.

Closing a window creates its processable version. Accepted late rows can create a later version. Earlier reports keep their selected version.

A stream writer can record **dropped**, **over\_budget** or **not\_connected** coverage gaps. A gap identifies missing collection, rather than a period with no observed activity.

## Retained evidence needs its inputs

A citation identifies a source version. It does not itself prevent expiry.

Saving a temporary result as retained evidence requires the complete effective input set. That includes context beyond the displayed citations. Protected evidence has its own storage obligation.

If required inputs have expired, record the missing interval. A retained report cannot depend solely on an expiring buffer or an unverified external archive.

World-model mentions also expire with their source and result. They do not create a retention hold.

## Pausing and resuming

Pausing enrichment does not pause ingestion. Resuming catches up over inputs still available under the selected definition.

A processing horizon can skip retained older material. Increasing it requires a full pass to revisit that material; an arrival pass checks new input only.

An expired enrichment result remains completed in processing history. A forced rerun is required to recompute unchanged input. Retain its data before relying on that new result.

Inspect coverage before comparing periods or interpreting missing detections. Use [Run ingestion and inspect processing](/vantage/bring-in-data/run-ingestion) and [Retention, deletion and legal holds](/administration/configuration-and-usage/retention-deletion-holds).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.