How the index is built.
PastTrace indexes institution-backed records through versioned adapters. Automated normalization keeps original identifiers, source links and uncertainty; it is not a claim that every item has been reviewed by a historian.
Ingestion, storage and search
The production application runs on Cloudflare Workers. Cloudflare D1 holds the persistent catalog and full-text index; private R2 storage retains imported source snapshots and provenance. Imports use checkpoints, bounded batches and conservative deduplication. All imported records are available to the persistent metadata search; a build-time catalog remains a fallback for the earlier 32,129 records. See the source register for the imported selection, not an institution’s complete holdings.
Cross-language retrieval
Original titles, descriptions and rights statements are not overwritten. Indexed aliases, country names and shared topic concepts support Arabic, English, Hebrew, Russian, Chinese, Japanese, French, German and Spanish queries. This does not mean every description has been translated into every language. Spelling suggestions and inferred filters help retrieval; visitors can choose literal-word search. The Arabic research directory provides translated guidance while preserving original item titles.
Meaning and intent
Optional meaning retrieval uses the pretrained multilingual E5 small model. Its versioned vectors currently cover the 32,129-record baseline, not the entire expanded catalog. When enabled, the model downloads to the visitor’s device and embeds the query locally. This is retrieval, not generated historical answers or a model trained by PastTrace. Related matches are not proof of a historical claim; missing material can correctly produce no match.
Dates and geography
Source-stated historical dates are kept separately from digitization and ingestion dates. Approximate periods stay approximate; one object is not duplicated to manufacture a record for every year. Country labels may describe a subject or another stated association; they do not automatically establish capture location. An institution’s country is never used to invent the recording location. Unknown places remain unknown. Review the evidence on the item page and the documented place index.
Playback, clips and shot indexing
Internet Archive playback URLs are resolved and cached during catalog processing, with anonymous media checks and access status. A restricted source remains discoverable without an unusable play button. Clip Maker operates on eligible accessible media with a ten-minute selection limit; it does not grant reuse rights. The shot-level V0.1 experiment uses measured FFmpeg scene boundaries and a distinct preview frame. This limited experiment does not imply every film has been segmented or transcribed.
Image inspection, citations and research samples
Supported public source images can be inspected with OpenSeadragon, using IIIF where available. Citation exports use Citation.js to retain original institution links, identifiers, rights and uncertain dates. A bounded offline pilot uses faster-whisper and PySceneDetect on the first 90 seconds of one public film, and Tesseract on one printed document. Machine output is unreviewed, may contain errors and never replaces original metadata. Cached samples are loaded on demand; no transcription job runs per visitor. See the research tools and examples or Arabic guidance.
Source health and rights
Source Health records technical observations separately from rights and access. Rate limits and timeouts do not prove a source has been removed. Publicly accessible material may remain RIGHTS_UNCLEAR; no license or public-domain claim is invented to fill structured-data fields. Health information reflects the last check, not a promise of continuous availability. Read the rights guide and access explanation.
Open data, versions and reproducibility
The catalogue metadata is published as a versioned dataset with a data dictionary that marks every column as verbatim from the institution, normalised by PastTrace, or created by PastTrace. Records can be harvested in CSV or JSON Lines and exported one at a time as BibTeX, RIS, CSL-JSON or JSON-LD; every citation names the holding institution as publisher and adds the date of access. See open data and how to cite. Source values are never overwritten, and a derived value such as a sortable year lives in its own column next to the date text and its precision.
Measured quality
On 31 October 2026 search was measured with objective checks, not human relevance judgements. For 63 known items searched by their own title, the item appeared in the first ten results 85.7% of the time and in first place 50.8% of the time (mean reciprocal rank 0.625). Every filter tested returned only matching records, and the same queries written in five other languages returned the same results as in English. An automated accessibility audit (axe-core, WCAG 2.2 A and AA rules) of 64 page loads in English and Arabic, at desktop and phone widths, found no violations; testing with screen readers by hand has not been done yet.
Two data problems were found and dealt with on 1 November 2026. About 1,250 older records carried the year 1900 as a stand-in for an unknown date; they now read “date not stated” and the dataset leaves their year empty. One harvested source, Internet Archive “Arabic & Regional Film” (800 uploads), mixed a few historical films with propaganda, pirated broadcasts and mislabelled files, so it was withdrawn from the catalogue until each item can be reviewed by hand.
Verification and corrections
Consult the original institution before publishing factual claims or reusing media. Our verification checklist and citation guide explain a repeatable research process. Report metadata errors or takedown requests through corrections and takedowns or founder@pasttrace.com. PastTrace is independently maintained and is not endorsed by the institutions it indexes.
