The spec drives the decision.
Every data procurement starts with a spec. Modality. Language and dialect. Rights for training versus retrieval. Scale. Quality thresholds. Refresh cadence. If an existing archive meets every line item, the procurement is a license negotiation. If any line item fails, you have a gap. That gap decides whether you commission new collection.
Define the acceptance criteria first.
Write the acceptance test before you evaluate a supplier. A speech model needs word error rate below a threshold on your target dialects. A vision model needs bounding-box precision on your object classes. A text model needs domain perplexity within range. The acceptance criteria are the contract between you and the data. If an archive cannot pass the sample test, it is not a candidate.
When an existing archive is enough.
An archive works when four conditions hold.
First, modality and language coverage match the spec. Hebrew and Arabic speech with documented dialect labels. Domain-specific text with verified source metadata. Video with synchronized audio and frame-level annotations.
Second, rights are documented for training. Not retrieval only. Not research only. The license must permit commercial model training, redistribution of derived weights, and internal evaluation.
Third, quality passes your sample test. You run a stratified sample through your evaluation harness. The results meet acceptance thresholds without re-annotation.
Fourth, refresh cadence supports your retraining schedule. The supplier commits to quarterly or semi-annual drops with change logs. You can diff the new data against the previous version.
If all four hold, you license the archive. You do not pay for collection. You pay for access and ongoing refresh.
When you must commission collection.
Commission when the archive fails any acceptance criterion.
Dialect gaps. The archive has Modern Standard Arabic but your spec requires Levantine, Gulf, and Maghrebi with native speaker verification.
Domain gaps. The archive has general medical text but your spec requires radiology report structure with de-identified PHI tags.
Noise condition gaps. The archive has studio speech but your spec requires in-car, far-field, and street environments with SNR distribution matching your deployment.
Annotation schema gaps. The archive uses coarse labels. Your spec requires fine-grained taxonomy with inter-annotator agreement measured.
Rights gaps. The archive grants retrieval rights only. Your spec requires training rights with indemnification.
Refresh gaps. The archive is static. Your spec requires quarterly refresh with version control.
In each case, the archive is a negative signal. It tells you what the market does not have. That negative signal becomes the brief for custom collection.
Control of spec is the lever.
Custom collection gives you control over the spec. You define:
- Speaker recruitment criteria: age, gender, region, proficiency, device ownership
- Recording protocol: device list, microphone placement, room treatment, script design
- Environment distribution: quiet, babble, traffic, music, reverb with target proportions
- Annotation workflow: guideline version, adjudication rules, QA sampling rate, acceptance threshold
- Delivery format: manifest schema, checksum, split definitions, metadata completeness
You do not get this control with an archive. The supplier made those decisions years ago for a different buyer. You inherit their choices.
Rights clarity reduces downstream risk.
An archive license is a static document. You review it once. You negotiate once. You own the risk if the licensor missed a chain-of-title defect.
Custom collection builds rights into the workflow. Contributor agreements are signed at recording. Consent forms cover training, commercial use, and model distillation. Metadata captures the consent artifact ID per utterance. The rights package travels with the data. Your legal review focuses on the template and the process, not a legacy contract.
Refresh strategy differs by model.
Archives refresh on the supplier’s calendar. You receive a delta. You test the delta against your acceptance criteria. If the delta fails, you negotiate a fix or you skip the refresh.
Custom collection refreshes on your calendar. You define the refresh spec: new speakers, new domains, new noise conditions, new annotation guidelines. You commission the refresh as a new work order. You control the timing and the content.
Hybrid refresh is common. License the archive for breadth. Commission targeted refresh for the gaps that matter to your model.
Cost structure types.
Archive licensing: fixed fee per hour of speech, per token of text, or per minute of video. Predictable. No variable collection cost. You pay for the entire catalog even if you use a subset.
Custom collection: variable cost per accepted unit. Recruitment, recording, QA, project management. You pay only for data that passes acceptance. Milestone payments align cash flow with delivery.
Hybrid: archive license for baseline. Custom collection for gap fill. Total cost is the sum. The trade-off is spec coverage versus unit economics.
Decision checklist.
Use this checklist at project kickoff. Each item is a pass/fail gate.
- Modality and language match spec without transformation.
- Dialect coverage matches spec with native verification.
- Domain coverage matches spec with source metadata.
- Rights grant training, commercial use, and redistribution of derived weights.
- Sample passes acceptance test on your evaluation harness.
- Refresh cadence matches retraining schedule with change logs.
- Annotation schema matches spec with measured agreement.
- Noise and environment distribution matches deployment profile.
- Supplier provides indemnification for rights defects.
- Cost per accepted unit fits budget at required scale.
If every item passes, license the archive. If any item fails, scope the gap for custom collection.
Hybrid approaches are standard.
Most production teams use both. License an archive for /specifications/domain-specific-text-data/ breadth. Commission /specifications/video-training-data/ for the specific interaction scenarios your product encounters. The archive trains the backbone. The custom data tunes the edge cases.
The hybrid model also manages risk. If the custom collection slips, the archive keeps the training pipeline running. If the archive refresh degrades, the custom data anchors quality.
Write the brief.
The brief is the spec plus the acceptance criteria plus the timeline plus the budget envelope. It is the document you send to suppliers. It is the document you use to evaluate proposals. It is the document you sign off before any collection starts.
We evaluate archives against your brief. We qualify suppliers against your brief. We run sample tests against your brief. We deliver data that passes your brief.
Send your brief. We will return a shortlist of qualified archives and a scoped collection plan for the gaps.
Where to go next.
Whichever route you choose, write the spec first: how to write a training data purchase brief. See also video training data and domain-specific text data, or send a data request.
Questions & answers
When is an existing licensed archive sufficient?
When the archive matches your modality, language, and dialect requirements; rights cover training (not just retrieval); quality meets your acceptance criteria; and refresh cadence aligns with model retraining cycles.
What spec control do you lose with off-the-shelf data?
You cannot define collection protocol, device mix, environment distribution, speaker demographics, or annotation schema. You accept the supplier's historical decisions.
How do cost structures differ?
Archives typically use per-hour or per-token licensing with fixed fees. Custom collection adds recruitment, recording, QA, and project management costs, often priced per accepted unit with milestone payments.
Can you combine both approaches?
Yes. Use archives for breadth and baseline coverage. Commission targeted collection for gaps in dialect, domain, noise conditions, or edge cases that the archive cannot satisfy.
Editorial notes: numerical examples are illustrative. Review our editorial and corrections policy.