— Digital Humanities
Primary Sources,
Acquired Responsibly
A research workstation for acquiring, filtering, preserving and preparing textual corpora from six traditions. It builds on Sefaria’s resumable exporter, adds source-specific adapters, and includes a Unicode-aware launcher for LLM dataset preparation.
The Problem
Scattered Corpora, Fragile Downloads
Many Sources, Many Formats
Jewish, Islamicate, Avestan, Sanskrit, Chinese and Daoist corpora live in different archives with different indexes, APIs and conventions.
Large, Interruptible Jobs
Bulk acquisition fails halfway; without checkpoints, hours of downloading can be lost.
Rights Are Easy to Miss
Licences differ by source and item, and are easily overlooked when text is prepared for model training.
The Solution
One Workstation, Six Traditions
Browse source trees, filter by metadata, select what you need and download it as individual files or one combined UTF-8 corpus.
Sefaria Exporter
Load the current export index, browse a lazy category tree, search, filter and select text records and special files, with a download tab and jobs view.
Source Adapters
Islamicate (OpenITI), Avestan (TITUS), Vedic and Sanskrit (GRETIL), early Chinese (Chinese Text Project), Kanripo KR5 and raw Daozang text.
Advanced Filtering
Filter by path, match mode, extension, size range, source, representation and file kind; apply thematic presets and date limits where supported.
Flexible Downloads
Download selections as individual files or merge them into one UTF-8 text corpus, with a manifest you can export.
Jobs & Resume
Pause, resume and recover after a crash, with checkpoint integrity checks and concurrent download workers.
LLM Dataset Preparation
Plan dataset composition, Unicode handling, tokenizer and objective, and a training platform, with rights safeguards built in.
Architecture
From Index to Corpus
Load the Index
Load or refresh a source index, or load a local metadata file when remote access is restricted.
Browse & Filter
Expand the source tree, search and apply filters, then select the records you need.
Download
Run a resumable job, individually or as one combined UTF-8 corpus.
Prepare & Document
Export a manifest and, if appropriate and permitted, assemble a dataset plan for training.
Rights & Licensing
Licensing is source-specific and nothing here overrides source copyright, licensing, authentication or rate limits. GRETIL and TITUS restrict their downloadable materials to scholarly, non-commercial use, and text rights vary by item at the Chinese Text Project. Loading and downloading the raw Daozang text is supported for research convenience, but training, redistribution and commercial use stay blocked behind a manual rights-review warning until you establish permission.
Transparency
Operating Notes
Browser Constraints
A browser-based tool is subject to CORS and rate limits at each source; the manual explains diagnosis and workarounds.
Credentials Stay Local
Optional API keys and GitHub tokens are never persisted by the application.
Verify Before You Train
Confirm rights for your use case before redistribution or model training.
Who Benefits
Built for Text-Centred Researchers
Religious-Studies Scholars
Assemble comparative corpora across traditions with provenance preserved.
Digital Humanists
Acquire large corpora reliably with resumable, documented jobs.
NLP Researchers
Prepare Unicode-aware datasets while keeping licensing front and centre.
Licensing
Flexible Plans
for Every Scale
Professional
Contact for Pricing
Full platform access for individual practitioners and researchers.
Institution
Contact for Pricing
Multi-seat licensing for teams, departments, and organisations.
Enterprise
Custom
White-label deployment with custom integrations and dedicated support.
Related Platforms
Explore Further
Contact
Ready to Get Started?
Reach out to discuss licensing options, schedule a demonstration, or request a custom integration proposal.
Email: [click to reveal]