The hidden layers: apparatus, morphology, and the wordlists in the attic
6 October 2026 · Nabu news
Most phases grow the library outward. This one grew it inward: almost everything below was already sitting in the fetched trees, unread.
The Pali canon gets its apparatus. SuttaCentral’s bilara-data has always shipped three sidecar trees beside the root texts, keyed to the very same segment ids — and the library had never read them. Now every root passage of the Tipiṭaka carries what its segment attests: 19,873 variant readings (“anabhiratisaññā → anabhiratasaññā (bj)”), 142,865 concordance strings mapping segments to PTS volume-and-page, VRI, the Burmese/Thai/Sinhala print editions and Taishō lines — the crosswalk that lets PTS-citing scholarship resolve into held passages — and some eleven thousand translator notes from the English translations, riding the root segments they explain. No new fetch was needed; the data was in the clone all along.
Middle High and Middle Low German grow up. The two German reference corpora (ReM and ReN, 406 and 235 texts, together some 650,000 passages) parse a TEI export whose upstream deposits carry a richer sibling: the project’s own CorA-XML layer. The fetch now brings both (CC BY 4.0 deposits, checksum-pinned), and with them: full part-of-speech and inflection morphology on all 2.58 million Middle High German tokens; real citation-form lemmas — the TEI attribute turns out to carry normalized word forms, while the true dictionary forms (“wër(e)lt”, “dër”, each with its Middle High German dictionary id) live only in the CorA layer, and the lemma lane now serves them; and dating for both corpora, which had stood as “undatable” — 396 of 406 ReM texts and all 235 ReN texts now carry their century-grid or year dates, with scriptoria and dialect areas where the headers name them.
The Hebrew Bible, in full detail. The ETCBC’s BHSA database ships about ninety word-level features; the library rode nine. It now rides every word-grain scalar the dataset publishes — 44 newly carried, from the pointed morpheme segmentation to lexical statistics — each under its upstream name, with every deliberate omission documented in the adapter rather than silently dropped. And the Dead Sea Scrolls corpus gains its ETCBC-harmonized layer, including the lexeme crosswalk: 422,325 of 422,424 scroll words — 100.0% — now resolve to their Hebrew Bible lexeme, so a form in the Temple Scroll joins hands with its biblical counterpart by plain equality.
The wordlists in the attic. The Tower of Babel’s Indo-European package has carried a LEXSTAT directory since 2005: thirteen lexicostatistical tables — ten Swadesh-grain wordlists (Baltic, Celtic, Dardic, Germanic, Indo-Aryan, Iranian, Romance, Slavic, a mixed list, and the Proto-Indo-European union table) whose every form carries a cognation number chaining into the etymological databases, plus three Indo-Iranian etymology tables. Fetched long ago, parsed never — until now: 24,522 entries across thirteen new shelves, credited to the package’s compilers, S. L. Nikolayev and S. A. Starostin. The wider licence ground firmed up the same week: George Starostin confirmed by email (2026-10-01) that all remaining downloadable StarlingDB databases fall under the original grant’s terms, with the works-in-progress and the Nostratic base explicitly reserved by their authors.
The Diet Library crawl continues. The National Diet Library shelf advanced to 8,789 books of its 97,731-item census, at the committed gentle pace, refusals ledgered and never retried.
Two desks open next. Jost Gippert extended the TITUS personal-use arrangement a second time (by email, 2026-10-06): the Tocharian A and B corpora from the Berlin Turfan collection — the only complete digital text set of the manuscripts — and the Middle Iranian families: the Pahlavi books, the Manichaean Middle Persian and Parthian texts, Sogdian, Khotanese and Bactrian. The terms are the ones three grants have now shared: one retrieval per corpus, local use only, nothing redistributed, TITUS and the editors credited wherever a passage is displayed. The harvest is queued; Iron Age Anatolia’s eastern neighbors are on their way.
The numbers, as of this gate: 2,314,941 documents · 107,833,142 passages · 174 live sources.