This document proposes an agent-driven integration between the Bibliotheca Hertziana Digital Humanities Lab and Seed Hypermedia. It covers the Read&Search digital library, the Heinrich Wölfflin digital edition, and their publication within the Bibliotheca Hertziana space.
Objective
Create a reliable, reviewable, and repeatable pipeline that transforms Hertziana catalogue records, OCR, TEI/XML, scholarly apparatus, and images into structured Seed documents while retaining provenance, stable identifiers, rights information, and links to the canonical sources.
The integration should use two source adapters feeding a shared normalization and publishing pipeline:
flowchart LR
A["Read&Search API"] --> C["Transkribus adapter"]
B["Wölfflin TEI/XML"] --> D["TEI adapter"]
C --> E["Normalized document model"]
D --> E
E --> F["Markdown, metadata, and assets"]
F --> G["Validation and editorial review"]
G --> H["Seed Hypermedia space"]Guiding principles
Preserve the canonical source and its identifiers.
Treat original XML, TEI, and image resources as archival source assets.
Distinguish machine-generated OCR from edited scholarly text.
Preserve intellectual structure rather than reproducing website presentation.
Make every import idempotent: rerunning it must update existing resources rather than create duplicates.
Require editorial review before corpus-wide publication.
Begin with representative pilots instead of immediately importing the complete corpus.
Phase 1 — Scope, ownership, and rights
Before implementation, agree on the intended coverage:
the entire corpus;
selected collections;
catalogue records that link to canonical sources;
complete content only for curated digital editions; or
a staged combination of these approaches.
Confirm the reuse rights for OCR, TEI, scans, photographs, and IIIF images. Document the required attribution, the canonical identifiers, the responsible editorial team, the expected update frequency, and whether Seed is an archival mirror or an independent publication.
Recommended starting point: publish one representative work from each source and evaluate it before approving a larger migration.
Phase 2 — Source inventory
Read&Search and Transkribus
The Read&Search application exposes structured data through public Transkribus services. The integration should inventory:
collection and document identifiers;
title, author, date, language, shelf mark, and hierarchy;
page count and page ordering;
OCR and PAGE XML availability;
image, thumbnail, and IIIF endpoints;
source revision information; and
record-level rights and attribution.
The deployed site currently draws on three Transkribus collections and exposes document, page, PAGE XML, and IIIF data. Because the corpus is large, the first production stage should favor catalogue-level records and selective full-text imports.
Heinrich Wölfflin digital edition
Obtain the original TEI files from the repository or TEI Publisher/eXist deployment instead of scraping the rendered HTML. Inventory:
edition volumes and textual divisions;
headings and page breaks;
people, organizations, places, artworks, and bibliography;
notes, apparatus, editorial interventions, and cross-references;
facsimiles and IIIF manifests;
editorial principles and project-level metadata; and
source identifiers and revisions.
Phase 3 — Seed information architecture
A proposed hierarchy is:
Bibliotheca Hertziana
├── About
├── Collections
│ ├── Read & Search
│ │ ├── Collection
│ │ └── Document
│ └── Digital Scholarly Editions
│ └── Heinrich Wölfflin
│ ├── About the edition
│ ├── Volume
│ └── Chapter or section
├── People
├── Places
├── Artworks
└── BibliographyA scanned work should normally become one Seed document with meaningful sections, stable page markers, and facsimile links—not a separate Seed document for every page.
A scholarly edition should follow its intellectual TEI structure: project, volume, introduction, chapter, section, apparatus, and related entities. Page boundaries should remain addressable without determining the whole document hierarchy.
Phase 4 — Source-to-Seed mapping
Define and approve a mapping specification before implementing the importer.
Source field or structure | Seed representation |
|---|---|
Transkribus |
|
Title | Document name |
Author or editor | Metadata and linked entity document |
Date or date range | Structured metadata |
Collection and source hierarchy | Seed document hierarchy |
PAGE XML text | Markdown paragraphs |
TEI divisions | Headings or child documents |
Page breaks | Stable, addressable page markers |
Notes and apparatus | Footnotes or dedicated apparatus sections |
Named entities | Links to canonical entity documents |
IIIF image or manifest | Embedded media and canonical source reference |
Original XML or TEI | IPFS asset recorded as |
Each imported document should also record:
canonical source URL;
source collection;
language;
rights and attribution;
import timestamp;
source revision or checksum;
importer version;
transcription or editorial status; and
OCR confidence where available.
Phase 5 — Pilot importers
Pilot A: one Transkribus publication
Choose a representative document of approximately 20–50 pages.
Fetch the document metadata.
Fetch each page record and PAGE XML.
Reconstruct reading order, regions, and paragraphs.
Normalize OCR conservatively without silently correcting the source.
Preserve page boundaries and original spelling.
Download original XML and the appropriate image or IIIF manifest.
Publish original source assets to IPFS.
Generate the Seed document body and explicit metadata.
Publish to a temporary review path.
Record the source-to-Seed mapping in the import manifest.
Pilot B: one Wölfflin section
Choose one coherent chapter that contains headings, page breaks, notes, named entities, and at least one image or artwork reference.
Fetch the canonical TEI source.
Parse its structural hierarchy and stable identifiers.
Transform supported elements to Markdown and Seed links.
Preserve notes, apparatus, editorial distinctions, and page markers.
Resolve entities against canonical person, place, artwork, and bibliography records.
Publish the original TEI as the archival source asset.
Generate a review report listing unsupported or lossy transformations.
Publish the result to a temporary review path.
Phase 6 — Quality assurance
Automated validation
Every source identifier is unique.
Page and section ordering is complete.
Internal references resolve.
Image and IIIF resources respond.
Imported text and page counts match the source within defined tolerances.
Every publication carries source, rights, and attribution metadata.
Re-running an unchanged import creates no duplicate documents or unnecessary versions.
Unsupported TEI elements and malformed PAGE XML are reported rather than silently discarded.
Editorial validation
Compare representative passages with the canonical source.
Verify headings, notes, italics, quotations, and page breaks.
Confirm rights and attribution statements.
Check that OCR is visibly distinguished from edited text.
Ask Hertziana scholars to approve the representation of apparatus, entities, and editorial interventions.
Phase 7 — Synchronization and provenance
Maintain an import manifest containing at least:
source system
source collection
source identifier
canonical source URL
Seed URL
source revision or checksum
last imported timestamp
importer version
publication status
error or review statusAn incremental synchronization run should:
Enumerate available source records.
Compare revision data or content hashes.
Regenerate only changed documents.
Publish a new Seed version for genuine changes.
Flag removed or unavailable sources without silently destroying published scholarship.
Produce a machine-readable and human-readable synchronization report.
Phase 8 — Production rollout
After both pilots are approved:
Publish project-level documentation and editorial policies.
Import complete curated scholarly editions.
Import catalogue records for the broader Read&Search corpus.
Add full OCR selectively after performance, usefulness, and storage tests.
Enable scheduled incremental synchronization.
Assign responsibility for error review and editorial approval.
Document operational recovery, importer upgrades, and provenance rules.
Suggested delivery schedule
Week | Deliverable |
|---|---|
1 | Scope, rights, source access, and representative samples agreed |
2 | Seed hierarchy and metadata crosswalk approved |
3 | Transkribus pilot importer and sample publication |
4 | TEI pilot importer and sample scholarly section |
5 | Scholarly review, corrections, and acceptance criteria |
6 | Idempotent synchronization, provenance manifest, and monitoring |
7–8 | First approved production collection published |
Principal risks and mitigations
Scale
Importing every OCR page immediately would generate substantial volume and review work. Begin with catalogue records and selected full-text publications.
Rights
Public visibility does not necessarily grant permission to republish scans or transcriptions. Store explicit rights and attribution on every imported resource.
Semantic loss
HTML scraping would discard TEI apparatus and entity relationships. Import from canonical TEI or PAGE XML wherever possible and report unsupported transformations.
Unstable presentation URLs
Application routes may change and should not substitute for canonical source identifiers. Retain source IDs, checksums, and canonical URLs in the manifest.
OCR quality
Machine transcription may contain significant errors. Label it clearly, retain the original XML, and avoid unreviewed normalization.
Identifier drift
Each Seed document must have a durable source identifier and a single manifest entry so that future runs update the same resource.
First milestone
The first milestone is a two-document proof of concept:
one Transkribus publication with metadata, OCR, page markers, XML source asset, and IIIF links; and
one Wölfflin chapter with TEI structure, notes, entities, apparatus, and source asset.
The two pilots should be reviewed jointly by the technical and scholarly teams. Corpus-wide ingestion should proceed only after the mapping, presentation, provenance, and synchronization behavior have been approved.
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime