gramps-web-toolkit¶

gramps-web-toolkit is a set of command-line tools for running AI research agents against a family tree in Gramps Web without letting them write guesses into it. Agents research in parallel and never touch the tree; a verifier checks each finding against its source, a judge decides, and one writer applies the rulings with a backup before each block and a whole-tree check after it. Each tool fixes a failure we hit while running about 30 research rounds over two trees of roughly 10,000 people each.

The toolkit works for genealogy in any country. gwt-spanish-poller and some gwt-fetch sources are Spain-only, and their pages say so.

Get started¶

• Getting started: requirements, install, connecting to a tree, a first run.
• The research pipeline: who researches, who verifies, who decides, who writes.
• The autonomous research loop: the queue, leads, ledger, gate and cache wired into one loop an agent runs on its own.

Tools that read and write the tree¶

• gwt-apply: applies a JSON Lines change list, reads before every write, skips what is already there.
• gwt-check: reads the whole tree and reports orphans, dangling links, broken pointers, likely duplicates, dates that contradict each other and what changed.
• gwt-import-excel: turns a research workbook into a change list for review.
• gwt-leads: reads a tree dump offline and ranks its gaps as queue tasks.
• Archives catalogue: where records are kept in about 80 countries, for gwt-leads --archives builtin:CC.
• gwt-report: how two people are related, in English, Spanish and French, and what is new since the family was last told.
• gwt-geocode: coordinates for the places events use, in any country, as a change list for gwt-apply.
• gwt-run: one change list through backup, dry run, apply, verify and check, in one command that goes on where it stopped.
• gwt-import-dna: turns the DNA match lists a testing site lets you download into a change list for Gramps Web's DNA view.
• gwt-lint: checks a change list against the tree before it is applied: scope, protected and living people, overlap with other lists, weak claims; every rule proves it can fail.
• gwt-import-gedcom: a GEDCOM file as a change list for review, with anyone already in the tree held for a decision; audit, split, and export with living people anonymized.
• gwt-corpus: search everything already downloaded before going back to a site, and cross a book or gazette with the tree.

Research tools¶

• gwt-queue: what to search next, handed out to lanes, with leases and back-off.
• gwt-ledger: the shared "already searched" ledger, so lanes stop repeating negative searches.
• gwt-rank: orders record-search results by how well they fit a person, and never hides a row.
• gwt-fetch: archive images and text without a browser: IIIF, Europeana, and Spanish sources.
• gwt-mh: MyHeritage records through a signed-in browser, on a free or a paid account.
• gwt-read: record search and records on FamilySearch (with your own session token) and PARES, index search on Genae, and a name in the BOE and Gaceta de Madrid, slow and cached by default; and INSEE's French deaths files, read offline.
• gwt-read sources: every site gwt-read reads, its canary controls, and how to add one.
• gwt-rawstore: source images in an S3-compatible bucket, a small manifest in git.
• ocr-bench: scores OCR, HTR and LLM readings of the same pages on the names they recover.
• gwt-spanish-poller: watches Spanish civil-registry requests and the DEHú inbox, read-only.

Safety and pacing¶

• Hard-won lessons: the failures behind the rules, and the rules.
• gwt-gate: one request pace shared by every agent on a machine, which backs off on a block.
• gwt-fscache: an on-disk HTTP cache for rate-limited genealogy APIs.

Development¶

• Development: tests, building this site, and the rules for a change.
• Shared rules and locales: one rule for who may be living, name folding and date reading, and a word table per language.

License¶

gramps-web-toolkit is released under the GPL-3.0-or-later license.