# gramps-web-toolkit ![gramps-web-toolkit banner]() [![CI]()]() [![License: GPL-3.0-or-later]()]() --- **gramps-web-toolkit** is a set of command-line tools for running AI research agents against a family tree in [Gramps Web]() without letting them write guesses into it. Agents research in parallel and never touch the tree; a verifier checks each finding against its source, a judge decides, and one writer applies the rulings with a backup before each block and a whole-tree check after it. Each tool fixes a failure we hit while running about 30 research rounds over two trees of roughly 10,000 people each. The toolkit works for genealogy in any country. `gwt-spanish-poller` and some `gwt-fetch` sources are Spain-only, and their pages say so. - **[Getting started]()** --- What each tool needs, how to install it, and a first dry run against your tree. - **[The research pipeline]()** --- How agents, verifiers, a judge and a writer share one tree. - **[gwt-apply]()** --- Applies a change list, reads before every write, and stays dry-run until you pass `--apply`. - **[Hard-won lessons]()** --- Every failure that cost us time, data or an account, and the rule that followed. ## Get started - [Getting started](): requirements, install, connecting to a tree, a first run. - [The research pipeline](): who researches, who verifies, who decides, who writes. - [The autonomous research loop](): the queue, leads, ledger, gate and cache wired into one loop an agent runs on its own. ## Tools that read and write the tree - [gwt-apply](): applies a JSON Lines change list, reads before every write, skips what is already there. - [gwt-check](): reads the whole tree and reports orphans, dangling links, broken pointers, likely duplicates, dates that contradict each other and what changed. - [gwt-import-excel](): turns a research workbook into a change list for review. - [gwt-leads](): reads a tree dump offline and ranks its gaps as queue tasks. - [Archives catalogue](): where records are kept in about 80 countries, for `gwt-leads --archives builtin:CC`. - [gwt-report](): how two people are related, in English, Spanish and French, and what is new since the family was last told. - [gwt-geocode](): coordinates for the places events use, in any country, as a change list for gwt-apply. - [gwt-run](): one change list through backup, dry run, apply, verify and check, in one command that goes on where it stopped. - [gwt-import-dna](): turns the DNA match lists a testing site lets you download into a change list for Gramps Web's DNA view. - [gwt-lint](): checks a change list against the tree before it is applied: scope, protected and living people, overlap with other lists, weak claims; every rule proves it can fail. - [gwt-import-gedcom](): a GEDCOM file as a change list for review, with anyone already in the tree held for a decision; audit, split, and export with living people anonymized. - [gwt-corpus](): search everything already downloaded before going back to a site, and cross a book or gazette with the tree. ## Research tools - [gwt-queue](): what to search next, handed out to lanes, with leases and back-off. - [gwt-ledger](): the shared "already searched" ledger, so lanes stop repeating negative searches. - [gwt-rank](): orders record-search results by how well they fit a person, and never hides a row. - [gwt-fetch](): archive images and text without a browser: IIIF, Europeana, and Spanish sources. - [gwt-mh](): MyHeritage records through a signed-in browser, on a free or a paid account. - [gwt-read](): record search and records on FamilySearch (with your own session token) and PARES, index search on Genae, and a name in the BOE and Gaceta de Madrid, slow and cached by default; and INSEE's French deaths files, read offline. - [gwt-read sources](): every site gwt-read reads, its canary controls, and how to add one. - [gwt-rawstore](): source images in an S3-compatible bucket, a small manifest in git. - [ocr-bench](): scores OCR, HTR and LLM readings of the same pages on the names they recover. - [gwt-spanish-poller](): watches Spanish civil-registry requests and the DEHú inbox, read-only. ## Safety and pacing - [Hard-won lessons](): the failures behind the rules, and the rules. - [gwt-gate](): one request pace shared by every agent on a machine, which backs off on a block. - [gwt-fscache](): an on-disk HTTP cache for rate-limited genealogy APIs. ## Development - [Development](): tests, building this site, and the rules for a change. - [Shared rules and locales](): one rule for who may be living, name folding and date reading, and a word table per language. ## License gramps-web-toolkit is released under the [GPL-3.0-or-later]() license.