HAdW GND-Based Web Services – Beaconizer & Discoverer (Hagrid)

Duration: March 1, 2025–December 31, 2025

The research projects based at the Heidelberg Academy are producing digital data on an ever-increasing scale: As part of this digital work, entities such as personal and place names are increasingly assigned persistent identifiers for the purposes of disambiguation and register creation—typically GNDs for persons and Geonames for toponyms.

From a user’s perspective, this presents two challenges: First, when searching for documents and information—for example, about a person or a place—there are many entry points, since each research project has its own website. Second, these project websites typically allow only a simple text-based search for people and places; corresponding indexes are rudimentary at best, whereas linking entities already uniquely identified by authority data across research institution boundaries or with other global datasets, as well as integration into larger research networks, has not yet taken place.

The Hagrid project aims to enable the active reuse of datasets compiled over many years by developing tailored interfaces to extract data from the respective data silos and make it available to the scientific community through links and Linked Open Data modeling. In this respect, Hagrid serves—much like the character of the same name from the Harry Potter book series, who is described there as a kind of friendly, helpful “Keeper of Keys and Grounds of Hogwarts,” serves as a central entry point and resource for researching authority data across the HAdW research projects.

Hagrid extracts standard data from the research centers’ specific databases and aggregates it into a central standard data database. In the pilot phase, standardized data from two thematically closely related HAdW research projects is used:
1) “Melanchthon Correspondence” and 2) “Correspondence between Theologians in the South-western Part of the Empire in the Early Modern Period (1550-1620)”, both of which are compiling editions of letters from the Reformation era. This data is exported as a CSV file; for this purpose, a generic exchange format (Normdata Interchange Format, NDIF) is being developed, which defines column information and data types.

Definition of the Normdata Interchange Format (NDIF)

This data is periodically updated by research projects, retrieved by Hagrid, and transferred to the central Hagrid authority data aggregator. Various interfaces enable a (partially) automated exchange of information between computer systems, while the Hagrid website allows for user-friendly interaction with the Hagrid database in a web browser.

Hagrid Infrastructure

As part of the Hagrid project, following an API-first approach, the APIs are developed first, and then, based on them, the Hagrid website. The Hagrid APIs are compatible with the OpenAPI standard (external link); documentation for the available interfaces is automatically generated using the Python framework FastAPI (external link).

As an example, let’s take a look at the Discovery API: Based on a GND, this API determines which individuals had some kind of relationship within the letter network—whether as the writer or recipient of a letter, or as a person mentioned in the letter’s text. The interface returns the data in either JSON or GraphML format; the latter can then be seamlessly imported, for example, into the network visualization program Gephi and subjected to further analysis there. This enables federated searches for individuals across research institution boundaries.

GraphML response from the Discovery web service (/api/discovery/100089003)

Visualization of the weighted graph of Georg von Erbach’s letter network in Gephi

The other interfaces assist researchers in matching identifiers already in use, such as Geonames, Wikidata, and Pleiades to corresponding GND identifiers. The publication of Beacon files makes the use of authority data in research projects’ datasets publicly available, which increases the visibility of research findings; Furthermore, the Beaconizer makes legacy data in open data repositories accessible in the form of dynamically generated Beacon files. 

Overall, Hagrid provides a central entry point for precise searches for and work with authority data in research projects at the Heidelberg Academy of Sciences and Humanities.

Grieshaber, F. (2023). GND-Based Authority Data Web Service Suite. Text+ Plenary 2023:
Connecting People and Data, SUB Göttingen. Zenodo. https://doi.org/10.5281/zenodo.10033934.
(Poster presentation as part of the 2nd Text+ Plenary: Connecting People and Data on September 28–29 at the SUB Göttingen.)