After examining more than 250,000 historical letters, researchers have introduced a new, automated approach for virtually unfolding and reading sealed documents, testing it on four letter packets from the Brienne Collection, a scholarly treasure chest of undelivered mail from Renaissance Europe — and even reading a sealed letter from 1697.

The interdisciplinary group was fascinated by letterlocking, a set of techniques for intricately folding letters that helped to protect their contents before the invention of the envelope. In the study, published March 2 in Nature Communications, the authors introduced a virtual unfolding pipeline that does not damage the material evidence of letterlocking. It offers an alternative to present-day archival practices that see specialists regularly cutting open sealed letters to access their contents.

“As conservators, we hope these results will cause others to seek alternative ways of extracting information from sealed historical documents,” said Amanda Ghassaei, a research engineer at Adobe Research at the time of the study and one of its co-first authors.

The team’s breakthrough computational pipeline was co-developed by Holly Jackson, an undergraduate at the Massachusetts Institute of Technology and another co-first author of the study. The researchers have also publicly released their code on Github, enabling others to scrutinize and build on their work.

In their pipeline, they first scanned the letters using X-ray microtomography, an imaging method that provides 3D models of objects at the scale of one millionth of a meter. The image data were then segmented, generating a 3D mesh that was flattened into an incomplete 2D representation of the unfolded document. They used a hybrid mesh propagation approach to fill in the gaps. 

Crucially, they recognized that two points that are near each other in 3D might not actually be near each other in the flattened, 2D object. For example, two points on separate halves of a folded sheet of paper might be close to each other in 3D but unconnected in the actual, unfolded letter. They then used texturing to provide a 2D representation of the virtually unfolded letter. The team reproduced the crease pattern, which is vital to understanding letterlocking, through a comparable series of steps.

Their virtual unfolding approach confirmed earlier categorizations of letterlocking. While this groundbreaking computational work began in 2016, another co-first author, conservator Jana Dambrogio, has been intrigued by the mystery of letterlocking for decades.

“She first noticed evidence of locked letters in an archive of legal documents at the Vatican,” explained Ghassaei. “Before we started working on the Brienne Collection, we had not had the chance to study letters in their unopened state. This postal trunk was a true game-changer in what we thought about letters — and the investigative methods we would have to develop in order to understand them.”

The letter from 1697 was sent by a lawyer in Lille, France, to his cousin in the Hague, the Netherlands. It requests a “legalized excerpt,” a notarized copy of the death register that served as a predecessor to the death certificate, to confirm the death of a relative.

“It is likely that the document was needed to settle an inheritance dispute, which could have been complicated by the fact that the death occurred in a foreign country and thus needed to be substantiated,” added Ghassaei. “The good news was that even foreign documents had legal effect if they were signed by a competent notary, the most important contact for day-to-day civil law transactions.”

“We discovered what might have been expected: an everyday letter from a lawyer in France to his cousin in The Hague,” said co-author Rebekah Ahrendt, a professor at Utrecht University. “But knowing the care with which the lawyer prepared his letter — the way he folded and sealed it — adds an entirely new dimension to the story and makes it much more meaningful. It’s also true that such everyday letters from ‘normal’ people of the past are very rare, making it so much more important to study them from multiple angles. These letters give us unique evidence of the strong relationships that even non-elite people had with paper.”

While the method can be used to reconstruct many old documents, Ghassaei explained that the carbon-based inks we use today are not easily imaged by X-rays.

“The inks imaged with our technique are an older form called ‘iron gall’ — the high iron content in the inks is what makes it show up clearly in the scans,” Ghassaei explained.

According to Ghassaei, the technique could also be used to reverse-engineer origami models made by deceased origami masters.

The authors noted that their method can be further improved, as it is vulnerable to segmentation errors and other distortions. In the Brienne letter packets, many of these challenges stemmed from dense pockets of sealing wax, adhesive, ink and other materials that impede scanning.

“We are in the process of making some improvements to the code so that it can more easily be applied to other projects and to speed it up significantly (from days to hours),” added Ghassaei.

The study, “Unlocking history through automated virtual unfolding of sealed documents imaged by X-ray microtomography,” published March 2 in Nature Communications, was authored by Jana Dambrogio, Holly Jackson, Martin L. Demaine and Erik D. Demaine, Massachusetts Institute of Technology; Amanda Ghassaei, Adobe Research; Daniel Starza Smith, King’s College London; Graham Davis and David Mills, Queen Mary University of London; Rebekah Ahrendt, of Utrecht University, Nadine Akkerman, of Leiden University and David van der Linden, of Radboud University.