Natural history illustrations often aim to show life-like flora and fauna. Depictions of birds poised to take flight, fish swimming upstream, and mammals mid-stride are common in 18^th and 19^th century zoology and botany publications. What is lost in these often lavish illustrations is a certain truth about the way many naturalists interacted with their objects of study: many species of plant or animal were first encountered not in the wild, but in the display case, in the form of carefully prepared specimens.

One British naturalist, Captain Thomas Brown (1785-1862), made the practical observation that mounted animal specimens allowed naturalists to “contemplate, without dread, the most destructive and furious quadrupeds, and the most noxious reptiles.” It follows that many dangerous and rare species may not have been studied if not for taxidermy. Similarly, the building up of large and comprehensive natural history collections would not have been possible without a means of preserving specimens for the long term. As early collectors and naturalists quickly realized, proper taxidermy techniques were necessary to ensure that museum and cabinet collections did not degrade and disintegrate over time. Thus, a significant part of natural history is the rather unnatural history of taxidermy and specimen preparation.

Plate III. Brown, Thomas. The Taxidermist’s Manual, or, The Art of Collecting, Preparing and Preserving Objects of Natural History. 1833.  Contributed in BHL by The Thomas Fisher Rare Book Library, University of Toronto.

While elements of modern taxidermy can be traced back to ancient Egyptian embalming practices, taxidermy as a tool for the study of natural history did not come about until the mid-18^th century. French entomologist René Antoine Ferchault de Réaumur’s treatise, ‘Divers means for preserving from corruption dead birds intended to be sent to remote countries, so that they may arrive there in good condition…‘, published in the Transactions of the Royal Society in 1748, is often cited as the first comprehensive discussion of the topic. However, Réaumur proposed techniques for preparing specimens for shipment only, not long-term preservation or display.

More comprehensive techniques developed in the 19^th century into what many referred to as the ‘Art’ of taxidermy, and it was during this time that popular printed manuals on taxidermy proliferated. The Biodiversity Heritage Library currently includes over fifty examples of such manuals, published between 1799 and 1927.

Captain Thomas Brown, better known for his work in the field of conchology (the species of sea-snail Zebina browniania is named after him), published The Taxidermist’s Manual, or, The Art of Collecting, Preparing and Preserving Objects of Natural History in 1833. The work covers in rather gruesome detail the methods for skinning and preparing a range of mammals, fish and birds – including species as large as the elephant – as well as techniques for mounting smaller crustacea and insects. It includes “Mr. Waterton’s method for stuffing birds,” referring to the more famous naturalist Charles Waterton (1782-186), gives recipes for chemical compounds used in preservation, describes two methods for constructing false eyes out of enamel, and lists common prices for some of the supplies required. The manual also gives creative advice for mounting and positioning each species: sloths, for example, should always be mounted in an upright position, as they are seldom seen walking horizontally.

Brown’s general advice to the taxidermist is “copy nature whenever you have it in your power.”

Plate II. Brown, Thomas. The Taxidermist’s Manual, or, The Art of Collecting, Preparing and Preserving Objects of Natural History. 1833. Contributed in BHL by The Thomas Fisher Rare Book Library, University of Toronto.

The text is illustrated by plates cut by Scottish engraver Robert Scott. In stark contrast to the life-like creatures often found in many illustrated natural history volumes, the animals here (including what appears to be a poor domestic cat) are very clearly dead, though they are still idealized. More macabre images may be found on plates four and five, which show the various surgical instruments – scalpels, mallets and pliers – used in the various stages of animal taxidermy. Perhaps the strangest inclusion, however, is a short appendix in which Brown describes a newly discovered French technique for preserving the human body.

Plate VI. Brown, Thomas. The Taxidermist’s Manual, or, The Art of Collecting, Preparing and Preserving Objects of Natural History. 1833.  Digitized by The Thomas Fisher Rare Book Library, University of Toronto.

Popular works like Thomas Brown’s Taxidermist’s manual reveal in morbid detail the unnatural materials and practices that coincided with the study of natural history – the arsenic, wire, stuffing, and false eyes that naturalists employed in their attempts to “avert the process of Time’s destroying hand.”

This post originally appeared on the Smithsonian Libraries blog and has been republished at the permission of the author, Julia Blakely.

Tab. 6. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

As a commemoration of the Imperial collection of shells in Vienna, the printed folio of Testacea Musei Caesarei Vindobonensis of 1780, is splendid. The eighteen engraved plates, carefully colored by hand, render individual specimens in the Habsburgs’ K.K. Hof-naturalien-Cabinet as if pieces of jewelry, casting shadows on a plain background of the thick, hand-made paper. Dedicated to the Empress of Austria, Maria Theresa (1717-1780), this production was also a work of science, as the task of arranging the shells in the Cabinet and describing them for publication was given to one of the leading scientists of the day, Ignaz Edler von Born (1742-1791).

Illustration. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Born, commissioned by Maria Theresa to arrange the entire royal natural history collections, was principally a mineralogist and metallurgist. But he had a great interest in shells and identified several new species. Systematists in malacology (the study of mollusks, including shells) still examine the originals, now in the Natural History Museum, Vienna (Naturhistorisches Museum), originally built to house the overflowing collections of the Habsburgs. The text of Born’s descriptions are also still used. It was first published in a much smaller format in 1778, in both Latin and German, with one plate as the only illustration. G.C. Kronenberg’s “Born’s strombs (Mollusca: Gastropoda),” a recent article that appeared in Annalen des Naturhistorischen Museums in Wien (2007), is one instance of scholarly work citing Born’s analysis.

Vignette. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Also attesting to the importance of Testacea Musei Caesarei Vindobonensis in scientific research, two copies of the folio second edition were once in the Division of Mollusks in the Smithsonian’s National Museum of Natural History, which holds the world’s largest shell collection (link). The books now reside in the Museum’s Joseph F. Cullman 3rd Library of Natural History. Alas, also indicating their scholarly worth, both volumes were rebound for practicality in sturdy library buckram, their original bindings lost (an earlier practice, happily no longer followed).

The second half of the 18^th century was a period when books were produced addressing the lack of illustrations in the enormously influential tenth edition of Carl Linnaeus’ Systema Naturae. In this two-volume work of 1758-1759, the Swedish author (now known more often by his modern name form, Linné) introduced binominal nomenclature for animals, but providing visual representations for publication was time-consuming and quite expensive. The Habsburg commission of Testacea Musei Caesarei Vindobonensis, employing the binominal system and current with Linnaean names and descriptions, filled the gap with essential information. The plates were designed by the artist Franz Fuxeder (1725-1797) and engraved by Karl Schütz (1745-1800) and J. Adam. The vignettes are sometimes signed by “Conti,” Schütz, or Johann Ernst Mansfeld (1739-1796).

Vignette. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Apart from the separately printed plates of carefully displayed shells, there are forty-eight small scenes or portraits interspersed in the text, all of them an absorbing delight of this sumptuous work on conchology. Given the royal funding, individually created vignettes, alluding to the subject at hand, were created. This was a significant expense in the hand-press period. Unlike woodcuts that could go through the printing press with the standing type, intaglio plates had to be separately printed after, but carefully registered with, the pages of text. The scenes portray shells in land- and seascape settings or within elaborate frames with lettered identifications matched up to their descriptions, serving the purpose of a visual footnote.

Venus and Mermen. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

The images are a world of their own. One imagines the artists having fun with their knowledge and creativity. There are representations of cavorting mythological creations; mermen and mermaids make appearances. Two of the sea creatures play the supporting roles in an interesting interpretation of Venus on the Half-Shell, hoisting her aloft. Rather than her flowing hair or hand, a piece of seaweed provides the element of modesty for the nude goddess.

Publisher’s device. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

The engraved publisher’s device on the title page is an allegorical vignette, with the god Hermes (Mercury in Roman mythology) presenting a volume to Athena (Minerva), goddess of wisdom, identified by the sacred (if startled) owl peeking over her shoulder. A putto hefts a book among a stack of other folios. The publisher, whose initials appear in a medallion festooned by a laurel adorning a carved platform, was Johann Paul Krauss (1701-1776), whose firm continued in business after his death. Following the dedication page is a more elaborate piece, with Poseidon sitting on an annoyed-looking porpoise (who served as the god’s messenger), presenting a shell to a female figure (representing the Empress?), who sits enthroned between a caryatid made up of shells and a cabinet of drawers from which puttis play with shells. Behind is an open book, with a plate of shells resembling this title. Visual homages to the Habsburg rulers, current and past, are made in here and throughout the work. Poseidon appears again in one small vignette, this time cruising along on a fierce-looking porpoise. His son, Triton (looking a little rough around the edges) emerges from the waves blowing a conch shell. He was known as the Trumpeter of the Sea.

Poseidon rides the waves. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Illustration of Triton. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

The book also honors the greats in natural history, conveyed in engraved scenes at the beginning of sections of text. The first, in a chapter of Testacea multivalvia, celebrates Pliny, author of the encyclopedic Naturalis Historia. Mermaids and mermen join the tribute to the author, laying seashells at a tomb that bears his name. In the background is the Bay of Naples, with a puffing Mount Vesuvius in the background (Pliny died in AD79 while attempting rescues by ship of those caught by the eruption of the volcano). The second order, Testacea bivalvia, opens with an homage to Ulisse Aldrovandi (1522-1605?), the author of the first important work on shells, De Reliquis Animalibus of 1606. The last, for the third order, Testacea univalvia, is devoted to Linnaeus. There are a few shells scattered about the landscape; guarding the naturalist’s tomb are an eagle and a lion, whose paw rests on an unfortunate turtle. A whale breaches in the background.

Testacea multivalvia section of text, dedicated to Pliny. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Aldrovandi memorial. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Linnaeus’ tribute. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

It is curious that there hasn’t been more published on the literature of shell books. There are legions of fine, detailed bibliographies in several natural history subject areas—birds, botany and horticulture, voyages of exploration, works of Charles Darwin—but little devoted to conchology. S. Peter Dance’s Shell Collecting: an Illustrated History(London, 1966) and a rare book dealer’s catalog by Antiquariaat Junk of the Netherlands, are two of the few resources.

This lack of scholarly studies in conchology publishing is remarkable, given that shells have been collected since the dawn of humankind, either for utilitarian purposes or for their beauty. In the Age of Discovery, European voyagers found an eager market for specimens collected in the South Seas. That the Empress of Austria commanded a publication on shells is an expression and record of her interest in Imperial trade and expansion to far-flung locales as well as a contribution to the science of natural history.

Tab. 15. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

Past Smithsonian curators, Isaac Lea (1792-1886), conchologist and geologist, and William Healey Dall (1845-1927), a renowned malacology paleontologist, were avid book collectors. The Cullman Library holds many books from their former collections and would be an excellent place to start compiling a bibliography. Dall’s copies of Filippo Buonanni’s Recreatio Mentis, et Oculi (Rome, 1684) and Thomas Martyn’s The Universal Conchologist (London, 1784) would be highlights; Born’s Testacea Musei Caesarei Vindobonensis another, once owned both by Lea and Dall. Many of these titles have already been digitized by the Biodiversity Heritage Library, available for exploring the imaginative worlds found in books on shells.

Tab. 7. Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Smithsonian Libraries.

All images in this post are from the Cullman’s copy of Testacea Musei Caesarei Vindobonensis, which has been scanned for the Biodiversity Heritage Library. For comparison, below you can see a copy of the above illustration (Tab.7) from the copy held by Naturalis Biodiversity Center, also digitized in BHL, which has, unusually, uncolored plates.

Tab. 7 (uncolored). Born, Ignaz, Edler von. Testacea Musei Caesarei Vindobonensis. 1780. Contributed in BHL from Naturalis Biodiversity Center.

“The Fig Tree”. Blackwell, Elizabeth. A Curious Herbal. v. 1 (1737). Contributed in BHL from the Peter H. Raven Library of the Missouri Botanical Garden.

In 1737, Elizabeth Blackwell published the first weekly installment of a very ambitious project. The final work, entitled A Curious Herbal (1737-39), ultimately consisted of 500 plates of plants alongside 111 pages of text providing descriptions of their medicinal uses. Endorsed by the Royal College of Physicians, the publication helped satisfy the need for an up-to-date reference work for apothecaries.

The motivation to produce this remarkable herbal arose from equally remarkable circumstances.

The daughter of a successful merchant, Elizabeth married Alexander Blackwell at the age of 28. Her husband had a long history of operating outside of the law, and his shady business dealings resulted in huge debts that ultimately landed him in debtor’s prison.

With her husband in prison and a child to support, Elizabeth decided to use her artistic skills to produce a new herbal to support herself and pay her husband’s debts. Using specimens from the Chelsea Physic Garden as models for her illustrations, Elizabeth drew, engraved, and colored all 500 plates herself and used Joseph Miller’s Botanicum Officinale (1722) as a point of reference for the text, while her husband provided the translated names of the plants.

The Herbal’s income secured Alexander’s release from prison. Sadly, he soon accumulated more debts, which required Elizabeth to sell some of the rights to her publication in order to raise additional funds. Ultimately, her husband left his family for Sweden, where he served as a court physician to the Swedish king until his involvement in a political conspiracy resulted in his execution. [1]

Little else is known of Elizabeth’s life, but her herbal is a lasting testament to her artistic skill and entrepreneurial prowess.

A Curious Herbal is the subject of a chapter in Dr. Anna K. Sagal’s first monograph project, Botanical Entanglements: Women, Plants, Literature, and Artwork in the Eighteenth Century, which will be published by the University of Virginia Press in fall 2021.

The book, which has been generously supported by the Newberry Library’s Monticello College Foundation & Audrey Lumsden-Kouvel Fellowship, explores how women — who were always considered amateur naturalists — participated in discourses which were often culturally barred to them. By engaging in literary and artistic projects that intersected with scientific discourse, these women made space for themselves in what was otherwise an overwhelmingly male-dominated conversation. The monograph features several women writers and artists, including Eliza Haywood, Charlotte Lennox, Mary Delany, Maria Edgeworth, Maria Jacson, Priscilla Wakefield, and Henrietta Maria Moriarty, along with Blackwell herself.

By providing free, online access to relevant literature, such as Blackwell’s Herbal, the Biodiversity Heritage Library has been an important resource for Sagal’s research on the project.

“Thanks to BHL, which provides access to texts by little-known women-naturalists and the male-authored texts that influenced them, I can conduct research around the globe without leaving my office,” shares Sagal. “Although I do have access to some research libraries and archives, many botanical texts are held by overseas libraries, which I do not have the funds to travel to.”

Dr. Anna K. Sagal. Image credit: McMaster University

Sagal, a Teaching Fellow at Cornell College, has been studying eighteenth century British literature for eleven years. She first discovered BHL about a year ago, while searching the Internet for botanical texts. Now, when actively working on her monograph project, she uses BHL daily, most often reading titles online, but sometimes downloading select pages for later offline reference.

“I really appreciate the breadth and depth of resources pertaining to eighteenth-century botany,” affirms Sagal. “Much of the content from the Peter H. Raven Library at the Missouri Botanical Garden is very useful to me, and a lot of it is available via the BHL. I also really like that you can download files and images to look at later.”

Many works accessed through BHL beyond Blackwell’s Herbal have informed sections of Sagal’s monograph. According to Sagal, perhaps the most influential title, used in multiple contexts within several chapters of her book, is The Botanical Magazine, or Flower-Garden Displayed, published from 1787-1800. First published by William Curtis, a botanist at Kew Gardens, the name of the publication was changed in his honour to Curtis’s Botanical Magazine following his death in 1799. Still published today by Kew Gardens, it is the world’s longest running botanical magazine illustrated in color, representing two centuries of botanical history and discovery. [3]

“Daphne collina”. The Botanical Magazine, or, Flower-Garden Displayed. v. 11-12 (1797-98). Contributed in BHL from the Peter H. Raven Library of the Missouri Botanical Garden.

The Magazine was also an incredibly important resource to women naturalists of the eighteenth and nineteenth centuries. An affordable work with frequent rates of publication, it was often an excellent way to keep up-to-date with botanical developments, and to learn about the global world of plants from the comfort of home. Especially for women without access to prestigious physic gardens or botanical research centers like Kew, The Botanical Magazine could be a vital conduit to the enchanting world of botany.

“Indian Chrysanthemum”. Moriarty, Henrietta Maria. Fifty Plates of Green-House Plants, Drawn and Coloured from Nature. 1807. Contributed in BHL from the Lenhardt Library of the Chicago Botanic Garden.

The Botanical Magazine was also a crucial resource for Sagal’s research on another subject for her monograph project, Henrietta Maria Moriarty and her Fifty Plates of Green-House Plants, Drawn and Coloured from Nature.

Moriarty’s work was “intended for those who take delight in plants, but have not the advantage of a gardener who understands them”. Along with her fifty illustrations of plants selected for “their beauty, their odour, or some peculiarity in their economy”, Moriarty also included each plant’s Linnaean binomial and brief cultivation information. [2]  

Fifty Plates of Green-House Plants is central to Sagal’s research because of Moriarty’s unique artistic vision and her somewhat complicated legacy. Often accused of copying images from Curtis’s Botanical Magazine, Moriarty has long been dismissed from the history of botanical illustration. However, with the help of the BHL, Sagal has been able to compare plates of Moriarty’s work with the contemporary runs of Curtis’s publication to find several plates which she clearly drew from nature or another source, rather than mimicking the composition of a plate from the Botanical Magazine. This allows modern audiences to revisit the question of Moriarty’s artistic originality and to appreciate her work as a hybrid of both unique and sourced images.

As Sagal completes her research on Botanical Entanglements: Women, Plants, Literature, and Artwork in the Eighteenth Century, we are thrilled to know that BHL has been such an important asset for the project. We look forward to learning more about the role of women in Enlightenment science thanks to Sagal’s work.

[1] The British Library. 2018. “A Curious Herbal.” Collection Items. Accessed on September 27, 2018. https://www.bl.uk/collection-items/a-curious-herbal-dandelion.

[2] Oak Spring Garden Foundation. 2018. “Women Botanical Artists, Part 2.” Google Arts and Culture. Accessed on September 27, 2018. https://artsandculture.google.com/exhibit/oQIi52WKeTK5KA.

[3] Royal Botanic  Gardens, Kew. 2018. “Curtis’s Botanical Magazine.” Accessed on September 27, 2018. https://www.kew.org/science/who-we-are-and-what-we-do/publications/curtiss-botanical-magazine.

This post was originally published on the rOpenSci blog on 28 August 2018 and is republished with permission of the author, Dr. Maëlle Salmon, and rOpenSci.

Armed with rOpenSci’s packages binding powerful C++ libraries and open taxonomy data, how much information can we automatically extract from images? Maybe not much, but, experimenting with gorgeous drawings from a natural history collection, we can least explore image manipulation, optical character recognition (OCR), language detection, and taxonomic name resolution with rOpenSci’s packages.

Free natural history images and appropriate R tooling!

A long time ago I had bookmarked the Flickr account of the Biodiversity Heritage Library (BHL). So many beautiful images of biodiversity, moreover free to use! In particular, I downloaded all pictures from one of the Birds of Australia albums (contributed in BHL by Museums Victoria).

I wanted to try to extract the bird names from images using packages of Jeroen Ooms’, rOpenSci’s post-doc hacker & C(++)-bindings wizard. For that I worked with magick for image manipulation, tesseract for optical character recognition (OCR), cld2/cld3 for language detection… Quite the armory! We’ll also sprinkle some taxonomy magic by Scott Chamberlain, one of rOpenSci’s co-founders, to resolve the names extracted.

OCR bird naming workflow, piece by piece

In this section, we explain the different elements of our R workflow: preparing images, extracting text, resolving taxonomic names.

Image preparation

I saved the pictures locally in a “birds” folder. Yes, I click-buttoned instead of using the Flickr API for which e.g. Jim Hester wrote a minimal R wrapper… I don’t do everything with R scripts (yet).

library("magrittr") 
filenames <- fs::dir_ls("birds")

Each image shows a bird and its name. Images are either landscape- or portrait-oriented, but for the sake of simplicity, I’ll act as if they were all portrait-oriented. A possible easy and lazy fix here would be to duplicate images rotated (magick::image_rotate) in all possible directions and then to apply the workflow to all 4 versions of each image, hoping to get good data from one of the rotated versions.

magick::image_read(filenames[1])

Mathews, Gregory M. The Birds of Australia. v. 11 (1923-24). Art by Henrik Grönvold. Contributed in BHL from Museums Victoria.

From that image, I wanted to extract the name indicated below the bird. To maximize the efficiency of OCR, I shall first prepare the image, since the accuracy of OCR depends on the quality of the input image which can be influenced a bit. This part could be tweaked even more, and in real life examples it’ll be worth spending time trying different magick functions and parameter values. Since I have in mind the case of a bunch of images to be batch-processed, nothing is done by hand.

crop_bird <- function(filename){
  image <- magick::image_read(filename)

  height <- magick::image_info(image)$height

  # crop the top of the image
  image <- magick::image_crop(image, 
                     paste0("+0+",round(0.75*height))) %>%
    # convert the image to black and white
    magick::image_convert(type = "grayscale") %>%
    # increase brightness
    magick::image_modulate(brightness = 120) %>%
    magick::image_enhance() %>%
    magick::image_median() %>%
    magick::image_contrast() 

  # we'll need the filename later
  attr(image, "filename") <- filename

  return(image)
}

crop_bird(filenames[1])

It does look cleaner now!

Text extraction

Now is the actual OCR step! The tesseract package provides bindings to the Tesseract OCR engine, free software currently sponsored by Google. It is a powerful engine, with a ton of parameters. Here again, tweaking a lot is warranted. Particularly useful reads are hocr vignette and this Wiki page of Tesseract about improving the quality of the output. The hocr package might be of interest for post-processing of OCR results.

Below, the only option changed from default is the page segmentation mode choosing 1 for “Automatic page segmentation with Orientation and script detection (OSD)”. When using Latin instead of English training data the results were not as good.

One can use either tesseract::ocr for a file path, url, or raw vector to image, or magick::image_ocr for a magick object which is quite handy in our pipeline.

The function below also filters results of the OCR using language detection. By only keeping text recognized as either Latin or English by one of the two language detection packages cld2 and cld3 that are interfaces to Google compact language detectors 2 and 3, one gets a first quality filter. If not doing that, the output would contain more unusable text.

get_names <- function(image){
  filename <- attr(image, "filename")
  ocr_options <- list(tessedit_pageseg_mode = 1)

  text <- magick::image_ocr(image, options = ocr_options)
  text <- stringr::str_split(text, "\n", simplify = TRUE)
  text <- stringr::str_remove_all(text, "[0-9]")
  text <- stringr::str_remove_all(text, "[:punct:]")
  text <- trimws(text)
  text <- stringr::str_remove_all(text, "~")
  text <- text[text != ""]
  text <- tolower(text)

  # remove one letter words
  # https://stackoverflow.com/questions/31203843/r-find-and-remove-all-one-to-two-letter-words
  text <- stringr::str_remove_all(text, " *\\b[[:alpha:]]{1,2}\\b *")
  text <- text[text != ""]

  # keep only the words that are recognized as either Latin
  # or English by cld2 or cld3
  if(length(text) > 0){
    results <- tibble::tibble(text = text,
                 cld2 = cld2::detect_language(text),
                 cld3 = cld3::detect_language(text),
                 filename = filename)

  results[results $cld2 %in% c("la", "en") |
          results$cld3 %in% c("la", "en"),]
  }else{
    return(NULL)
  }


}

(results1 <- filenames[1] %>%
  magick::image_read() %>%
  get_names())

## NULL

Nothing! Now if we replace magick::image_read with the previously defined crop_bird function that crops and cleans the image…

(results2 <- filenames[1] %>%
  crop_bird() %>%
  get_names())

## # A tibble: 2 x 4
##   text                 cld2  cld3  filename                          
##   <chr>                <chr> <chr> <chr>                             
## 1 climacteris picumnus <NA>  la    birds/n115_w1150_42399797481_o.jpg
## 2 brown tree creeper   en    <NA>  birds/n115_w1150_42399797481_o.jpg

We get a result! So we see that the image transformation was quite useful.

Now, these names look fine, but how to be sure they’re actually taxonomic names?

Taxonomic name resolution

The taxize package by Scott Chamberlain, is a taxonomic toolbelt for R, providing access to many fantastic data sources and tools for taxonomy. One of them, the Global Name Resolver, provides, well, resolution of taxonomic names, sadly not common names. taxize::gnr_resolve has many options, of which only one is used below: best_match_only = TRUE means it’ll only return the best match from the different data sources.

latin <- results2$text[results2$cld2 == "la"|
                         results2$cld3 == "la"]
taxize::gnr_resolve(latin,
  best_match_only = TRUE)

## # A tibble: 1 x 5
##   user_supplied_name submitted_name  matched_name   data_source_tit~ score
## * <chr>              <chr>           <chr>          <chr>            <dbl>
## 1 climacteris picum~ Climacteris pi~ Climacteris p~ NCBI             0.988

English names could be cleaned up a bit using the spelling package, also an rOpenSci package, since spelling::spell_check_text would output possible typos.

OCR bird naming workflow in action!

First the two steps image processing and OCR are applied to all images.

bird_names <- purrr::map(filenames, crop_bird) %>%
  purrr::map_df(get_names)

Out of 51 images only 17 are present in the final table with possible names which is a bit disheartening, but one could surely do better in the image processing and OCR tweaking steps! Maybe one could frame the parameter search as a machine learning problem. Please also keep in mind that such natural history images are quite hard to parse.

The name resolution information can be added to this table.

safe_resolve <- function(text){

  results <- taxize::gnr_resolve(text,
                                 best_match_only = TRUE)

  if(nrow(results) == 0){
    list(NULL)
  }else{
    list(results)
  }
}

bird_names <- dplyr::group_by(bird_names, text) %>%
  dplyr::mutate(gnr = ifelse(cld2 == "la" | cld3 == "la",
                             safe_resolve(text),
                             list(NULL)))

We do not get much resolution, but we knew the names weren’t very good to start with. A better (untested here!) idea here might be to get a full list of names of Australian birds, potentially leveraging the taxizedb package by Scott Chamberlain, and to then fuzzy-match them with the names we have.

unique(bird_names$gnr)

## [[1]]
## # A tibble: 1 x 5
##   user_supplied_name submitted_name  matched_name   data_source_tit~ score
## * <chr>              <chr>           <chr>          <chr>            <dbl>
## 1 climacteris picum~ Climacteris pi~ Climacteris p~ NCBI             0.988
## 
## [[2]]
## [1] NA
## 
## [[3]]
## NULL
## 
## [[4]]
## # A tibble: 1 x 5
##   user_supplied_na~ submitted_name  matched_name  data_source_title  score
## * <chr>             <chr>           <chr>         <chr>              <dbl>
## 1 austrodicaeum ii~ Austrodicaeum ~ Austrodicaeu~ The Interim Regis~  0.75
## 
## [[5]]
## # A tibble: 1 x 5
##   user_supplied_name submitted_name  matched_name   data_source_tit~ score
## * <chr>              <chr>           <chr>          <chr>            <dbl>
## 1 melithreptus laet~ Melithreptus l~ Melithreptus ~ CU*STAR          0.988
## 
## [[6]]
## # A tibble: 1 x 5
##   user_supplied_na~ submitted_name matched_name  data_source_title   score
## * <chr>             <chr>          <chr>         <chr>               <dbl>
## 1 rad isdlvorniode  Rad isdlvorni~ Rad Baker & ~ The Interim Regist~  0.75

Conclusion

rOpenSci packages supporting this (and your) workflow

In this post, we made use of R packages quite useful to wrangle information from diverse formats:

• magick for image manipulation,
• tesseract for optical character recognition (OCR),
• cld2/cld3 for language detection.

We also used a function from taxize allowing us to use the Global Name Resolver. Discover more packages from our suite here.

Applicability of this OCR bird naming workflow

Actually, the BHL itself provides OCR output for its collection, see this example. I wasn’t able to find information about the software powering this OCR. What I was able to find out is that the BHL uses purposeful gaming in its OCR workflow. The raw OCR results aren’t much better than what we got in this post which is comforting.

More data from the Biodiversity Heritage Library

If you’re interested in other types of data from the BHL, in addition to the images, have a look at the rbhl package, part of rOpenSci’s suite, that interacts with the BHL API. One can e.g. search all books by the same author as the one we used images from.

author <- rbhl::bhl_authorsearch("Gregory M Mathews")
books <- rbhl::bhl_getauthortitles(creatorid = author$CreatorID)
head(books$FullTitle)

## [1] "A manual of the birds of Australia,"                                                                                                                                                                                                                                          
## [2] "A list of the birds of the Phillipian sub-region : which do not occur in Australia. "                                                                                                                                                                                         
## [3] "A manual of the birds of Australia /"                                                                                                                                                                                                                                         
## [4] "A list of the birds of Australia : containing the names and synonyms connected with each genus, species, and subspecies of birds found in Australia, at present known to the author /"                                                                                        
## [5] "Austral avian record; a scientific journal devoted primarily to the study of the Australian avifauna."                                                                                                                                                                        
## [6] "Arcana, or, The museum of natural history : containing the most recent discovered objects : embellished with coloured plates, and corresponding descriptions : with extracts relating to animals, and remarks of celebrated travellers; combining a general survey of nature."

Or we could get all books whose title contains the words “birds” and “australia”.

australia_birds <- rbhl::bhl_booksearch(title = "birds Australia")
head(australia_birds$FullTitle)

## [1] "Handbook to the birds of Australia. : [Supplementary material in Charles Darwin's copy]."
## [2] "An introduction to The birds of Australia /"                                             
## [3] "The useful birds of southern Australia : with notes on other birds /"                    
## [4] "The Birds of Australia"                                                                  
## [5] "The birds of Australia,"                                                                 
## [6] "The birds of Australia,"

And to get the OCR results of the pages of the book we used, we could write:

library("magrittr")

# ocr=TRUE to extract OCR for all pages
rbhl::bhl_getitempages("250938", ocr = TRUE) %>%
  # for each page transform the type into a string
  dplyr::group_by(PageUrl) %>%
  dplyr::mutate(page_type = toString(PageTypes[[1]])) %>%
  # keep only the illustration pages
  # that are like the ones we used 
  dplyr::filter(page_type == "Illustration") %>%
  # from the data.frame extract the OCR
  dplyr::pull(OcrText) %>%
  head()

## [1] "491 \nFAL CUNCULUS LEUCOGASTER. \n( WHITE -BELLIED £ If BIKE - TIT) \nFALCUNCULUS FRONTATUS. \nSHRIKE - TIT). \n"                                                                                                                   
## [2] "492 \nA** \nOREOICA GUTTURALIS. \n(CRESTED BELL-BIRD). \n"                                                                                                                                                                          
## [3] "APHELOCEPHALA LEUCOPSIS \n( WHITE FACE ). \n"                                                                                                                                                                                       
## [4] "* \nAPHELOCEPHALA PE CTORALIS. \n(CHE <3 TNUT -BREASTED WHITEFA CEj. \nAPHELOCEPHALA NIGRICINCTA. \n(BE A CK-BAH.DED WHITE FA CEj. \n"                                                                                              
## [5] "H . Gronvold. del. \nWitherLy & C° \nSPHENOSTOMA CRIS TATUM \n(WEDGE BIEL). \n"                                                                                                                                                     
## [6] "49 6 \nH \n(jronvolcl. del. \nN E O SIT TA LE CJ C O CE PHAI.A. \n( WHITE ¦ HE AID EE THE EH UN HE FL). \nNEOSHTA ALBATA \n(F IE E> T Ft E EE UNNEFlj. \nNEOSITTA CHRYSOPTERA \nf OFi. A. NGE - wing-e d tree runner). \nWitWLjA \n"

So there’s quite a lot to explore!

More birding soon!

Follow this series on the rOpenSci blog to explore this topic further! In the meantime, happy birding!