In November 2024, I embarked on a dream job as the Wikimedian-in-Residence at the Biodiversity Heritage Library (BHL), aiming to enrich Wikimedia’s platforms with BHL’s incredible biodiversity collections. This crucial role was shaped by community-driven recommendations outlined along with the dedicated efforts of the BHL-Wiki Working Group, including key figures such as JJ Dearborn, Jake Orlowitz, Giovanna Fontenelle, and Siobhan Leachman, who continuously support the integration of BHL content with the Wikimedia ecosystem.

Connecting the Dots

One of our first achievements was creating a comprehensive Meta-Wiki portal clearly showcasing the partnership across Wikimedia’s diverse projects and adding rich documentation on the different ways both communities are connected. Throughout the residency, we transparently documented our ongoing activities through detailed status updates, offering insights into our step-by-step progress.

The BHL-Wiki Meta-Wiki portal

Tools and Impact

We developed several exciting tools, including BHL Arena, a fun interactive game highlighting BHL’s visual treasures and collecting insights about community preferences.

To capture and understand the broader impact, we built a dedicated dashboard leveraging Wikimedia’s Commons Impact Metrics API. Remarkably, BHL images attract around 20 million monthly views—vastly surpassing BHL’s own website traffic, which averages an already impressive 1.5 million accesses per month.

Monthly views for BHL content on Wiki (with spike in April motivated by traffic to “dire wolf”)

Structuring Data for Maximum Reach

A key achievement was enhancing metadata through Structured Data on Commons (SDC). Collaboratively, we designed a comprehensive, structured metadata model, supported by detailed workflows using tools like Open Refine to efficiently add rich structured data to thousands of BHL images.

We successfully developed scripts integrating metadata from Flickr, the BHL API, GBIF, as well as Wikimedia Commons community-curated categories. With the scripts, the initial goal was to add structured data to 5000 files by the end of March. The combination of the automated approaches with the tireless work of the BHL-Wiki participants, namely User:Ambrosia10, led us to the current coverage of structured data additions for over 18,000 files!

A Tool for the Future: BHL Image Explorer

One particularly impactful creation was the BHL Image Explorer, an intuitive tool enabling easy discovery and reuse of BHL images on Wikipedia. Developed iteratively with user feedback, it offers robust taxon- and location-based searches drawing from the GBIF API, interactive navigation, and detailed Wikimedia reuse statistics. It is a demonstration of how the structured data can be leveraged to improve navigability of image collections as well as aiding Wikimedia volunteers in extending the reach of illustrations.

Search for plants in the BHL Image Explorer

Looking Ahead

Through these six months, I have grown a passion for the BHL Image collection, and I really want to see them everywhere (e.g. every time I open a new tab). In an age of cheap, AI-generated illustrations, there is something grand in seeing these human-made pieces of scientific art, tales of the biodiversity-loving nature of humankind across the centuries.

Although my formal residency has concluded and new futures for BHL are in preparation, I am confident that the outcomes of this work will continue to flourish. Some of the key contributions poised for lasting impact on the data/metadata for BHL Images include:

1. A robust, adaptable, structured metadata model ready for future use;
2. Efficient scripts for large-scale structured data integration from diverse APIs;
3. Automated workflows transforming Wikimedia Commons categories into structured data; and
4. The accessible and powerful BHL Image Explorer tool, enhancing global access to biodiversity imagery.

These advancements significantly amplify the visibility and accessibility of biodiversity heritage content, inspiring ongoing volunteer engagement and enriching global biodiversity knowledge.

I warmly invite everyone to explore the BHL Image Explorer and actively participate in ongoing efforts listed on BHL’s Meta-Wiki page. Together, we can continue bridging biodiversity’s rich past with Wikimedia’s vibrant, open-access future!

Since the release of our official Call for Support on 11 June 2025, the global response has been inspiring and heartwarming. Institutions and individuals from across the biodiversity, library, and research communities have reached out with offers of support, ideas, and encouragement.

The #BHLTransition is making headway

We’re charting the course to a strong and sustainable future for the Biodiversity Heritage Library. Behind the scenes, many exciting conversations are underway, including offers of technical hosting, funding support, and staffing partnerships. While we’re not yet able to share full details, we’re confident these discussions will translate into confirmed commitments.

Reimagining BHL

This is a rare and powerful opportunity to think boldly about what BHL could become. The Call for Support outlines BHL’s existing roles and services – but not its limits.

As we transition from Smithsonian hosting, we’re seeking ambitious ideas for BHL’s future. 

We welcome proposals that explore: 

• Alternative staffing models, including distributed teams or new hires.
• Aspiring visions for BHL’s technical architecture, including new tools or platforms.
• Shared infrastructure or services hosted across institutions.
• Pioneering approaches to how BHL serves global biodiversity knowledge.

Whether your institution can host a current role, contribute infrastructure, support a new function, or propose a radical new approach – we want to hear from you. 

What’s Underway (and Ahead)?

• We’re continuing to meet with potential hosts and supporters across the globe.
• Our fiscal sponsorship agreement is in its final stages. Once confirmed, it will provide the administrative foundation for BHL’s next phase.
• We’re working closely with Smithsonian leadership to coordinate a smooth transition of administrative and technical functions.
• We’re exploring new development opportunities to reimagine BHL’s technical infrastructure.
• We’re collaborating with partners on new grant applications to help fuel the transition and future innovation. 
• We continue to support our Partners in growing and enhancing BHL content: June saw the addition of 108,584 pages and 6,143 articles to the BHL collection.

Initial expressions of interest to the Call for Support are encouraged by 31 August 2025, but we will continue to welcome offers after this date on a rolling basis.

How You Can Help

📢 Keep sharing the Call for Support: If you haven’t already, please share the Call for Support with your institutions and networks. Every connection helps. 

💬 Get in touch: Have an idea, contact, or offer of support? We’d love to hear from you. Contact: [email protected] 

💚 Show your support: Use the hashtag #ILoveBHL on social media to spread the word.

Thank you to everyone who has shared encouragement, advice, and support. The global BHL community is the heart of this effort. Your energy and vision are helping us navigate a new course for the Biodiversity Heritage Library, ensuring a bright, innovative and sustainable future. 

Stay tuned for more updates.

In the past several weeks, we’ve seen a large, disruptive increase in traffic to the BHL main website. This blog post is meant to summarize the event, the effect on BHL and its servers, our response, and what we have planned should it happen again.

The problems experienced on BHL’s website are not related in any way to the transition away from the Smithsonian. We believe the timing of this activity and the resulting downtime to be purely coincidental.

In early June, the Technical Team observed increased load on the full-text-search server, though it remained within acceptable limits and BHL performance was unaffected. Since this is not unusual for short periods of time and the server continued handling traffic, we weren’t  too concerned.

In the week of 7 June, we started to see that BHL was sometimes slower to respond. It was then that we noticed that traffic to the search server was dramatically higher than normal. A quick analysis of the server logs found that BHL was serving more than 10 times the usual number of searches. And more importantly, none of this activity was appearing in Google Analytics. It was clear that we were dealing with traffic from a bot. The bot wasn’t simply performing searches, it was also loading other parts of the sites, usually calling each part with a different taxonomic name each time.

BHL’s Search Traffic slowly increased in April and May.

What is most surprising is that this was not API traffic. BHL’s API provides rich data to anyone who uses it — data that is meant to be consumed by software. This bot was loading the BHL web pages as a person would using their computer’s web browser. It was getting results that were less computer-friendly than it would if it had used the API!

Unfortunately, the traffic was coming from all over the world and the bot was clearly not identifying itself as GoogleBot, BingBot or even OpenAI’s GPTBot. Slowing it down or blocking it entirely was proving to be a challenge. We activated rate limits on the web server, but the distributed nature of the bot prevented that from being effective. We investigated implementing Cloudflare Turnstile, but configuring that on short notice is a challenge. We thought about slowing down all queries to the server, but we knew that would affect all BHL users. In the end, we found that the bots had a sort of “fingerprint” that was different than that of regular users. We were prepared to start limiting traffic using the fingerprint when…. the traffic stopped. On 19 June, BHL started once again responding with its usual speed.

Looking at the graph of how many times the /search URL is called every day, we see some surprising numbers before traffic dropped on 19 June.

BHL’s search traffic from May to June increased dramatically.

In general, BHL’s web infrastructure is able to easily support the current level of approximately 200k-300k searches per day. It is even possible to support up to about 400,000 searches per day, but that’s reaching the limit of what our infrastructure can handle. This is evidence that the hardware and software we have dedicated to search is very robust and well-suited to a busy site.

We haven’t performed an extensive analysis of the sources of this traffic, but at a glance they do seem to be from a variety of locations around the world. While the traffic may have been centered on certain countries or networks (such as the Google Cloud or others), we have no definitive evidence that it’s any one service.

One note: In the second chart, 6 June and 15 June show a much lower number of searches compared to the days before and after. This is not due to the bots slowing down, but instead were caused by BHL itself performing regular weekly and monthly tasks at the same time the bots were busy. This combination of activities overwhelmed the system and BHL was not responding at all.

When we look at the news, we find that our experience mirrors that of other cultural heritage sites, such as those reported in The Register:

Claburn, Thomas. (17 Jun 2025) Bots are overwhelming websites with their hunger for AI data. The Register.

…and also from smaller organizations such as our colleagues at FromThePage:

Brumfeld, Sara. (20 June 2025) Bot Traffic, AI Training, and Infrastructure Strain. FromThePage Blog.

Knowing we are not alone in this challenge is comforting, but it doesn’t eliminate the challenges. We continue to closely monitor BHL’s website performance and remain committed to keeping the website and API up and running as much as possible while we explore the best ways to mitigate the effects on BHL.