November 2020 was a month for discoveries and rediscoveries. Conservation International’s Trond Larsen announced that a spooky-looking, devil-eyed frog (Oreobates zongoensis) had been rediscovered high in the Bolivian Andes. Previously recorded just once, it was thought to be extinct. Then, Flora and Fauna International’s Frank Momberg announced the rediscovery of a primate, the Popa langur (Trachypithecus popa). Last filmed in 2018 and not seen since, DNA from a 100-year-old specimen stored in London’s Natural History Museum was used to match the one found in fresh droppings to prove that the species lives on in Myanmar’s forests. Meanwhile, Jay Barlow of the Scripps Institution of Oceanography announced that his team might have discovered a new species of beaked whale living off Mexico’s west coast.

Finding rare devil-eyed frogs, long-lost primates, and pointy-snouted whales is both remarkable and photogenic, so it’s no surprise these discoveries made headlines around the world.

It may seem odd to start an article in Online Searcher with discoveries of rare and endangered species in the manifestly offline animal kingdom. But the rare and endangered also occur in the online information kingdom. In November 2020, a new tool, Policy Commons (policycommons.net), was launched to make it easier to track down hard-to-find—and often endangered—policy content.

GREY LITERATURE OF POLICY PUBLICATIONS

Policy touches every aspect of our lives—and, indeed, the lives of frogs, langurs, and whales. Yet most policy publications are not formally published. In the jargon of information professionals, it is published as “grey literature” because it lives beyond the information pale that is managed by publishers, librarians, and bookdealers.

Sure, every year, you’ll find a few thousand new policy articles in academic journals and a few hundred policy books issued by mainstream publishers. Yet the vast bulk of the 100,000-plus new reports, papers, and other items published by the world’s policy experts at international organizations (IGOs), non-governmental organizations (NGOs), and think tanks is self-published, informally, on the websites of these organizations. This makes them hard to find and puts them at risk of being lost.

They’re hard to find because, in common with other grey literature, they lack detailed and standardized metadata. This also makes the articles hard to cite and catalog. Policy content is beyond the pale because it is not routinely indexed alongside other scholarly or professional content in specialized discovery or delivery systems like Ex Libris’ Primo or Google Scholar.

These articles are at risk of being lost because, with the exception of a handful of the large IGOs, policy publishers don’t use persistent identifiers or think about archiving, so link rot is rife.

VITAL, YET OFTEN ENDANGERED

Policy content is vital because it can influence policies and change laws that affect our lives and our planet—sometimes dramatically. Let me share an example.

In 2015, a well-funded advocacy group, Business for Britain, published Change, or Go: How Britain Would Gain Influence and Prosper Outside an Unreformed EU. This 1,030-page report underpinned Michael Gove’s seminal and widely quoted “We hold all the cards” speech, and it was serialized for a week on the front page of The Daily Telegraph, a major U.K. national newspaper. There is little doubt that this report played an important role in winning support for the “leave” side in the 2016 Brexit referendum. Change, or Go’s impact in the Brexit debate means it now has historical value. It is very likely to be needed for research, studies, and Ph.D. theses on Brexit for years to come.

Yet the Business for Britain website, launched in 2013, disappeared in 2017. A copy of the report is hosted on the website of another pro-Brexit organization, Brexit Central, but that website published its final post on Feb. 1, 2020, and, now that Brexit is famously “done,” how long before it, too, disappears?

Business for Britain did not arrange for the report to be distributed by Amazon or other bookselling channels. I wager that there are no copies deposited in many, if any, libraries. [WorldCat lists only the University of Oxford as owning a copy. A Google search indicates that the University of Glasgow also has digital access to the report. —Ed.] Nor did it make preservation arrangements with the likes of Portico: There is no LOCKSS or CLOCKSS for policy content. Like the Bolivian devil-eyed frog, Change, or Go is endangered.

You might think this is an exception, an edge case because it was published by a transitory advocacy group. You’d be wrong.

Exhibit one: In 2020, one of the U.K.’s most respected and long-established think tanks rebuilt its website using a new CMS. Every single link was changed, and redirects were put in place for only the small number of reports it considered important. Worse, any report older than 3 years was judged out-of-date and excluded from the new website.

Exhibit two: Early in 2021, a major international organization will also change over to a new CMS. It too will exclude the bulk of its previously published grey literature because, I’m told, the cost of migrating this content can’t be justified. (I refrain from sharing either organization’s identity because they are not alone: In building Policy Commons, we’ve learned that many IGOs and NGOs clear out older content from their websites—one of many problems we are seeking to tackle.)

This exclusion of older content is part of a deeper disappearance problem. In compiling a directory of more than 16,000 policy organizations for Policy Commons, we discovered hundreds that no longer exist: Their websites—together with the content—are gone.

Take, for example, FRIDE (Foundation for International Relations and Foreign Dialogue), launched in 1999 “to expand ideas on Europe’s role in the international arena.” By 2010, its burgeoning reputation led to an invitation for it to be an official adviser to that year’s G20 Summit. Yet by 2015, when funding difficulties forced its closure, FRIDE’s website—together with its 2,600 publications—simply evaporated.

Like the Popa langur, it seems the long tail of NGOs and think tanks live day-by-day in a hostile environment, at risk of extinction.

By our reckoning, “cleaning” and “closure” result in tens of thousands of policy reports disappearing from websites every year. So, it’s hardly a surprise that, typically, a quarter of the links offered on syllabi and reading lists for policy-oriented courses are broken. Or that librarians share tales of researchers and students wasting hours trying to chase down references to policy reports and papers. They yelp with frustration when yet another link in a journal article, bibliography, or Wikipedia yields a 404-error message.