SUBSCRIBE NOW!

Vol. 16 No. 8 — September 2008

By the way — and hats off to Wikipedia again — the origin of the term comes from a Persian fairy tale entitled the Three Princesses of Serendip via Horace Walpole, the English writer and fourth Earl of Oxford. Apparently Walpole once read the “silly fairy tale,” as he described it in a 1754 letter to Sir Horace Mann, and was charmed by how the three royal ladies kept discovering things they were not seeking, mainly by sagacious observation. So he started calling this kind of occurence serendipity.

Of course, when you get paid for your sagacity, as professional searchers do, you’d better find some way to minimize the need for lucky accidents to produce the best results. You’d better find some way to put serendipity on a leash or teach it to come when you whistle.

Most of us might consider web search engines as cornucopias for serendipitous discoveries. Drawing on the ocean of web content, Google and other web search engines seem to find things we never knew existed. Of course, finding the things we do know exist gets a little trickier. Relevance ranking does its best, but nightmare terrors can arise — 956 of 1,247,854; 14,581 of 1,247,854; 145,902 of 1,247,854; 1,247,850 of 1,247,854.

The need for finding “outliers” permeates many research situations. Sometimes it can affect sales or customer satisfaction. For example, I subscribe to Netflix for renting DVDs. As an experienced — some might say, obsessive — fan of old movies, I would seem to be Netflix’ natural prey, but its recommendation engine has failed almost completely to find me items to order. The problem is that it apparently lacks an “outlier” function. For example, I am a long-standing fan of Westerns and continue to challenge Netflix’s distribution system, as I order every ancient “oater” for which it has probably only one or two copies in the system. But every time it follows up on one of my Western orders with recommendations, the DVDs chosen are always for major Western classics (The Searchers, Shane, Red River, etc.). Who hasn’t seen each of those a dozen or more times? Sheesh!! (You haven’t??!! Run! Do not walk!!) The Netflix system can’t seem to grasp that someone who wants movies starring Hoot Gibson, Johnny Mack Brown, Ken Maynard, Col. Tim McCoy, et al. must have already seen all the movies with John Wayne, Gary Cooper, Randolph Scott, and so forth.

Looking for the outer edges of research, the unexpected, even the unimagined is not just trying to win a research lottery. Finding unusual, provocative research can often give clients the edge in attracting funding, winning contracts or grants, getting published in prestigious sources, making a splash. Searching for the orthodox, the conventional, the established literature is necessary, particularly for avoiding embarrassment at having missed the obvious. However, you don’t lead the field by running away from mistakes.

So how do you find useful outlier research? Sometimes by being traditional, even retro, in your searching. Frankly, the line of thought for this column came to me after working on a couple of Infotoday.com NewsBreaks covering the ProQuest takeover of Dialog [http://newsbreaks.infotoday.com/nbReader.asp?ArticleId=49578, http://newsbreaks.infotoday.com/nbReader.asp?ArticleId=49579]. Doubts emerged from both fans and foes of ProQuest’s new acquisition that the abstracting/indexing and even full-text databases had lost their market and were sinking into inevitable irrelevance. Well, maybe. But those databases and their carriers satisfy the first critical requirement in searching for outlier research: These databases only carry high-quality content.

In searching for the unusual, you must avoid letting the degree of unusualness become a strong selection criterion. What you want to find is something that someone else does not consider unusual. You want to find solid, well-documented, well-conducted research that your client never knew existed. You want to find excellence that doesn’t know how excellent it is, scholars that don’t know your client’s field impinges on their own. And that means you must first restrict your searching to high-quality sources. Should you skip the Google search then? Of course not. But let’s go to Google Scholar, then to respected full text and abstracting and indexing services, and then back to Google Scholar. Why the return? Because one of the key elements in searching serendipitously involves finding new terminology that works in alternative research fields. You may ping-pong back and forth between Google Scholar and the traditionals again and again. I invoke Google Scholar so you can check on the cheap what leads the traditionals can supply.

Many years ago, back in my previous life as an intermediate searcher, back in the days before end-user searching dominated the market, back in the ... Hey! this is getting depressing! Anyway, I had finally gotten a request from the assistant head of a research department that our library had never managed to snag as a client. This was my big chance to show them what we and our machines could do. So I identified all the databases that covered their field, crafted a meticulous search strategy, gathered a bucket of search results, examined each and every result drop by drop, organized the final results into precise categories, boldfaced the key information, and walked up to present it personally to the new client I hoped to add to my fans. Just before I left for the trip to the client, I ran one key search term from the strategy through all the files in DialIndex, which were all the files in Dialog. Wanted to make absolutely sure I hadn’t missed a key file. I hadn’t, naturally, but I did notice that the term appeared all by itself — one hit only — in three files covering areas way outside the client’s field. So, just for fun, I printed those three too.

So how did the client react to my best work ever? Indifferently. She explained that I had found everything she had already found in the journals she already read. The other items were on target, but appeared in second-tier journals she did not read and did not need to monitor closely. Thunk! My spirits sank. I got up to leave, then, remembering that last little maraschino cherry I had hoped to place on the sundae of my search, I gave her the three oddities I’d found. Whoosh! Her face lit up; she smiled from ear to ear; her body language began to shout. “This is amazing! Where did you find this?!” I had snatched victory from the jaws of defeat. The outliers, the research on the edge, that’s what did it.

— bq