Open Library Blog

Or, not so wild actually, it’s the Library of Congress!

We were thrilled to see our BookReader on the read.gov site today. The Library is using it to showcase of some gorgeous books from their Rare Book Collection, like “A Wonder-Book for Girls & Boys,” “The Baby’s Own Aesop,” and “A Christmas Carol.”

You might also be interested to follow along with a “book in progress” called The Exquisite Corpse Adventure, “an episodic progressive story game” with more than 20 contributors.

There’s information about the BookReader software on the Open Library site if you’re code-y too. We love it when the BookReader gets used!

We’re planning for a scheduled downtime on Wednesday, October 28 for hardware upgrade of Open Library servers. Open Library will be unavailable for 2 hours during 7:00 AM PST - 9:00 AM PST.

We’ll post here when the site’s back online.

Update at 8:36am: And we’re back!

I was listening to the radio over my morning coffee this morning when I heard that today is Mohandas Karamchand Gandhi’s birthday. I thought I’d have a peek and see if Open Library holds references to any of his writings. Turns out we do.

From Third Class in Indian Railways, one of a short series of essays including The Moral Basis of Co-operation, published in 1917:

The compartment itself was evil looking. Dirt was lying thick upon the wood work and I do not know that it had ever seen soap or water. (From Page 5)

Or from A Guide To Health, published in 1921.

Exercise is as much of vital necessity for man as air, water and food, in the sense that no man who does not take exercise regularly, can be perfectly healthy. By “exercise” we do not mean merely walking, or games like hockey, football or cricket; we include under the term all physical and mental activity. Exercise, even as food, is as essential to the mind as to the body. The mind is much weakened by want of exercise as the body, and a feeble mind is, indeed, a form of disease. (Page 59.)

We have many more titles by him, including translations and other works such as Indian Home Rule, The Pilgrim’s March & Freedom’s Battle. (Just look for the “View” link to read the texts, scanned by the Internet Archive.)

Happy Birthday to this great man in human history. And in general, I wonder how we might begin to deliver “timely history” for visitors to Open Library. What timely historic texts might we display, driven by the editorial of the day?

It’s Banned Books Week this week, and we’re celebrating!

As Joan E. Bertin suggests over on Huffington Post, “for a country that venerates its First Amendment guarantee of freedom of speech, the United States tries to ban books with alarming frequency.” Sadly, it isn’t only the USA that has banned books in the past, as this list of banned books on Wikipedia shows. As Cara noted over on the shiny new Internet Archive news blog, we have scans of quite a few previously banned books for you to gaze upon, including but not limited to The Wonderful Wizard of Oz:

There are also a bunch of Banned Book events and displays happening all over the USA this week if you’re interested! If you happen to know of any alternative online sources of any of these titles, please feel free to leave a comment to let us know. Or, you could add a link in the description on their Open Library pages.

On a side note, we released a little upgrade to the open source Book Reader this morning too - now you can print pages from within the book reader. There are lots of nice little touches, for example, if you’re looking at a fold-out page, the reader will change the page’s orientation on the fly to landscape. Nice work, Mang! (Mang’s the lead developer on the Book Reader. And also has a great list of beautiful books on his bookmarks page that he uses for testing.)

We’re doing some work on improving our search engine at the moment. As we release the new code, search performance may be intermittent. Apologies for the interruption, but, search will be much faster when everything settles down. We’ll drop a note in here when it’s back online.

Update, 6PM PST: We didn’t quite get as much done today as we’d planned, but search should be stable. More tomorrow!

Update, 9AM PST, 8/28: Holy search, Batman!! Before… searching on Open Library was a slog. But now! It’s a breeze! Our search guy, Paul, has been tightening knobs and flipping switches (aka making good use of SOLR stored fields), and our chief data munger, Edward, helped push out the new code this morning. Just see how fast our 24,781 bacon records show up! Then, there’s the “collection” of digitized books about cheese… Please let us know if you come across anything untoward.

I just stumbled on a beautiful, recently-scanned book about snowflakes, published in 1863. Apart from its gorgeous illustrations, the author’s opinions about snowflakes are also fascinating.

By the way, the other day we added a little link on any Internet Archive pages that are echoed in Open Library that sends you straight to our open, editable record for that item, so, if you’re surfing around the Archive’s Texts collection and you find a book we have a record for, you can just jump across and - if you’re so inclined - help to flesh out the information we have about it.

The even more interesting thing about this is that Hank (who coded up the stuff from the Archive side) simply has to construct a URL at Open Library containing an Internet Archive item ID, like this:

http://openlibrary.org/ia/snowflakeschapte00warr

This is a small step towards something much more awesome. (We should note that what makes this work is that we’re just “across the hall” from Hank, so we were able to do a little testing and tweaking to make sure the link was offered only when we could count on there being an Open Library page for the book, but still!) The hope is that external sites can send along, say, and ISBN, or an LCCN identifier to poke into Open Library and see if we have a record.

The next logical step is to make a web service that can handle this sort of inquiry. We don’t have that just yet, and current performance is slow, but, it’s really exciting and something we aim to strengthen. It would mean that people could query Open Library for records even if they don’t know an Open Library identifier (but have an alternative identifier that we know about). The current list of identifiers we can record are: Dewey, LC, ISBN 10, ISBN 13, LCCN, IA & OCLC.

Another wish is to open up the sorts of identifiers that you might attach to an Open Library catalog record, for example, from our friends over at LibraryThing, or GoodReads, or any number of awesome book sites out there. Things get interesting if you consider the possibility that there may never be a canonical identifier for a book, but rather that there will always be many different ones. That consideration allows you to open up a pipe to any identifiers we can find, which makes for many more potential connections.

Snowflakes, indeed.

It is now possible to access Open Library book metadata in an  RDF/XML format. The access is through the RESTful API. For an example, view:

http://openlibrary.org/b/OL6807502M.rdf

The returned RDF/XML relies heavily on Dublin Core metadata terms, and uses some elements from bibliontology and the registered RDA schemas. Although soundly based on RDF, the output can be used like any XML and presents (most of) the Open Library metadata in the easily understood Dublin Core terms.

It has been suggested that this format include links to cover images, where available. It is also on our list to add tables of contents to the output. Other suggestions are very welcome — add them here, or send them to the ol-tech discussion list.

We’d love to hear about the uses you make of this API, and anything we can do to help you get more out of the Open Library.

There can be more than way to say the same thing, for example gramophone record, phonograph record and vinyl records. When libraries write catalog records they pick one of these terms and sticks to it, they use what is known as a ‘controlled vocabulary‘. This makes it easier to browse library catalogs.

Traditionally it has been thought that patrons want to browse by author and subject headings, so these fields have been controlled. The data in these fields can be used in other ways, Ross Singer has been experimenting with geographic subject headings.

Publisher is an uncontrolled field. Penguin and Penguin Books are the same publisher, but their name has been entered in catalog records differently, making it difficult to browse by publisher.

A workaround is to use the ISBN field in the catalog record. Almost every book published since 1970 has an ISBN. English-language books start with a 0 or 1, followed by a variable-length publisher code, item number and finally a checksum digit.

For example: 0-14-043531-X
0 = English language
14 = Publisher code
043531 = Item number
X = checksum

We are able to build a list of ISBN publisher codes by picking the most popular publisher name, as it appears in library records, for each code. Using ISBN we can start the process of making publisher a controlled field.

The results:

• To browse: list of 2 and 3-digit publisher codes for ISBNs that start with a 0. Includes details for each code
• To download: complete list of publisher codes for ISBN that start with a 0 or 1.

We’re very excited about a little integration that the smartypants team over at Flickr have done with Open Library. If you happen to use Flickr and just happen to photograph the covers or insides of books you read, there’s an easy way to connect them to the catalog records we have on Open Library. You just have to add a thingy called a “machine tag,” the same way you would add any other tag, making use of a special format for the tag.

For example, here is a photo of a book that Heather read, posted on Flickr:

This book also has a record on Open Library, at this URL:

http://openlibrary.org/b/OL23086206M/Tethered

Now, to link them together, all we need to do is add a specific machine tag to the Flickr photo that references the Open Library ID, like this:

openlibrary:id=OL23086206M

And hey presto, now you see a link to Open Library under Additional Information on Heather’s photo page:

You might find yourself asking, why is this good? Well, it’s good because it creates another channel for content to come into Open Library. We’ve been thinking about how much the rest of the web knows about all the books in our catalog, and we’ve begun the process of actively seeking out this content, and piling it onto our catalog records. So, each photograph or cover that we now have access to via Flickr is like another node in the network that surrounds our book records. Rather than treat these records as isolated, we want to connect them to as many things as we can find, which in turn, will begin to make Open Library richer with more points of entry than a search on Open Library itself.

There’s a curious example of this already, from STML on Flickr. He added the same machine tag to several photographs he’d taken of his copy of the “Progressive Atlas” book:

Nice to be able to see inside the book too!

We were also thrilled to discover this morning that one of the largest independent publishers in the USA, W.W. Norton, has added these Open Library machine tags to some 100 or so of their beautiful covers, archived on Flickr! Awesome!

There are a few fiddly bits emerging as people try this out. Like, when you do a search for a book, Open Library displays all the editions it knows about in the search results, and you might even see two records that have the same publisher name and publish date… That sometimes makes it a bit tricky to work out which particular Open Library record to link to, but our advice at this stage is just to pick one and run with it. (Later, we’d like to provide the option to merge two records into one, but we’re not there yet.)

Another interesting question being asked internally here, and also on our Flickr group is “what conventions should we be using for machine tags?” Our attitude here is that this integration is only very new, so it’s not the time to be impressing standards or conventions. We’d much rather just step back and see what people come up with on their own. There was a funny example yesterday, where dumbledad asked whether it was OK to tag this “action shot” of himself reading a book sitting outside with the openlibrary:id= machine tag. The response is, yes! Go for it! Or, create your own machine tag that seems to work for you, perhaps openlibrary:actionshot= or openlibrary:inside=, and we’ll just see what happens.

The next step, of course, is to have all these lovely bits and pieces show up on Open Library itself. Stay tuned!

(Disclaimer - I thought it might be important to say that I used to work at Flickr, but I had absolutely no say in the development of this new feature. There are also several other services that are “connectible” using this method. You might like to read the Flickr Code blog for more details on that.)

A few hours ago we released a couple of new bits and pieces we thought it was worth mentioning.

First, we’ve re-arranged the way search results display so our search facets are more obvious, there’s a new cover view, and the pagination is tidier.

You’ve always been able to see facets on the search page, but we were trying to find a way to make them more exploratory and interactive - hopefully, this redesign is a start. So, you can click on a facet to narrow your search, then another, and another. It starts to get interesting when you remove previously selected facets from the search, and begin to move sideways through the catalogue. (The team has wasted some hours playing with this!)

As I was bouncing around, I found a few gems, including 6 digitized books about the Masai, written between 1857 and 1905, including the fascinating Vocabulary of the Enguduk Iloigob and Through Masai land: a journey of exploration among the snowclad volcanic mountains and strange tribes of eastern equatorial Africa.

There’s also Cookery recipes by St. Mary’s Guild, Mill Valley, California - just around the corner from us here in San Francisco - published in 1902 and available to read online. Pickles, Marmalades, Jellies, Preserves is “swooning in sweetness” on Page 71, and the scan is full of hand-written notes, as any good cookbook should be!

And, as NASA celebrates the 40th Anniversary of the Apollo mission, here’s a bit of Mars-related science fiction to whet your appetite. If you like space stuff, you’ll love the collection of fantastic 16mm videos shot on board Apollo, hosted over at nasaimages.org, another project of the Internet Archive.

The other cool thing that we released is integration with the new, improved book reader available on archive.org. Improvements include a one-page view, access to the full resolution of the original scan (in that one page view), and the ability to link into a specific page in a scanned book, just by grabbing the URL in the navigation bar whenever you’re looking at a certain page, like I did above to link to Page 71 of the cookery book. (The URL updates on the fly as you turn the pages - super cool!) There’s more information over at the Open Content Alliance blog.

We’d love to hear what you think of the new search results page, so please leave us a comment!