Alice: The On-Line Catalog

By George Oates

Ohio University's Alden Library Alice Catalog, 1983

So awesome. That’s 1983 in Ohio, folks.

New Titles in Lending Library!

By George Oates

Our little lending library is continuing to grow, this time with 90 new titles purchased directly from two fabulous eBook publishers: A Book Apart & Smashwords.

3 titles from A Book Apart are all must-reads for any discerning web professional…

Thanks to Mandy, Jeffrey and Jason at A Book Apart for joining in the fun. (Incidentally, Mandy’s blog, A Working Library, is a great read.)

There are also 87 ePubs from Smashwords, by authors like Amanda Hocking, Ruth Ann Nordin and Gerald M. Weinberg

Thanks to @markcoker at Smashwords for working with us to get these new titles online.

Loans through the Open Library are exclusive to one Open Library account holder at once, for up to two weeks. For most titles, you can access the eBooks in one of three ways: directly in your web browser (using our BookReader), as a PDF or ePub (downloaded into Adobe Digital Editions). The new Smashwords titles are a bit different – they’re only available in ePub format, so only downloadable and readable in Adobe Digital Editions.

The Internet Archive (and Open Library) is actively seeking publishers who’d like us to buy their eBooks and make them available in the Lending Library. If you are a publisher interested to sell us your wares, please get in touch!

While that Lending Library — available to anyone with an Open Library account — is growing, we’re also working to expand the collection for our “In-Library” loans, currently at about 85,000 eBooks. This special In-Library program is a bit different, because it requires patrons to literally be inside a participating library’s network. Once that’s the case, patrons can see all the books available in the In-Library collection on Open Library, from all the libraries in the In-Library pool, currently around 150 North American libraries.

Public Library: An American Commons

By George Oates

Public Library: An American Commons
is a photography exhibition on at the San Francisco Public Library’s Jewett Gallery, running from April 9 to June 12. The photographer, Robert Dawson, has captured the American relationship with public libraries across the country in a series of intimate portraits. From the Design Observer review:

What’s at stake here is more than access to a room full of books. The modern American public library is reading room, book lender, video rental outlet, internet café, town hall, concert venue, youth activity center, research archive, history museum, art gallery, homeless day shelter, office suite, coffeeshop, seniors’ clubhouse and romantic hideaway rolled into one.

Minimum Viable Record?

By George Oates

Having worked more closely with bibliographic data than I had ever expected to over the last couple of years, I still can’t quite believe how complicated it can be. I keep holding tight something Karen Coyle told me when I first started at Open Library, that “library metadata is diabolically rational.” Now that I’ve witnessed the cataloging from lots of different sources and am more familiar with the level of detail that’s possible in a library catalog, I have a new fondness for these intensely variegated information systems; at times devilishly detailed, at others wildly incomplete or arcanely abbreviated. Everyone likes to arrange things and classify them into groups. It’s when you try to get people to put things into groups that someone else has come up with that it starts getting messy.

At Open Library, we’re attempting to ingest catalog data from, well, everywhere. Every “dialect” of cataloging practice makes this mass consumption harder. In spite of the rational goal of standardized data entry, there is an intense diffusion of practice. (Have a look at Seeing Standards: A Visualization of the Metadata Universe by Jenn Riley and Devin Becker if you haven’t already.)

A challenge I think we face today is a metastasized level of complexity, particularly as we attempt to begin to catalog works that have no physical form, but only exist electronically. Any challenge presents opportunity, and the opportunity here is to radically simplify the way things are represented in catalogs.

In February, I gave a presentation at the recent API Workshop held at the Maryland Institute of Technology and the Humanities (MITH). I talked about Open Library and paid particular attention to the resources we’re trying to put in place for developers to hook into the system.

Part of the presentation was an impromptu survey of the audience, where I passed around an index card for everyone, and asked people to write down the 5 fields they thought were adequate to describe a book. I framed the survey as a search for a “minimum viable record,” and it was fascinating to watch the audience squirm a bit as they asked for more guidance on the challenge. Can fields repeat? What’s the audience for this description? etc.

I’ve collated the results of the forty or so respondents into an ugly spreadsheet. There are 4 sheets, linked in the green strip at the bottom of the page:

  1. Book Raw – unfiltered results, in the order they were written
  2. Book Cooked V1 – all results blended, sorted alphabetically
  3. Book Merged – all results grouped
  4. Summary – with counts and a graph!

Here’s the final result:

So, on the shoulders of “minimum viable product“, a way for web application developers to get working code deployed quickly and effectively, I wonder if it’s time for a “minimum viable record” in place for bibliographic systems. Enough detail for a computer to match, correlate and compare, but not so much that having to process each record stops everything in its tracks.

You might have heard of the Open Publication Distribution System (OPDS) Catalog specification, which is a syndication format for electronic publications. Certainly, this new standard is a great step towards simpler representations of books — in this case, OPDS was initially designed to represent eBooks specifically — but I find myself wondering if it could be reduced further still, to pave the way for even easier exchange between systems. (Please note that all our edition records are now available in OPDS format, as well as RDF and JSON.)

Something like Title, Author, Date, Subject[s] and Identifier[s] might just do the trick, though it is of course irresistibly debatable. It’s an idea we’re going to look to as we work on our new Write API for Open Library. This minimum viable record will play gatekeeper for any new records we ingest (or that you export).

What do you think of this minimum viable blog post?

Book as Art Object

By George Oates

The Making of Tree of Codes, written by Jonathan Safran Foer, constructed by Die Keure, a printing house in Belgium.

Plus a fabulous write up from the publisher, Visual Editions: “The book is as much a sculptural object as it is a work of masterful storytelling: here is an “enormous last day of life” that looks like it feels.” [via foe]

In these reaction snippets, I love that a chap mentions, “OK, I’m getting the hang of it now.”

Open Library Architecture Diagram

By raj

Here is a diagram of the current Open Library architecture:

click for full-size image

A Library Primer

By George Oates

Just discovered this wonderful book of 60 short chapters on how to start a library: A Library Primer by John Cotton Dana, Fourth Edition, published 1906 by Library Bureau.

To the librarian himself one may say: Be punctual; be attentive; help develop enthusiasm in your assistants; be neat and consistent in your manner. Be careful in your contracts; be square with your board; be concise and technical; be accurate; be courageous and self-reliant; be careful about acknowledgments; be not worshipful of your work; be careful of your health. Last of all, be yourself.

And, it’s fantastic that our Read To Me feature in the BookReader can understand the librarian’s neat hand on the page of examples in the Ink and Handwriting chapter.

Specimen Alphabets and Figures

Scheduled Downtime: (Again) 9:30AM PST, 2011-03-10

By George Oates

Original post, 2011-03-07: The time has come for Open Library to migrate fully to the Internet Archive’s new virtual machine architecture. We expect the site to be down for about 2 hours as we move data and update various config files. Please bear with us… there are lots of balls in the air that we need to catch!

Also, we’ll post updates here if the plan changes.

Update, 11:30pm PST, 2011-03-07: Ok! The site’s back online, on brand new hardware. Everything looks about right, and we’re warming various caches and testing performance on various elements. Fingers crossed everything will warm up nicely over the next few hours. Yay!

Update, 9:30am PST, 2011-03-08: Just a little note to let you know that we’re still working on the migration. Our coverstore is struggling, and we’re tweaking our Gunicorn & lighttpd config in the new system. Apologies for the service interruption – you might see covers loading slowly, amongst other things we’re still discovering. More updates as they come to hand…

Update, 8:45am PST, 2011-03-10: Apologies for the short notice, but we’ll be bringing the site offline around 9:30am PST this morning, since we need to downgrade our lighttpd install from 1.4.26 to 1.4.19. The theory is that the newer version is still a bit unstable, and that’s part of the reason the site has been a little “bouncy” since Monday.

Heads Up! Data Center Migration in progress

By George Oates

You might notice a few hiccups, timeouts or slow-loading pages as you wander around Open Library over the next few days. The whole Internet Archive is migrating to a new virtual machine data center, which is no mean feat.

From Open Library’s point of view, that means moving data and services to the new virtual machine configuration, and making sure that everything’s running smoothly. We’re hoping this move will result in faster performance, and flexibility for increasing hardware and improving tools into the future.

Your patience is appreciated. See you on the other side!

UPDATE, 3:25pm PST: Our cover service is spluttering at the moment. That’s affecting the whole site’s performance. We’re looking into it. Apologies for the service interruption.

UPDATE, 5:40pm PST: OK. We’re pretty sure we’ve fixed the covers trouble. Yay! Also, we’re considering taking the site offline on Sunday evening (PST) to do the heavy lifting associated with the migration to the new virtual machines. We’ll let you know as far in advance as we can exactly when and for how long.

Get Thee to a Library!

By George Oates

For our first big release of 2011, we’d like to introduce you to a couple of new bits and pieces on Open Library:

  1. A new home page design
    New Homepage When we launched the site redesign almost a year ago, the home page was trying to make it clear that it was possible to edit the Open Library site, and that we welcome your contributions. You might remember the cheeky "Ever wanted to play librarian?" phrase. Now that the new design has settled somewhat, and we have a great level of activity across the site, we wanted to shift the focus again, to make it clearer that you can actually get to books as well. Not only over 1 million free eBooks, but also our small, but growing Lending Library.

    So, the new home page displays 3 new "carousels" that display an assortment of free eBooks to read, a small curated selection of titles from the Lending Library, and Version 1 of a new "Return Cart" feature, that shows you eBooks that have, well, been recently returned.


    We’ve also added some activity graphs at the bottom of the page, which tell you that in the last 28 days (at time of writing), we’ve had:

    • 5,794,587 unique visitors,
    • 14,219 new members sign up,
    • 39,939 edits to the catalog, 
    • 990 new lists created, and
    • 3,340 eBooks borrowed.


  2. The "In Library" lending program
    In one small step for library kind and readers around the world, today we’re announcing a new collection of "In Library" eBooks available for loan. Here’s the idea: there’s a group of libraries participating in the pilot program, each of which has added eBooks to the new pool.

    See a map displaying the participating libraries – Yay OpenStreetMap!

    The interesting part is that you, dear patron, need to get your bones into the actual libraries themselves to borrow any of the titles from any of the libraries in the pool. Once you’ve done that, the loan acts just like the "normal" Lending Library loans that are available to any Open Library account holder around the world, 5 books at a time, for up to 2 weeks. Cool, huh?

Read the Internet Archive announcement about the In Library program, or if you happen to be from a library that’s interested in joining in the fun, please get in touch with us.

As an aside, I’m attending the Maryland Institute for Technology in the Humanities API Workshop this weekend to talk about the Open Library API and how people are using it, so if you happen to be there, please come and say hello!