Category Archives: Open Source

Highlights open source activities

It takes a Classroom to build an Open Library

On most days, the Open Library is hard at work improving the experience it offers to students and teachers in classrooms. But for the past few months, Open Library has had the privilege of enjoying contributions from 7 students around the globe who had been assigned by their universities to participate on open source software projects.

First and foremost, the entire Open Library community extends our deep gratitude to AUEB / Athens University of Economics and Business‘s Dr. Diomidis Spinellis (professor of Software Engineering, who taught the course Software Engineering in Practice) and NYU‘s Dr. Joanna Klukowska (Clinical Associate Professor of Computer Science, who taught the course CSCI-UA.0480-061, Open Source Software Development) for incorporating open source contributions into the curriculum of their classrooms. As we hope you’ll see, the decision to promote hands-on development has an outsized impact on supporting open source projects like ours.

In the spring semester of 2022, four students from Greece’s AUEB (Constantina Z., Vassilis B., Dimitris B., and Philippos P. / Φιλιππος Π.) and three students from NYU (Michelle T., Crystal C., Chloe Q.) spent time participating in community calls, problem solving, and improving the Open Library service for the public. In return they received mentorship and first-hand experience learning how to contribute to a platform trusted by millions of international readers.

This year, the foci of Open Library’s roadmap is improving core experiences for patrons. Towards this goal, each of these students exceeded our expectations by contributing meaningful improvements like: Chinese internationalization of the website, google analytics to help inform us on meaningful ways to improve the organization of the website, fixing broken mobile navigation for our Books Page, UI improvements for sharing books on social media, adding APIs for Trending Books, and much more. We’re extremely proud of and grateful for the work these students were able to contribute.

In the past, Open Library has reserved a special honorary title of “Open Library Fellow” for exemplary contributors who have demonstrated exceptional commitment, leadership, and impact with the Open Library project. Our list of previous Fellows include Sabreen Parveen (who designed our onboarding experience), Yash Saravgi (who developed our mobile Progressive Web App), and Bharat Kalluri (who helped standardize our import pipelines). Each dedicated several months implementing features which redefined core behaviors and experiences of the Open Library.

This year, we believe one student in particular, Constantina Zouni, stands out as being especially deserving of this special Fellowship distinction, for her initiative, participation in engineering and design process with stakeholders, and outstanding work ethic.

Please join us in celebrating the work of this 2022 international student cohort, sharing our gratitude, and congratulating Constantina on her inspiring example.

Improving Experiences for Open Library Patrons

By Constantina Zouni

As this semester of my studies is coming to an end, I want to do a retrospect about my experience with the open library project.

My Journey with Open Library

In the beginning of the semester my professor Dr. Diomidis Spinellis for the course “Software Engineering in Practice” announced that in the context of an assignment we had to choose an open source project to make contributions thought out the semester. As a result, I started searching for a project and I was lucky to quickly find open library’s repository. Some of the main reason that made me to choose that project is that the community was very friendly and really open to contributors. The documentation of the project was really detailed and there were videos that helped me understand how the project works. Also, another good thing was that the issues of the project were well organized with labels and the context was explanatory enough. Moreover, the project seemed to be very active with quick responses in the comments section and pull request merges almost every day. After the first communication with the team everything went very smoothly. I was welcomed in the slack channel, and I was invited to participate in the weekly meetings. Mek quickly stepped in and helped me to get started. Because that period was busy and contributors from other universities also chose to contribute to open library the project’s team made effort to create a GitHub project and assign issues to everyone. I started solving minor issues related with text appearing when not needed, adding the subtitle to the search results and some UI improvements. Ultimately, in collaboration with another student from my university Vassilis Bubis we created the twitter social card that enables users to share their book lists. Through out the whole period that I contributed to the project I was impressed that Mek and the other members of the open library team dedicated time answering our messages and even jumping on small zoom meetings.

Book page header in mobile

One of the issues that I think had a big impact in the open library users is the improvement of the book page header in the mobile environment. When users visited a book page from mobile the experience wasn’t that pleasant. The book title and other important information like the author, the subtitle etc didn’t fit in the phone screen and the user had to scroll down to see them. The issue was more significant in the cases where the book covers were ambiguous, and it made difficult for the user to understand if they were in the correct page. Jim Champ recommended to follow a specific layout for the book page in mobile in order to fit all the important information in the mobile page. The challenge was the layout had to be different depending on the device of the user. My first implementation involved some java script code that change the order of the elements and an event listener that was activated when the screen had a specific size. The open library team quickly informed me that this implementation was causing a delay in the loading of the page, and they recommended me to use HTML and CSS. This time with a new implementation and the help of Jim Champ who was reviewing my pull request I managed to solve the issue using an HTML file that included only the title summary and some CSS commands.

Book header in mobile before and after

Dynamic book list preview for sharing

This new feature was a little more challenging than the previous one. This time I collaborated with Vassilis Bubis in order to create a dynamic preview for the book lists of the users that displays the first 5 books of the list. Then this preview is passed to the twitter social card and every time a user wants to share a list with the URL the preview image appears. This is a more interactive way for users to show their book lists to others and makes open library more recognizable across twitter users. The first challenge was to create a mock-up of the preview. To achieve that I used a design tool called Figma to create prototypes with different colour combinations and I let the open library team to decide which on they like more. For the design I used colours from the open library’s webpage, and I added a twist in the preview that represents a self where the books are placed. Alongside with the mock-ups Vassilis worked on retrieving the book covers that we need and place them above the background with the help of a Python library called Pillow. Then I stepped in, and I made sure that every book cover was resized in a way proportional to the original dimensions that it had. We noticed that some covers were stretching so it was important that every time we changed the width of a cover the height was adjusted properly. Another challenge was the text that we wanted to add in the preview. The text had to change dynamically, and we had to change line every time the characters exceeded a specific number to achieve an aesthetically pleasant result. One issue that we faced was that the coordinates of the covers that we had figured out with Figma had to change because in python the coordinates are applied from upper left corner compared to Figma that apply to the center of an image. After solving that Vassilis and I proceeded on storing the image in an in-memory binary array for better performance and finally creating the API for the list page.

Twitter social card for book lists sharing

Book page editing improvement

While working on some issues in the book page I realised that compared to other library webpages open library gives users the ability to edit the details and the information of a book. That feature is very valuable because users can add important details for a book that were missing when it was added, they can update that information and they can add descriptions and subjects that might be useful for other users. Although this feature is really important the editing user interface is not that pleasant. When users click on the edit button, they are directed to another page. My recommendation regarding that is to use a modal that pops up when the button is clicked. In that way users will feel like they have more control because they won’t be directed to another page, and they can still see the book page behind the modal. Another issue with the existing editing form is that users can discard the changes with the cancel button, but they can’t undo a change without deleting all the changes. In the mock-up that I created I added an arrow in the right upper side that symbolizes the undo action. I noticed that the examples for every field were placed next to the field title, and I opted to move them inside the text box for a clearer look. Finally, I added the info symbols beside every field that provides details on how you should fill out that specific field. Overall, the purpose of those recommendations is to make the booking editing more simple, compact and user friendly.

Book editing page now
Edit book page with modal created with Figma

The Open Book Genome Project

We’ve all heard the advice, don’t judge a book by its cover. But then how should we go about identifying books which are good for us? The secret depends on understanding two things:

  1. What is a book?
  2. What are our preferences?

We can’t easily answer the second question without understanding the first one. But we can help by being good library listeners and trying to provide tools, such as the Reading Log and Lists, to help patrons record and discover books they like. Since everyone is different, the second question is key to understanding why patrons like these books and making Open Library as useful as possible to patrons.

What is a book?

As we’ve explored before, determining whether something is a book is a deceptively difficult task, even for librarians. It’s a bound thing made of paper, right? But what about audiobooks and ebooks? Ok, books have ISBNs right? But many formats can have ISBNs and books published before 1967 won’t have one. And what about yearbooks? Is a yearbook a book? Is a dictionary a book? What about a phonebook? A price guide? An atlas? There are entire organizations, like the San Francisco Center for the Book, dedicated to exploring and pushing the limits of the book format.

In some ways, it’s easier to answer this question about humans than books because every human is built according to a specific genetic blueprint called DNA. We all have DNA, what make us unique are the variations of more than 20,000 genes that our DNA are made of, which help encode for characteristics like hair and eye color. In 1990, an international research group called the Human Genome Project (HGP) began sequencing the human genome to definitively uncover, “nature’s complete genetic blueprint for building a human being”. The result, which completed in 2003, was a compelling answer of, “what is a human?”.

Nine years later, Will Glaser & Tim Westergren drew inspiration from HGP and launched a similar effort called the Music Genome Project, using trained experts to classify and label music according to a taxonomy of characteristics, like genre and tempo. This system became the engine which powers song recommendations for Pandora Radio.

Circa 2003, Aaron Stanton, Matt Monroe, Sidian Jones, and Dan Bowen adapted the idea of Pandora to books, creating a book recommendation service called BookLamp. Under the hood, they devised a Book Genome Project which combined computers and crowds to “identify, track, measure, and study the multitude of features that make up a book”.

Their system analyzed books and surfaced insights about their structure, themes, age-appropriateness, and even pace, bringing us withing grasping distance of the answer to our question: What is a book?

BookLamps-Theme-Currents-for-Carrie

Sadly, the project did not release their data, was acquired by Apple in 2014, and subsequently discontinued. But they left an exciting treasure map for others to follow.

And follow, others did. In 2006, a project called the Open Music Genome Project attempted to create a public, open, community alternative to Pandora’s Music Genome Project. We thought this was a beautiful gesture and a great opportunity for Open Library; perhaps we could facilitate public book insights which any project in the ecosystem could use to create their own answer for, “what is a book?”. We also found inspiration from complimentary projects like StoryGraph, which elegantly crowd sources book tags from patrons to help you, “choose your next book based on your mood and your favorite topics and themes”, HaithiTrust Research Center (HTRC) which has led the way in making book data available to researchers, and the Open Syllabus Project which is surfacing useful academic books based on their usage across college curriculum.

Introducing the Open Book Genome Project

Over the last several months, we’ve been talking to communities, conducting research, speaking with some of the teams behind these innovative projects, and building experiments to shape a non-profit adaptation of these approaches called the Open Book Genome Project (OBGP).

Our hope is that this Open Book Genome Project will help responsibly make book data more useful and accessible to the public: to power book recommendations, to compare books based on their similarities and differences, to produce more accurate summaries, to calculate reading levels to match audiences to books, to surface citations and urls mentioned within books, and more.

OBGP hopes to achieve these things by employing a two pronged approach which readers may continue learning about in following two blog posts:

  1. The Sequencer – a community-engineered bot which reads millions of Internet Archive books and extracts key insights for public consumption.
  2. Community Reviews – a new crowd-sourced book tagging system which empowers readers to collaboratively classify & share structured reviews of books.

Or hear an overview of the OBGP in this half-hour tech talk:

Introducing: Community Reviews

You can now publicly review books using structured book #tags on Open Library with Community Reviews. Take a look, try it out, and send us feedback!


Many social book websites including Goodreads & LibraryThing feature text reviews from the community. Why hasn’t Open Library?

As a non-profit library service with a small staff, there are three reasons we’ve resisted the urge to add text reviews to Open Library. First and foremost, we feel strongly about preserving Open Library as an inclusive, safe, neutral place where readers can trust the information they receive. Some opinionated reviews, even though valid, may contend with this goal. Secondly, we’re cautious about adding features which may require a large time investment to moderate well. We’d rather spend our time making it easier for people across the globe to find books in their native languages than sink all of our time reviewing spam. Finally, there are indeed already several websites which feature text reviews. We’re excited to link patrons to these resources and think our time may be better served exploring new ways of adding unique value back to the book ecosystem.

This all said, reviews are one of the most requested features by book lovers on Open Library and we feel its important readers to have their voices heard. So what are our options?

A review of reviews

One super-power of text reviews is that they are unstructured. Their open-ended format allows reviewers to express very nuanced and deep thoughts like, how impressively the male author Arthur Golden was able to portray the emotional turmoil of the female characters portrayed in Memoirs of A Geisha. This super-power does come with a trade-off. It can be challenging to compare reviews and know which should be trusted; two reviews may have completely diverging styles or focus. One reviewer may be reacting to the story line while another may be critiquing the book’s pace. Reviews are often not easily digestible. A lot of information is lost when one tries to compress a review into a single star rating. Because of these challenges with “digestibility”, it’s also challenging to summarize text reviews as data which may be used to help people discover new books. Amazon has some techniques which we considered:

A collaborative approach

How can Open Library empower readers to share their impressions about books in a new way, facilitate useful reviews which are structured and easily digestible, while maintaining a safe and neutral library landscape?

Open Library’s collaborative approach, which we’re calling Community Reviews, borrows from an old (now defunct) project called BookLamp and a more recent project called StoryGraph, which let participants use tags to vote on & review various aspects of books like pace, genre, mood, and more:

StoryGraph crowd sources tags like genre and mood from the community and use this information to help readers find the right book for them
BookLamp used a hybrid of robots and crowd sourcing to identify themes and topics within books.

The more participants who vote using review tags, the more accurate and meaningful the review becomes for the community. Instead of sifting through dozens of text reviews, Community Reviews gives readers a birds-eye view across many publicly listed dimensions they might care about like Pace, Enjoyability, Clarity, Difficulty, Breadth, Genre, Mood, Impressions, Length, Credibility, Text Features, Content Warnings, Terminology, and Purpose.

Here’s what Open Library Community Reviews looks like:

By clicking “+ Add your community review”, any logged in reader may submit their own public, anonymous reviews:

Building Together

Community Reviews features a public schema which anyone may reference or propose changes to. It’s a work in progress and will undoubtedly need the community’s feedback to become useful over time.

Feedback

Community Reviews is a beta work in progress and we expect it to change drastically over the coming weeks based on feedback from our community. We also anticipate issues and bugs may emerge — you can help by reporting bugs and issues here.

We do have every intention for Community Reviews to be included (in an anonymized form) in our public monthly data dumps for the benefit of our community and via our APIs, though this may take some time to implement.

As the number of Community Reviews increases, our plan is to include them in our search engine so you have ever more ways to identify the best books for you.

We know many patrons would still love to see text reviews on Open Library and that Community Reviews isn’t a replacement for every use case. We sincerely appreciate this and still, we hope that readers will find this new feature valuable and provide us with feedback to improve it over time.

Thanks

We’d like to sincerely thank Jim Champ who recently joined as staff member on Open Library and whose leadership was indispensable in bringing this feature to life. Thank you to you Drini Cami, also staff at Open Library, for his contributions to improving the user experience. If you hate the idea or execution, blame Mek but do give us feedback to improve.

An old brass key dropped on the ground in the woods.

Open Library Tags Explained—for Readers Seeking Buried Treasure

As part of an open-source project, the Open Library blog has a growing number of contributors: from librarians and developers to designers, researchers, and book lovers. Each contributor writes from their perspective, sharing contributions they’re making to the Open Library catalog. As such, the Open Library blog has a versatile tagging system to help patrons navigate such a diverse and wide range of content.

Continue reading

Giacomo Cignoni: My Internship at the Internet Archive

This summer, Open Library and the Internet Archive took part in Google Summer of Code (GSoC), a Google initiative to help students gain coding experience by contributing to open source projects. I was lucky enough to mentor Giacomo while he worked on improving our BookReader experience and infrastructure. We have invited Giacomo to write a blog post to share some of the wonderful work he has done and his learnings. It was a pleasure working with you Giacomo, and we all wish you the best of luck with the rest of your studies! – Drini


Hi, I am Giacomo Cignoni, a 2nd year computer science student from Italy. I submitted my 2020 Google Summer of Code (GSoC) project to work with the Internet Archive and I was selected for it. In this blogpost, I want to tell you about my experience and my accomplishments working this summer on BookReader, Internet Archive’s open source book reading web application.

Continue reading