Sorry to say, but our search engine is all kinds of weird this morning (San Francisco time). Lots of the pages you see around the site, like a Work page or a Subject page, or indeed the Search Results page are driven largely by search.

So, while we’re working on fixing the problem, please excuse the various gaps you might encounter around the place as we resolve it.

Update, 12:50pm PST: We’ve tried running Lucene CheckIndex on the work search index, but it came back with “No problems were detected with this index.” Hmm. Next, we’ll try restoring from backup. Apologies again for the continued oddness around the site!

Update, 2:30pm PST, 7/13/11: OK. We’ve just removed the site alert that you might have noticed across the top of every page, because we think the search is back up online. It turned out that we had to rebuild the whole index from a 2-day-old backup, and then process the last 2 days of changes made on the site into the new index. Phew! If you continue seeing weirdness or results that look less correct than usual, please let us know.

Apologies again for the road bump – and a word of advice? Be sure to keep an eye on how and where your log files are being stored, and accumulated. Turns out it was a huge blob of log files is what ended up choking our search engine, stopping it from being able to be updated. We’ve since modified where we store logs, and how often they are cleaned out.