Test From Home Under Load

[elfs]

I learned an important lesson today: never put on hold a book from the library that’s already listed on the shelves. Just go get it. If it’s on hold, they’ll take it off the shelves and put it “into the system,” which means you won’t actually be able to get it for 24 hours.

I also learned another important lesson. Always start a few load-creating automated tests, then go to the most hostile wi-fi enabled cafe you know and try to use your web application from there.

The King County Library System recently spent a lot of money upgrading their computerized system. It looks like there was a desperate attempt to make it “more 2.0″ by throwing a thin layer of AJAX into the mix: searches now update page partials rather than whole pages, and various divs for holds, fines, checkouts, and wishlists come and go as you click on the appropriate tabs. All fairly unimpressive, by my standards. The developers, whomever they were, used Adobe’s library as their basis as the Javascript is full of Dreamweaver-related (“MM_”) tags. Where they didn’t use Dreamweaver, they wrote the code themselves; there’s a strong Not Invented Here feel to much of the Javascript they used.

And I’m sure, when they first started out, it looked okay. They tried to search for a few books, the spinner spun for a second and then the page rendered. Only now, under load, the spinner spins for a several seconds, and then the page goes blank. A long pause later, the page begins drawing again. The clue that the spinner needs to stop and the signal that the data is ready to be drawn are two different signals.

There are several examples of this, but the book search is the most egregious. All in all, the upgrade has made a lot of people mad: renderers don’t work, often fail to deliver, time out without excuse. And this doesn’t have to be hard. KCLS has slightly under a million items, and had 21 million check-ins last year, or about 3 a second during business hours. Prodigious, but seriously middle-tier as far as businesses go. A modern implementation of the website should not be as difficult as they’ve made it.

This entry was automatically cross-posted from Elf's technical journal, ElfSternberg.com

Comments

[jry.livejournal.com]

A well-connected friend of mine said they switched systems because the company that provided the old system wasn't open to making any changes to support KCLS. So they switched to this thing which is supposedly supported open source. Here's their site: http://evergreen-ils.org/ I'm sure they'd love to have your help and I know you need more projects ;-)

[jry.livejournal.com]

Interesting account of the go-live process (http://www.galecia.com/documenting-king-county-library-system-going-live-on-evergreen/) this month.

[elfs.livejournal.com]

"The truth of the matter is that much of what is happening is still just-in-time development."

Shudder

Okay, I'll be fair. I've been there, and done that. And I even took a look at the Evergreen source repository which, quite frankly, scares the bejeezus out of me. I respect people who can code in multiple languages (I am one, after all), but man, that thing is huge, sprawling, and multilingual to the point of dysfunctional unreadability.

Huh. I kinda want to figure out how they created "distributed ORM," though. That looks fairly interesting.