Multnomah County Library looks to understanding access
Aiming to better serve the growing Portland, Oregon community, Multnomah County Library aimed to transcend its role as a traditional repository of knowledge. With a massive library access population second only to New York City, the county embarked on a visionary journey. In collaboration with a grant from the Paul Allen Foundation, they set out to introduce an unprecedented concept: a Virtual Librarian service, poised to redefine access to information. This pioneering service sought to answer questions and cater to information requests in real time, revolutionizing the digital landscape for library patrons.
The challenge was manifold. Multnomah County Library faced the intriguing confluence of a forward-looking grant provider, intricate third-party technology vendors, and a vibrant community with diverse linguistic and urban dimensions. The county needed to validate the accessibility and usefulness of its groundbreaking Virtual Librarian service to both patrons and internal stakeholders.
Multnomah County Library had never conducted usability testing. The tool had been built and deployed and was live. Getting feedback would be critical as the County was balancing the expectations of a funder (Paul Allen), the third-party vendors that provided the software technology, and the community of library users. Up to now, feedback had come from opinionated librarians internally. A more empirical way to get feedback was needed: professional usability testing. Diverse patrons including patrons with disabilities would be included for feedback.
Recognizing the importance of empirical insights, Multnomah County Library sought the expertise of Experience Dynamics. The challenge was unique: the service was already deployed and live. Empirical feedback was crucial as the county juggled the expectations of funders, technology partners, and the dynamic library community. Experience Dynamics proposed a comprehensive usability testing approach that would uncover valuable feedback and provide actionable insights for enhancement. We would intercept live library patrons conducting rapid recruiting interviews (based on what they were checking out from the Library). In addition, we would conduct Library and home visits with users with disabilities to understand their needs holistically.
Testing a website typically follows the following: taking users through typical scenarios (unguided, without help) and having them Think Aloud. The problem with the Virtual Librarian tool was that it would need to be realistic to get useful feedback. The decision was made to test the tool with a live librarian service– without informing staff that testing was taking place not to bias the quality of their performance.
Experience Dynamics conducted 1:1 sessions with users at library branches in a make shift usability lab we set up before the sessions. We recruited users, moderated the test sessions and analyzed the data.
Experience Dynamics’ empirical approach resulted in eye-opening insights. The data gleaned from usability testing empowered Multnomah County Library to adjust its service using evidence-based decisions. This enabled the library to enhance user experiences, address accessibility gaps, and streamline user interaction options both online and in the physical library branches.
For example, while testing with blind users, we discovered the Library software screen reader (JAWS) had a 20-minute trial limit. This finding allowed the Library to adjust its access software policy and provide “full versions” of screen readers and other Assistive Technology, a must for users with disabilities to access digital information.
Experience Dynamics gave us the ability to both convey a best practices baseline and an analysis unique to the experience being analyzed…This not only helped the team’s knowledge but allowed for helpful, precise communications to other stakeholders as well.
Image: Accessibility impact graph showing it was 2.5x more difficult to perform a task for the users with disabilities group.
One of the biggest insights from the user testing was with blind users. Blind users were 2.5X less successful chatting with the Virtual Librarian. This was due to a poorly coded form element (a minor thing with a major impact on access, typical with accessibility projects). The Library gave this feedback to their third-party vendor to address this accessibility issue, thereby improving equitable access.
Overall, users loved the concept. Many had not heard of it, and it became obvious that this was due to “burying” the key content on the site. The other insight was that there were too many options to connect with a librarian, and users felt overwhelmed!
Using the Test Report with findings and recommendations from Experience Dynamics, the Library could report evidence-based specifics to stakeholders, understand where exactly users were struggling with the service, and prioritize improvements.
Multnomah County Library’s endeavor to amplify patron experiences shows how innovative a library can be. Collaborating with Experience Dynamics, the library embraced an empirical approach that unveiled user insights, bridging internal access assumptions with tangible data. This case study underscores the potency of empirical usability testing, showcasing its pivotal role in fine-tuning digital solutions, fostering stakeholder engagement, and enhancing the dynamic relationship between technology, patrons, and libraries of the future.