User requirements report

Posted on April 13, 2012 by Keren Mills

Aim

The purpose of gathering user requirements was to ensure that the development of the mobile search resulted in a user-friendly interface with functionality that would make it useful and attractive to OU students and staff.

Method

Members of Open University staff were invited to participate in the user requirements study via notices on the OU intranet and via a Mobile Technologies Special Interest Group (M-Tech SIG). All participants would be entered into a prize draw for one of ten Amazon vouchers. See Appendix 1 for the wording of the invitation. The project manager recruited some academic staff, and some non-academic staff who are current OU students. The participants were asked to use the mobile search interface for any library searches they needed to do during January 2012. They were asked to send us feedback as soon as they completed a search, telling us what the experience was like and what they would want to be able to do with a mobile search tool.

In order to prompt participants to record their experiences we provided a feedback form linked from the mobile search results page. As users had to be logged in to use the search tool we were able to fill in their name and the date and time for them, so all they had to do was tell us where they undertook their search, how they found the experience and what sort of functionality they would like on the search page and results page.

Participants were asked to use their own mobile phones or small tablets where possible, but were offered the opportunity to borrow a mobile device if they didn’t have a suitable one.

Unfortunately the site’s device detection failed during the study so mobile users were finding themselves on the desktop site rather than the mobile version of the site.

To test new developments on the search interface we also held a focus group on 27 February. We invited all 15 of our volunteer participants, but only two attended.

Participants

The initial response to the invitation via the intranet and mailing list led to 16 staff volunteering to take part. One later dropped out due to going on study leave. Of the remaining 15 only six participants actually gave us feedback on the mobile search interface.

Participant type F M Total
Academic 1 3 4
Student 4 4 8
Tutor 2 2
Research student 1 1
Total 5 10 15

The participants represented a range of subject areas such as Arts, Business, Classical Studies, Earth Sciences, Educational Technology and Languages.

Results

Five participants completed the feedback form, and left the following comments (supplied verbatim, without corrections):

Student: Using my smartphone, I have accessed library resources during “empty time” as I like to think of them – e.g. if the girlfriend is in a shop while I wait outside; while waiting for a bus/other queue; while on a bus if I have an audio resource. I have a HTC Desire with Android. The search screen fits well, and the option to view a page (where available) is very useful when I do not wish to download a large (2MB+) PDF. The search refinement is OK but the drag bars can be tricky to use on a smallish screen when non-mouse controlled.

Research student: It was excellent. I managed to download PDfs from ScienceDirect and lots of the options I have from PC are there from mobile as well. If using wi fi, it’d even be fatser and better than the network’s internet especially if the signal or coverage not that good.

Tutor: very smooth I had no problems

Academic: I was not automatically directed to the mobile site. My username and password were not accepted. I didn’t get an error message, I just kept being redirected to enter my user name or password. So I was unable to read any of the articles.

The “search refinements” and “drag bars” referred to by the student are part of the EBSCO Discovery Service desktop site, and not part of the mobile search interface. The prototype search used for this study did not link to EBSCO’s mobile optimised interfaces if participants left the library website.

The problem with signing in experienced by the academic is likely to have been due to the difficulty with entering the complex alpha numeric passwords via small touchscreen keypads. One of the aims of the MACON project is to investigate a solution to make authentication via mobile devices easier.

Participants were asked to indicate how long they had spend on the mobile search interface during a single search session. Their responses indicate they either spent 2 to 5 minutes or 10 to 30 minutes searching. This seems to indicate that at 40% of them were willing to persist with their search despite the limitations of small screen.

Chart showing preferences for mobile search resultsThe project team also considered it important to ascertain how users might want to interact with their results once they had performed a search. We were aware that some participants were likely to be reluctant to read the full text of an article or ebook on a small screen, so we wanted to know what they would do with their results instead. Participants were given a multiple choice question about this in the feedback form and could select as many options as they wanted to.

MACON focus group, 27 February 2012

The focus group consisted of two participants who were also part of the project’s diary study. The devices used by the participants were an HTC Desire and a Nokia N900.

Impressions of interface

Notes Action
Nokia N900 is mostly used in landscape mode and the smartphone homepage icons don’t reflow Consider allowing icons to reflow
Placement of navigation icons in relation to search box makes them look related to the search. Participants suggested moving the search box below the “Search” heading to separate these elements. Also provide text under icons. Move “Search” heading above search box and try text under icons
One participant asked if it’s really necessary to have the helpdesk hours and helpdesk contact details separately. Suggested combining them under the headset icon. Combine helpdesk hours into “Help & Support” page

Search results interface

Notes Action
Both participants liked the buttons in the search results and didn’t feel the item title needed to be a link
Buttons not left aligned on Samsung Check whether this relates to “mobile view” setting on Android browsers
One participant would prefer to be able to choose the number of results to show per page. Investigate whether this option could be incorporated without crowding the screen too much
Participants would like to see total number of search results above page number navigation Expose number of search results between search box and page number navigation
Participants would like help/tips on which Boolean limiters can be used in the search Provide limiters as part of ‘Advanced search’
It needs to be made clear what is being searched (i.e. articles & ebooks, not journal titles). Provide suitable introduction on search page
One participant would like to be able to browse journal titles through the mobile interface Not part of project – note for later development
Both participants would like the search terms to be highlighted in the search results list and in the abstract when viewed. Highlight search terms within results.

Accessing full text

Notes Action
Redirection errors experienced for some results – possibly those not on EBSCO Test ‘get text and get PDF’ results to identify where this problem is occurring.
DOIs not resolving consistently to full text Investigate DOI resolver
Participants perceived an error when tapping “Get text” led to “body text is not available” message. Don’t show “get text” if full text not available in html or other non-PDF format.

Native app

When asked the participant with the Android phone said that when the mobile search interface was first publicised he had expected it to be an app, and would normally expect an app for such a specific service, but both participants were quite happy with the web interface.

Conclusions

Methodology

A mobile-friendly feedback form may lead some participants to give very brief feedback.

User requirements

  • Three of the five participants wanted to read the abstracts of items in their search results, four wanted to save their references so that they could read them or cite them later and only one wanted to read the full text.
  • The mobile search interface should provide users with flexibility, and allow them to mark results to read or cite later on a different device.
  • Participants didn’t mention authentication as being a particular barrier to a good search experience.
Posted in Evaluation, Mobile discovery | Tagged , , | Comments Off on User requirements report

March progress update

Posted on April 5, 2012 by Keren Mills

A focus group held on 27 February provided useful feedback on the mobile discovery interface which has influenced development and led to an advanced search option being added.

Extensive exploration of authentication options led to the project team concluding that a simpler PIN authentication process cannot be provided for mobile users at this time.

Activities completed in March

  1. Completion of diary study and analysis of participant feedback
  2. Work has started on inclusion of local audio and video content from the OU Learn channel on the university’s podcast server in the mobile discovery tool.
  3. The mobile discovery interface has been tested on small screens up to 7 inches and now shows larger icons on tablet screens.
Macon budget chart November to February

Macon budget forecast and actual costs from November 2011 to February 2012

Posted in Project management | Comments Off on March progress update

Developing simpler authentication for mobile users

Posted on February 22, 2012 by Keren Mills

One of the deliverables of this project is a “mobile bookmarklet tool”, which is intended to help address the difficulty of entering a complex password into a mobile keyboard. The difficulties encountered tend to be with accessing special characters and achieving complete accuracy, especially in password fields where characters are hidden as soon as you type them.

The original concept of the bookmarklet tool was that once a user had linked to a content provider’s website through EZproxy and had been asked to authenticate via the OU SAMS authentication system they would then be able to bookmark each site within a mobile application. The user could then easily return to the content provider’s site without having to remember the URL, connect through EZproxy or enter their authentication details. However, the project board raised concerns about the security of this approach as it could enable anyone who got hold of a mobile phone belonging to a valid user to access our eResources.

Our alternative approach is inspired by an earlier OU Library project called RISE (Recommendations Improve the Search Experience), which experimented with a token based method to authenticate users from a Google Gadget.

The planned approach for MACON will be

  1. Pass the user to a SAMS-Protected page generates a hash token, and redirects the user to a mobile authentication page with the token as a URL parameter, and stores this token in the database.
  2. The user is provided with a 6 digit PIN which they can thereafter use in place of their SAMS password for signing in to the mobile website
  3. When a user accesses the mobile website and is prompted to sign in, they enter their OU Computer Username and PIN, which is checked against the database entry stored in step 2.

We are just starting to develop this prototype and will have to ensure that the OU’s IT department are happy with this approach before we roll it out. The advantage of this approach is that no data is stored locally on the device, but it will need testing with users to ensure they are able to recall both a PIN and their usual password.

Posted in Mobile bookmarklet tool, Project management | Tagged , | 3 Comments

February progress update

Posted on February 21, 2012 by Keren Mills

Work on the mobile discovery solution is progressing well despite a dearth of feedback from user requirements diary study participants. The little feedback we have had has been mostly positive. We plan to hold a short focus group to gather more user requirements and evaluate the new developments.

After a useful discussion at the January project board meeting about the mobile bookmarklet tool we have started to prototype an approach to that.

Activities completed in February

  • Work has started on the development of the mobile discovery interface and the mobile authentication tool
  • The project team has discussed mobile site design elements and wireframe with internal project consultants and agreed functionality likely to be need for search results interface. This will include the means for users to export references as a RIS file and save them to their preferred bibliographic management tool.
Posted in Project management | Tagged | Comments Off on February progress update

January progress update

Posted on January 22, 2012 by Keren Mills

Development of the mobile search interface is progressing well despite a disappointing lack of feedback from our user requirements diary study. We had hoped to complete our diary study to establish user requirements by the end of January, but found that we’d had very little feedback so have extended the study into February.

Activities completed

  • A project developer was appointed and started work on 19/01/12.
  • The user requirements diary study has attracted 16 participants.
  • We started to benchmark our mobile search interface against others.
  • The project team met with Open University colleagues developing an accessible Open Media Player to explore the possibility of using it for mobile delivery of rich media search results.
  • First Project Board meeting took place on 18/01/2012
Posted in Project management | Tagged | Comments Off on January progress update

Invitation to OU Staff: search the library on your mobile phone and win an Amazon voucher!

Posted on December 20, 2011 by Keren Mills

OU Library Services would like to invite you to take part in assessing our mobile library search tool in order to help us improve it. If you are willing to take part you will be asked to carry out any library searches that you need to do as part of your studies via our mobile search.

The mobile search tool will search across ejournals, ebooks and our own Open Research Online repository.

What’s involved?

You would also keep a journal telling us what you liked or didn’t like about the experience, how relevant you found the search results and whether you wanted to read the full text on the mobile or save a reference to read later. You would be asked to do this for a short period in December or January.

Your diary entries would be analysed anonymously by the project team and any research outputs would not reveal your identity as a participant.

Who can take part?

This invitation is open to non-academic staff who are students and OU academic staff (including Associate Lecturers).

All participants will be entered into a prize draw for one of ten £25 Amazon vouchers in return for taking part.

How do I take part?

You can use your own mobile phone, iPod touch or small tablet if you have one. The interface is designed for use on small screens (5 inches or less) where the desktop version is not suitable. Alternatively if you are a member of staff on campus we can supply a Wi-Fi enabled mobile phone or iPod touch for you to use.

Please note that if you use your mobile data connection (GPRS or 3G) rather than Wi-Fi your normal data charges will apply. Pay monthly contracts sometimes include mobile data, but we advise you to check your contract or you’ll be charged per use. Pay As You Go users will also generally be charged per use. For staff there are instructions for connecting to the campus Wi-Fi available from IT or an auto-configure tool for iPhones from MCT.

If you would like to take part please email Library-Webmaster by 13/01/2012.

Posted in Evaluation, Mobile discovery | Tagged | Comments Off on Invitation to OU Staff: search the library on your mobile phone and win an Amazon voucher!

User requirements – diary study

Posted on December 20, 2011 by Keren Mills

In order to ascertain user requirements for our mobile discovery interface, we are going to ask members of Open University staff to try out a prototype mobile search interface we already have. We will recruit some academic staff, and some non-academic staff who are current OU students. They will be asked to use the mobile search interface for any library searches they need to do for an agreed time period. As soon as they complete a search they’ll be asked to send us feedback telling us what the experience was like and what they would want to be able to do with a mobile search tool.

In order to prompt participants to record their experiences we’ll add a link to a feedback form to the mobile search results page. As users will have to be logged in to use the search tool we’ll be able to fill in their name and the date and time for them, so all they have to do is tell us where they undertook their search, how they found the experience and what sort of functionality they would like on the search page and results page.

Participants will be asked to use their own mobile phones or small tablets where possible, but will be able to borrow a mobile device if they don’t have a suitable one.

The results of this study will inform our development of the Mobile Discovery tool and the mobile authentication bookmarklet and will be followed up with further user testing later in the project.

Posted in Evaluation, Mobile discovery | Tagged , | Comments Off on User requirements – diary study

User requirements gathering – choosing a methodology

Posted on December 5, 2011 by Keren Mills

One of our first steps in this project has to be finding out what users really want from a mobile discovery interface. In this case users include OU students, OU staff and OU librarians. We considered the following three methods:

1. User stories – an idea borrowed from agile development.

  • Method: This would involve asking users to complete the following statement “As a student/academic/librarian I want to …….. in order to ……….”
  • Pros: This task is simple to set up, easy for participants to complete and might attract a higher completion rate.
  • Cons: The resulting data doesn’t provide much detail regarding specific aspects of the tool.

2. Remote usability testing using a tool such as Loop 11 or Usabillia –

Method: Setting a series of tasks for users to complete and analysing their success rate and which aspects of the interface design supported success or led to failure.

Pros: Participants would not need to travel to campus (the Open University is a distance learning institution).

Cons: Users would not be interacting in normal circumstances which might skew their reaction to it. This method might work better for evaluating particular aspects of the interface when it is more fully developed.

3.  User diaries

Method: This involves giving people access to a prototype search tool and asking them to use it for searches they need to do anyway in their studies/research/work over a given period of time. Participants keep a diary logging how they fared with the tool.

Pros: Participants are using the tool in real circumstances and getting a feel for how useful it is in their day-to-day studies or work. The data acquired from the diaries has the potential to be quite detailed and participants can include both requests for functionality and feedback on the existing tool.

Cons: Participants may forget to do their searches through the mobile interface or may lose interest in a short time. Participants may not log their diary entries immediately after using the tool and may forget what they wanted to say about it.

After some discussion the MACON team chose to use the diary method. We will be inviting OU staff and students to participate by posting news items on the staff intranet, the library news, the Library Facebook page and OU’s Platform. Participants will be offered the chance to enter into a prize draw to win one of ten Amazon vouchers for taking part.

Posted in Mobile discovery, Project management | Tagged | Comments Off on User requirements gathering – choosing a methodology

What is MACON?

Posted on November 1, 2011 by Keren Mills

MACON will address challenges involved in delivering quality academic content to mobile devices in a seamless and user-friendly manner. The project will work with EBSCO, a major content and systems provider in order to prototype a mobile friendly resource discovery interface which will discover and expose quality academic content from both third party & local collections.

This project is part of the JISC Mobile Infrastructure for Libraries programme. It will start in November 2011 and finish in August 2012.

The overall objectives of the MACON project are to:

  1. Increase the availability of quality academic content for mobile users
  2. Explore and understand user requirements for the mobile delivery of academic content from both third party and local collections
  3. Identify and document the risks and barriers involved in making academic content accessible to mobile devices.
  4. Develop and evaluate an interface for discovering and exposing quality academic content to users via small screen mobile devices
  5. Work with EBSCO to develop and test a prototype system using a combination of open source software (Mobile Web from MIT) and the API from EBSCO Discovery Solution (EDS)
  6. Include open access Open University audio-visual collections in this system and identify and document the issues in accessing this type of content on mobile devices
  7. Produce a mobile bookmarklet tool to significantly reduce the authentication barriers for mobile users.
  8. Evaluate the prototype system to ascertain the usability and quality of the user experience.
  9. Produce a ‘good practice’ toolkit for content providers, based on the experience of the project
  10. Disseminate the experience of the project though workshops, conference papers, journal articles and blogs.

The project will have been successful if we have

  1. Developed and tested a user-friendly mobile discovery interface which allows users to search for and read full text e-resources,
  2. Developed and tested an authentication bookmarklet which allows users to be signed in to subscription resources securely and seamlessly, with minimum effort on their part
  3. Shared what we learn from the experience through a ‘good practice’ toolkit for content providers.
Posted in Project management | Tagged , , | Comments Off on What is MACON?