Search focus groups

In parallel with the RISE project OU Library Services have been running an evaluation of the One-Stop search system now it has been in place for a few months.  So we’ve had a survey running and Duncan from our Learning and Teaching team has now run a couple of focus groups, one with undergraduates and the other with postgraduates.  As part of the focus group activities RISE asked the focus group team to raise the subject of whether they would find recommendations to be useful as one way of testing the project hypothesis.

One of the suspicions that we had was that there might well be different attitudes to recommendations based on the level they were studying at.  The team running the focus groups have now written them up and we have some initial feedback about what students had to say about the value of recommendations.   Thanks to the One-Stop evaluation team and particularly to Duncan for covering recommendations in the focus groups.

Undergraduates
The focus group comprised six undergraduate students, three studying level 1 courses, 3 at level 2.  Two had previously studied several modules up to level 3.  The students were studying a range of subjects.  The group were asked if they would make use of recommendations.  

There was a general consensus that ratings and reviews from other students would be beneficial (because ‘other people’s experiences are valuable’) especially if it was known which module the student leaving the rating had done, and how high a mark they had got for their module.

Postgraduates
This focus group was made up of five postgraduate students (one of which was also a member of staff) studying a range of different subjects through arts, science, social sciences and educational technology.  The main feedback was that:

• Students use citation information as a form of recommendation
• Students are wary of recommendations when they don’t know the recommender e.g. tutor recommendations are valued
• It was felt that recommendations specific to a module should be fed through to that module’s website e.g. for good databases
• Students would appreciate recommendations of synonyms when searching our collections e.g. stress/anxiety
• Resources from the institutional repository are trusted as authors can be contacted (this comment from a student who is also a member of staff)

Reflections on the comments in the focus groups
Knowing the provenance of a recommendation is clearly important and that seems to be a clear difference between academic recommendations and an ‘amazon-type’ purchasing recommendation.  There is a critical element of trust that is needed.  You could characterise it as ‘I don’t know whether to trust this information until I know more about who or where it comes from’  That implies a good level of academic caution about the quality of resource recommendations.  So that is possibly a qualification to our hypothesis

“That recommender systems can enhance the student experience in new generation e-resource discovery services”

‘Qual 1 … as long as it is clear where the recommendations come from and users have trust in their quality’

Another reflection is that there is a slightly different focus between undergraduates and postgraduates.  Undergraduates see quality as being represented by the success of students studying their modules, postgraduates see quality as being represented by recommendations being made by people they trust.

Pushing recommendations into module websites is an interesting idea.  There has been some discussion about methods of pushing tutor recommendations to students so this sounds like an area for further work at some stage.  The idea of a synonym tool that could provide suggestions of related terms that could help with searching is also quite a good idea.

Next steps
RISE will be running some invidual sessions with users over the next month or so to test out the tools that have been built and to do some more detailed work to help to understand the circumstances where recommendations about e-resources are of use and what type of recommendations are best.   Invitations are currently going out to a pool of students.