OER quality checklists

open educational resources – training

OER Evaluation Matrix

OER Evaluation Matrix CC BY SA


Authors:                                Level:

Dr Vivien Rolfe                             OER community

OER Features:

Guides to checking the QUALITY of OER.

OER Description:

The UK OER Toolkit talks about quality considerations. Institutions will be concerned about the quality of OER released from a reputation perspective. Staff (and students) will feel unsure that their materials will be good enough to share to a community to reuse. In a series of interviews with senior university staff, the quality of OERs were referred to several times:

How could it be ensured that OER released by a university was high quality and met learner needs?”“The traditional academic quality processes govern the quality of awards, not content.” “How would OERs released remain current and reflect the quality of information required by professional bodies?” (2012-Rolfe-and-Fowler-How_institutional_culture_can_change).

From student perspectives, in interviews with learners in a health and life science faculty, quality was not identified so strongly, but what was highly valued were resources that were easy to access and clear to scan:

“I think it does because the longer I spend finding it, I tend to lose my concentration and think about what I will do next.” “…the thing is, you know, when trudging through books for example, you never really know what you’re going to get without reading the index or the content and you still don’t know what you’re going to get until you’ve re-read that thing. Whereas on the internet or online you can just scan and have a quick look through and pick out so many key words for what you’re actually looking for and get the right research that you need.” (2013 Libor Hurt_Student perceptions of OER).

How is ‘quality’ defined?

There are many features of an OER that may influence the perception of quality – as above, ease of availability and discoverability, accuracy of content, production values, reputation gain. Quality will vary on the stakeholder – those reusing OER might select OER based upon different criteria – accuracy of content, being a ‘stand-alone’ resource or ease of repurposing and remixing.

What are the main features?

As outlined in the UK OER Toolkit, as with any learning resource, authors should consider:

  • Accuracy
  • Reputation of author/institution
  • Standard of technical production
  • Accessibility
  • Fitness for purpose

So, how to check your OERs before release? Authors as gatekeepers of quality.

This simple checklist adapted from the Open University SCORE resources (original version CC-BY 2010) is a useful ‘pre-release’ guide. I’ve updated it to include a check on the inclusion of patient and medial data, to consider accessibility by diverse users by releasing in multiple file formats, and to encourage sustainability by releasing OER to at least two different locations.

OU SCORE OER Checklist_Adapted_CCBY_2015 (Download checklist document)

Students as gatekeepers of quality.

It is obvious that we all gather information from the internet to support our learning in a chaotic and unregulated manner. In one of our UKOER projects, a group of students arranged focus groups to understand how fellow students search for, and critically evaluate, web-based materials. They evolved an ‘OER Evaluation Matrix’ which a series of basic checks that a student can make to judge the quality of any multimedia resource – not just those openly licensed.

OEREM_Version1_26Sept2012 (Editable Word Document)

OEREM_Version1_26Sept2012 (PDF File)

OEREM Version 1 by Towlson and Hurt (2012) based on Leigh, Mathers and Towlson (2009)


Technorati Tags: ,

Related posts:

Patient consent and considerations for health and medical OER

open educational resources – training


Authors:                                Level:

Dr Vivien Rolfe                             OER community

OER Features:

Guide to creating health and medical OER.

OER Description:

As part of the UKOER Programme (2009-2012) there was a wealth of important information produced, and in the context of creating health OER, the Medev Subject Centre team worked on some outstanding projects. I hope over the next few weeks that many of the OERs produced by this team can be hosted on this blog, as the original URLs to the subject centre pages are no longer valid. However, here are some resources to get you started in considering some of the additional checks that need to be made when creating OER involving health and medical information, and that might require cooperation from patients and families.

The team comprised Suzanne Hardy, Megan Quentin-Baxter and Gillian Brown who worked for the subject centre and were based at the University of Newcastle. You can hear them talking about the key projects on YouTube.

OER Projects

OOER Organising OERs – (Pilot Phase)
PORSHA Pathways to Open Resource Sharing (Phase 2)

During these projects when the team were working in the creation of health OERs, they observed challenges in working across NHS and academic settings, not simply in terms of dual-location of students, staff often with more than one job role, but the simple sharing of resources across separate IT networks was a problem. The team also started thinking about the issue of consent. The involvement of patients in clinical education was well established, as it is today, but the notion of recording and releasing patient materials openly gave rise to new questions. Any resource would have the issues of IPR and copyright to consider, but additional checks would have to ensure compliance with Schedule 2 of the Data Protection Act where consent is given by the ‘data subject’ to the processing of that data.

There are other scenarios that require consideration relating to ethical questions. Patients are protected by the Data Protection Act, but what when they are no longer patients? What if they die? What about cadaveric materials?

Here is some further reading on the subject.

HEA Medev S Hardy_Medical OER_2009 (PPT slides from a talk by Suzanne Hardy)

OER IPR Support Kit and Web2Rights LTD (Support kits for IPR – diagrams, charts and videos)

GMC guidance on audio and visual of patients (General Medical Council guidance)

Medev ‘Risk Kit’ – risks and consent. (Hopefully to be located here soon).


The reuse of OER in health and life sciences: a check-up

Using health OER (Download PDF guide on “the reuse of OER in health and life sciences”)

By Vivien Rolfe, Jacqui Williams and Richard Windle. This guide talks through some of the author experiences of creating and using health OER. In the UK alone, health is one of the largest subject sectors, with one fifth of students studying Medicine, Biological and Veterinary Sciences. A further one in ten study Nursing, Midwifery, Pharmacy and biomedical sciences. THE SCOPE FOR SHARING TEACHING MATERIALS IS VAST BUT YET TO BE REALISED.

The recommendations from this work were to:

  1. develop partnerships and communities around repositories of resources for health OER to be more widely adopted.
  2. to take the OER where the ‘real world’ users are likely to be interested in them. There is a barrier to the academic and technical language of universities. “We must work with approaches and routes of access that these communities are conversant with and that promote confidence”.

Technorati Tags: , , ,

Related posts:

Researching OER impact

open educational resources – training

Sickle Cell Disease

Authors:                                Level:

Dr Vivien Rolfe                             OER community

OER Features:

Researching OER!

OER Description:

It is lovely to see that in 2015, three years after the end of the HEFCE-funded UKOER programme, open education is becoming mainstream in education and with a number of high-impact OER projects in existence. The community are developing OERs, open courses and influencing institutional policy. A number of researchers have attempted to define and evaluate the impact of these OER projects, and this information is providing vital messages to reach and influence further corners of the education sector. This blog post will provide links to OER research tools, including openly licensed surveys and questionnaires that are available for reuse!

Open University OER Research Hub

The Open University’s ‘OER Research Hub’ team are leading the way in harvesting existing OER studies onto a global impact map. They have also defined 11 impact hypotheses for researchers to consider when evaluating the benefits of OER activities and projects. The OER Research Hub web pages also contain useful information on ethics for the increasing number of people now researching ‘openly’ on the web, outside he jurisdiction of any institution.

What is research and what is evaluation?

In terms of approaches required, if someone is setting out to explore the benefits or impact of their work, it is important to distinguish between evaluation and research. One grows from the other, and a simple evaluation study can provide the pilot data or starting point for a more robust programme of study. The most important point to make is that within universities, there may be a lack of clarity as to whether ethical approval is required. The danger for someone producing exciting data from a small evaluation is they may then wish to publish it but not have the consent to do so in place.


  • Testing the benefits of an ‘intervention’ (e.g. control versus test group; pre-intervention versus post-intervention comparison).
  • Formative evaluation will improve classroom practice, or the progress of a project.
  • Limited in the number of research methods employed, e.g. interviews or a questionnaire.
  • May inform internal changes.
  • May fall into the trap of being externally published without ethical consideration.


  • Research will test hypotheses and theories.
  • May employ a number of methods to cross-reference and validate results.
  • Be more extensive and generalisable to a wider population.
  • Externally published / presented at conference.
  • Will require ethical approval.

Openly licensed survey instruments

The ‘Reusable Learning Object Centre for Excellence in Teaching and Learning’ (RLO CETL) has a wealth of superb materials, from templates for preparing high quality multimedia resources, to checklists and ideas for evaluation and research. The following document includes focus-group, interview and questionnaire questions that can be reused and adapted.

RLO CETL Evaluation Toolkit_CCBYNCSA_2005 (questionnaire and interview questions)

OER Community of User Perception Survey (questionnaire questions)

OER Open Learner User Survey (questionnaire questions)

OER Staff Awareness and Perception Survey (questionnaire questions)

OER University Student Awarenss and Use Survey (questionnaire questions)


Technorati Tags: ,

Related posts: