Improving the evaluation of outreach

Understanding effective evaluation of the impact of outreach interventions on access to higher education: an institutional approach

Aims

This programme of work will help us to understand more about the most effective ways for universities and colleges to reach out to people from groups that are currently under-represented in higher education in England, and raise their aspirations and attainment so that they may apply to higher education.

It aims to:

Why do we need to do this?

Outreach plays an important role in widening participation by raising aspirations and attainment among potential students from disadvantaged backgrounds, so that they apply to go to higher education and have the grades needed to get in. Universities and colleges therefore invest substantial sums of money in outreach schemes (a total of £136.1 million in 2016-17 access agreements). OFFA, the government, and the institutions themselves, all want to make sure that money is being spent strategically for the maximum possible benefit. 

Some universities and colleges use robust evaluation to find out how much impact their outreach schemes are having, so they can they can take a strategic, evidence-led approach to choosing where to invest their time and money, focusing on the activities and ways of working that are most effective in their own context. But many others are currently not evaluating as effectively as we would like.

OFFA has challenged universities and colleges to improve their evaluation and this programme of work aims to support all universities and colleges in meeting that challenge. It will do this by helping them to understand more about effective evaluation and thus evaluate more effectively and more consistently across the whole higher education sector, taking into account each university or college’s specific context and aims. It is a key aspect of OFFA’s support for institutions to take an evidence-led approach to improving performance across the whole student lifecycle, as discussed in our strategic plan.

What has happened so far?

Understanding current evaluation practice of outreach for young people who are the first in their family to go to higher education

In August 2016 OFFA commissioned a team led by Dr Claire Crawford and Professor Robin Naylor from the University of Warwick to investigate how universities and colleges evaluate their outreach schemes, working in partnership with a group of universities and other key sector bodies.

Drawing on survey data collected from universities with an access agreement, along with interviews with widening participation staff in eight institutions, the team have developed proposed standards of evaluation practice which are designed to be usable across diverse institutions within a changing higher education landscape

Project team

Principal Investigator

Dr Claire Crawford, Assistant Professor of Economics, University of Warwick and Research Fellow, Institute for Fiscal Studies

Deputy Principal Investigator

Professor Robin Naylor, Professor of Economics, University of Warwick

Researchers

Dr Siobhan Dytham, Research Fellow in the Centre for Education Studies, University of Warwick

Dr Sally Hancock, Lecturer in Education, University of York

Dr Emma Uprichard, Reader, Centre for Interdisciplinary Methodologies, University of Warwick

Paul Blagburn, Head of Widening Participation and Outreach, University of Warwick

Professor Christina Hughes, Pro-Vice-Chancellor for Student Experience, Sheffield Hallam University

Understanding the impact of outreach activity for mature learners with low or no prior qualifications

Practical tools and publications

This stream of our programme of research into the impact of outreach has now concluded, and we have published:

We encourage universities and colleges to use the evaluation tool, and build this in to the ongoing development and delivery of their access agreements and wider institutional strategies.

Background

In September 2016, OFFA commissioned a team led by Dr John Butcher at The Open University to investigate how universities and colleges evaluate the impact of their outreach schemes on access to higher education for disadvantaged adult learners with low or no prior qualifications. 

In recent years, we have asked institutions to focus on mature learners as a particular target group. Research shows that mature students are more likely than younger students to be from an ethnic minority, hold non-traditional qualifications and have disabilities, among other measures of disadvantage, and this creates specific barriers for access to higher education. Many institutions are addressing this through outreach approaches specifically designed to reach mature students. Our aim in this project was to develop, in partnership with institutions, a common approach to evaluation which takes account of the particular challenges in determining the success of work with this relatively small group of students.  

The team for this project comprised researchers from The Open University, alongside researchers from Birkbeck College, the University of Bristol and the University of Leeds. Each institution took a distinctive approach to the brief, providing five case studies showcasing different approaches to outreach to adult learners. Their combined expertise covered areas including distance learning, pre-entry curricula, lifelong learning, evening degrees, and subject specific outreach.  

Project team

Principal Investigator

Dr John Butcher, Senior Lecturer and Deputy Director, Centre for Inclusion and Collaborative Partnerships, The Open University

Researchers – Centre for Inclusion and Collaborative Partnerships, The Open University

Wendy Fowle, Senior Manager, Research and Evaluation (Widening Access and Success)

Kate Hawkins, Research and Evaluation Officer

Partner institutions

Birkbeck College, University of London

Dr Kerry Harman, Programme Director, Higher Education Introductory Studies

University of Bristol

Tom Sperlinger, Reader in English Literature and Community Engagement

Zoe Pither, Student Support Manager, UK Student Recruitment

University of Leeds

Lindsey Fraser, Deputy Director (Partnerships), Lifelong Learning Centre

What is happening now?

Developing a common evaluation methodology for outreach activity for young people who are the first in their family to go to higher education

The proposed standards of evaluation practice developed in the first phase of this project now need to be tested by institutions and third party organisations who offer outreach schemes to ensure they can be applied across a variety of contexts. We have put out an invitation to tender (PDF) for a Principal Investigator to coordinate and lead this pilot testing. The deadline to tender for this work is noon Wednesday 12 July 2017.

Understanding the impact of outreach activity for mature learners with low or no prior qualifications

In September 2016, OFFA commissioned a team led by Dr John Butcher at The Open University to investigate how universities and colleges evaluate the impact of their outreach schemes on access to higher education for disadvantaged adult learners with low or no prior qualifications. 

In recent years, we have asked institutions to focus on mature learners as a particular target group. Research shows that mature students are more likely than younger students to be from an ethnic minority, hold non-traditional qualifications and have disabilities, among other measures of disadvantage, and this creates specific barriers for access to higher education. Many institutions are addressing this through outreach approaches specifically designed to reach mature students. Our aim in this project is to develop, in partnership with institutions, a common approach to evaluation which takes account of the particular challenges in determining the success of work with this relatively small group of students.  

The team for this project comprises researchers from The Open University, alongside researchers from Birkbeck College, the University of Bristol and the University of Leeds. Each institution will take a distinctive approach to the brief, with their combined expertise covering areas including distance learning, pre-entry curricula, lifelong learning, evening degrees, and subject specific outreach.  

 

Want more information?

Contact Richard Shiner, Head of Evidence and Effective Practice (0117 931 7171, richard.shiner@offa.org.uk