Improving the evaluation of outreach
Understanding effective evaluation of the impact of outreach interventions on access to higher education: an institutional approach
This programme of work will help us to understand more about the most effective ways for universities and colleges to reach out to people from groups that are currently under-represented in higher education in England, and raise their aspirations and attainment so that they may apply to higher education.
It aims to:
- improve understanding of the impact of outreach on access to higher education for students from disadvantaged groups
- improve evaluation practice across the higher education sector in England
- help OFFA make robust, evidence-led policy that supports and challenges institutions to identify and use the most effective approaches to improving access.
Why do we need to do this?
Outreach plays an important role in widening participation by raising aspirations and attainment among potential students from disadvantaged backgrounds, so that they apply to go to higher education and have the grades needed to get in. Universities and colleges therefore invest substantial sums of money in outreach schemes (a total of £136.1 million in 2016-17 access agreements). OFFA, the government, and the institutions themselves, all want to make sure that money is being spent strategically for the maximum possible benefit.
Some universities and colleges use robust evaluation to find out how much impact their outreach schemes are having, so they can they can take a strategic, evidence-led approach to choosing where to invest their time and money, focusing on the activities and ways of working that are most effective in their own context. But many others are currently not evaluating as effectively as we would like.
OFFA has challenged universities and colleges to improve their evaluation and this programme of work aims to support all universities and colleges in meeting that challenge. It will do this by helping them to understand more about effective evaluation and thus evaluate more effectively and more consistently across the whole higher education sector, taking into account each university or college’s specific context and aims. It is a key aspect of OFFA’s support for institutions to take an evidence-led approach to improving performance across the whole student lifecycle, as discussed in our strategic plan.
Understanding the impact of outreach activity for mature learners with low or no prior qualifications
Practical tools and publications
This stream of our programme of research into the impact of outreach has now concluded, and we have published:
- a report, which draws on five case studies of specific approaches to outreach for disadvantaged adult learners
- a three-step evaluation tool, which universities and colleges can use to inform the development of their outreach for adult learners
We encourage universities and colleges to use the evaluation tool, and build this in to the ongoing development and delivery of their access agreements and wider institutional strategies.
In September 2016, OFFA commissioned a team led by Dr John Butcher at The Open University to investigate how universities and colleges evaluate the impact of their outreach schemes on access to higher education for disadvantaged adult learners with low or no prior qualifications.
In recent years, we have asked institutions to focus on mature learners as a particular target group. Research shows that mature students are more likely than younger students to be from an ethnic minority, hold non-traditional qualifications and have disabilities, among other measures of disadvantage, and this creates specific barriers for access to higher education. Many institutions are addressing this through outreach approaches specifically designed to reach mature students. Our aim in this project was to develop, in partnership with institutions, a common approach to evaluation which takes account of the particular challenges in determining the success of work with this relatively small group of students.
The team for this project comprised researchers from The Open University, alongside researchers from Birkbeck College, the University of Bristol and the University of Leeds. Each institution took a distinctive approach to the brief, providing five case studies showcasing different approaches to outreach to adult learners. Their combined expertise covered areas including distance learning, pre-entry curricula, lifelong learning, evening degrees, and subject specific outreach.
Dr John Butcher, Senior Lecturer and Deputy Director, Centre for Inclusion and Collaborative Partnerships, The Open University
Researchers – Centre for Inclusion and Collaborative Partnerships, The Open University
Wendy Fowle, Senior Manager, Research and Evaluation (Widening Access and Success)
Kate Hawkins, Research and Evaluation Officer
Birkbeck College, University of London
Dr Kerry Harman, Programme Director, Higher Education Introductory Studies
University of Bristol
Tom Sperlinger, Reader in English Literature and Community Engagement
Zoe Pither, Student Support Manager, UK Student Recruitment
University of Leeds
Lindsey Fraser, Deputy Director (Partnerships), Lifelong Learning Centre
Understanding current evaluation practice of outreach for young people who are the first in their family to go to higher education
In August 2016 OFFA commissioned a team led by Dr Claire Crawford and Professor Robin Naylor from the University of Warwick to investigate how universities and colleges evaluate their outreach schemes, working in partnership with a group of universities and other key sector bodies.
Drawing on survey data collected from universities with an access agreement, along with interviews with widening participation staff in eight institutions, the team have developed proposed standards of evaluation practice which are designed to be usable across diverse institutions within a changing higher education landscape.
Dr Claire Crawford, Assistant Professor of Economics, University of Warwick and Research Fellow, Institute for Fiscal Studies
Deputy Principal Investigator
Professor Robin Naylor, Professor of Economics, University of Warwick
Dr Siobhan Dytham, Research Fellow in the Centre for Education Studies, University of Warwick
Dr Sally Hancock, Lecturer in Education, University of York
Dr Emma Uprichard, Reader, Centre for Interdisciplinary Methodologies, University of Warwick
Paul Blagburn, Head of Widening Participation and Outreach, University of Warwick
Professor Christina Hughes, Pro-Vice-Chancellor for Student Experience, Sheffield Hallam University
Testing the standards of evaluation practice in different contexts
The proposed standards of evaluation practice developed in the first phase of this project now need to be tested by institutions and third party organisations who offer outreach schemes to ensure they can be applied across a variety of contexts.
OFFA has commissioned a collaborative project, led by a team from the University of Exeter, to carry out this pilot test in a range of different case study sites. Working with a number of partners, the research team will investigate how the standards can be applied in different contexts. The work will involve:
- a series of workshops to reflect on the standards, and identify and implement evaluation methodologies for selected outreach interventions
- detailed work between partners and the research team to analyse the evaluation data, drawing on qualitative and quantitative approahces
- the development of conclusions and recommendations for practice, backed up by practical case study examples.
Professor Debra Myhill, Pro-Vice-Chancellor: College of Social Sciences and International Studies, University of Exeter
Associate Professor Anna Mountford-Zimdars, Director of the Centre for Excellence in Widening Participation Research and Practice, University of Exeter
Dr Pallavi Amitava Banerjee, Lecturer in Education, University of Exeter
Joanne Moore, Social Researcher, Aimhigher Research and Consultancy Network
University of Liverpool
University of Plymouth
Royal Northern College of Music
Third sector partners
The Access Project
The Sutton Trust
Want more information?
Contact Richard Shiner, Head of Evidence and Effective Practice (0117 931 7171, email@example.com)