Improving the evaluation of outreach
Understanding effective evaluation of the impact of outreach interventions on access to higher education: an institutional approach
This programme of work will help us to understand more about the most effective ways for universities and colleges to reach out to people from groups that are currently under-represented in higher education in England, and raise their aspirations and attainment so that they may apply to higher education.
It aims to:
- improve understanding of the impact of outreach on access to higher education for students from disadvantaged groups
- improve evaluation practice across the higher education sector in England
- help OFFA make robust, evidence-led policy that supports and challenges institutions to identify and use the most effective approaches to improving access.
Why do we need to do this?
Outreach plays an important role in widening participation by raising aspirations and attainment among potential students from disadvantaged backgrounds, so that they apply to go to higher education and have the grades needed to get in. Universities and colleges therefore invest substantial sums of money in outreach schemes (a total of £136.1 million in 2016-17 access agreements). OFFA, the government, and the institutions themselves, all want to make sure that money is being spent strategically for the maximum possible benefit.
Some universities and colleges use robust evaluation to find out how much impact their outreach schemes are having, so they can they can take a strategic, evidence-led approach to choosing where to invest their time and money, focusing on the activities and ways of working that are most effective in their own context. But many others are currently not evaluating as effectively as we would like.
OFFA has challenged universities and colleges to improve their evaluation and this programme of work aims to support all universities and colleges in meeting that challenge. It will do this by helping them to understand more about effective evaluation and thus evaluate more effectively and more consistently across the whole higher education sector, taking into account each university or college’s specific context and aims. It is a key aspect of OFFA’s support for institutions to take an evidence-led approach to improving performance across the whole student lifecycle, as discussed in our strategic plan.
What is happening now?
Developing a common evaluation methodology for outreach activity for young people who are the first in their family to go to higher education
In August 2016 OFFA commissioned a team led by Dr Claire Crawford from the University of Warwick to investigate how universities and colleges evaluate their outreach schemes, working in partnership with a group of universities and other key sector bodies.
Learning from the most effective examples, the team will recommend a set of practical solutions and approaches that would be effective for all institutions to use when collecting, reporting and interpreting data from their evaluation of their outreach schemes. These will be suitable for use across diverse institutions within a changing higher education landscape; we are not looking to impose a blanket ‘one size fits all’ process.
We are supporting this section of the project in partnership with the Sutton Trust.
Dr Claire Crawford, Assistant Professor of Economics, University of Warwick and Research Fellow, Institute for Fiscal Studies
Dr Emma Uprichard, Reader, Centre for Interdisciplinary Methodologies, University of Warwick
Professor Christina Hughes, Pro-Vice-Chancellor (Teaching and Learning), University of Warwick
Paul Blagburn, Head of Widening Participation and Outreach, University of Warwick
Dr Siobhan Dytham, Associate Fellow, Department for Sociology and Centre for Educational Studies, University of Warwick
Dr Sally Hancock, Lecturer in Education, University of York
Aimhigher West Midlands
University of Birmingham
Higher Education Access Tracker (HEAT) team
Nottingham Trent University
University of Sheffield
Developing a common evaluation methodology for outreach activity for mature learners with low or no prior qualifications
In September 2016, OFFA commissioned a team led by Dr John Butcher at The Open University to investigate how universities and colleges evaluate the impact of their outreach schemes on access to higher education for disadvantaged adult learners with low or no prior qualifications.
In recent years, we have asked institutions to focus on mature learners as a particular target group. Research shows that mature students are more likely than younger students to be from an ethnic minority, hold non-traditional qualifications and have disabilities, among other measures of disadvantage, and this creates specific barriers for access to higher education. Many institutions are addressing this through outreach approaches specifically designed to reach mature students. Our aim in this project is to develop, in partnership with institutions, a common approach to evaluation which takes account of the particular challenges in determining the success of work with this relatively small group of students.
The team for this project comprises researchers from The Open University, alongside researchers from Birkbeck College, the University of Bristol and the University of Leeds. Each institution will take a distinctive approach to the brief, with their combined expertise covering areas including distance learning, pre-entry curricula, lifelong learning, evening degrees, and subject specific outreach.
Dr John Butcher, Senior Lecturer and Deputy Director, Centre for Inclusion and Collaborative Partnerships, The Open University
Researchers – Centre for Inclusion and Collaborative Partnerships, The Open University
Wendy Fowle, Senior Manager, Research and Evaluation (Widening Access and Success)
Kate Hawkins, Research and Evaluation Officer
Dr Kerry Harman, Programme Director, Higher Education Introductory Studies
University of Bristol
Tom Sperlinger, Reader in English Literature and Community Engagement
Zoe Pither, Student Support Manager, UK Student Recruitment
University of Leeds
Lindsey Fraser, Deputy Director (Partnerships), Lifelong Learning Centre
What will happen next?
Practical tools and publications
At the end of each stream of this programme of work we will publish:
- a common set of measures for data collection and reporting
- an evaluation methodology that higher education providers can use, with a guide to help them do so
- a technical report setting out the processes of how this work was developed, key principles for developing evaluation of outreach, and institutional learning points.
At this point, universities and colleges may start using the common evaluation methodology if they wish.
We hope we will then be able to run pilot phases in which groups of higher education providers will try out using the common methodology. This will depend on the availability of further funding at that time.
We plan to carry out similar projects for outreach with other under-represented groups, including black and minority ethnic (BME) students.
Want more information?
Contact Richard Shiner, Head of Evidence and Effective Practice (0117 931 7171, firstname.lastname@example.org)