The Office for Fair Access closed at the end of 31 March 2018 and responsibility for higher education access regulation transferred to the Office for Students

Financial support evaluation toolkit

Why do you need to evaluate your financial support?

In 2015-16, financial support accounted for £448 million (in cash terms), representing 62 per cent of higher education providers’ total access agreement expenditure.  Institutions have a duty to invest this money effectively and in an evidence-led way to improve outcomes for under-represented and disadvantaged students.

Where you have committed significant resource to financial support, OFFA have required you to provide strong evidence of how your financial support will help to improve outcomes for under-represented and disadvantaged students.  Institutions with substantial expenditure on financial support should be led by a range of evidence that examines impact on students’ behaviour (such as improved access, retention, degree attainment and graduate outcomes) combined with evidence that reviews student experiences of how financial support affects their outcomes.  Institutions are expected to take a robust approach to evaluating financial support to provide this evidence, although the approach may be different in diverse institutional contexts. 

What are the tools in this toolkit?

To support higher education providers in evidencing the impact of their financial support, OFFA commissioned research that developed and piloted three evaluation tools for institutions to use in evaluating their own financial support provision. The tools are now brought together in this toolkit, and facilitate different approaches to financial support evaluation to help you understand the impact of your expenditure.

The tools are:

Which tools should you use?

Most institutions will need to use all of the evaluation tools to ensure their financial support is informed by robust evidence. 

The survey and interview tools can be used by any institution, with any number of financial support recipients. 

The statistical tool is suitable for institutions with more than around 300 recipients per year.  For the statistical tool, institutions with over 800 financial support recipients should have data readily available annually to support their statistical analysis. Institutions with between around 300-800 financial support recipients per year will need to combine data from multiple years to increase their sample size and achieve meaningful evaluation.   

Developing or changing the tools

We strongly encourage you to use the evaluation toolkit resources as they are, although you may use other methods or work with experts to further develop the tools if you can provide evidence that these methods are robust.  These tools were developed by experts through a process of multiple pilots and cognitive testing to develop their robustness. 

Before you start

Understand what skills, expertise and resources are needed

It is likely you will need to draw upon various staff and expertise within your institution to complete an effective evaluation. The guidance for using each specific tool includes a summary of skills and resources you are likely to need.

Understand the strategic role of your financial support

You should use evidence to inform your strategic decisions of how your investment in financial support will improve access, student success and/or progression of under-represented and disadvantaged groups at your institution. The purpose of evaluating is to find out whether and how your financial support for students is having the impact you intend.

Before you begin the evaluation, you will need to clearly understand the nature of your financial support packages and their aims (i.e. what impact you intend these to achieve). 

Most financial support packages aim to ‘level the playing field’ for disadvantaged young people to access and experience higher education. You may have a variety of aims and objectives for your financial support related to different groups of students’ needs, and different support packages.

Understanding the intended impact will give you the objectives you can evaluate against, and enable you to effectively analyse and interpret findings.

Using the tools

Implementation guidance and downloadable resources are available for each tool:

Interpreting your findings and next steps

Interpreting what your findings mean

Your findings should be interpreted in the unique context of your own institution and informed by available evidence from evaluation and research. We recommend that you involve your key stakeholders in this process (e.g. practitioners, financial advisers, experienced researchers, student representatives, and strategic decision makers). 

Some questions that could assist facilitating the interpretation process are:

  1. What have you learned about how your financial support is working and the impact it has on student outcomes?
    1. Is it having the intended impact?
    2. What should you keep doing?
    3. What should you do more of?
    4. What should you do less of?
  2. How can you share and understand the findings with others at your institution who have a stake in effective financial support?
  3. How do these findings confirm, develop or challenge your understanding from previous evaluations or research?
  4. What possible changes could you make that are likely to improve impact?
  5. How would different groups be affected if you redirected financial support?
  6. Is your financial support an effective use of money to achieve the desired impact?
  7. What new questions are raised to drive future evaluation or research?

Next steps: planning future evaluations

We recommend these evaluation tools are used annually, to feed into your regulatory reporting and strategic planning cycle for widening participation. Timely evaluation is particularly important to guide your strategic planning where the nature of your financial support changes, or if the environment changes significantly.

Related resources

Detailed guidance on how to design your financial support and tell us about it in your access agreement

Topic briefing on financial support

Understanding the impact of institutional financial support: OFFA commissioned research