Graduate Outcomes
Data collected through the DLHE and Graduate Outcomes surveys are used to assess performance of providers for both access and participation and the TEF (subject to current review of the TEF). All providers can currently see data up to the 2016/17 academic year on the Office for Students dashboard relating to the progression of their students into highly skilled employment. This is calculated from the DLHE data, as the survey finished that year and has been replaced by the Graduate Outcomes survey, the key change for which is that graduates are contacted 18 months after completing their course instead of the previous period of 6 months for DLHE.
Performance Measures
The first experimental statistics for the 2017/18 cohort in Graduate Outcomes was released on Thursday 18th June. It “includes high level findings and characteristics of graduate outcomes at a national level. It provides an overview of the survey results with information about graduates’ activities, employment, salaries and their own assessments of success.” A more detailed set of statistics that includes some tables broken down by provider were released on the 23rd June, this is a more useful release as it will allow some comparison between providers.
Performance indicators will also be released but this has been delayed to Autumn 2020, and there are currently no details on what these performance indicators will be due to delays caused by COVID-19 measures. Although HESA will be publishing the numbers of graduates in highly skilled employment, they are not anticipating that there will be a performance indicator associated with this metric from them. However, the metric is likely to be published initially on Discover Uni, and the OfS are working with relevant groups to define how this will be defined.
The OfS has sector wide targets for Access and Participation, but none of these currently relate to progression (i.e. highly skilled employment). Some are expected to be published in winter 2020, and are likely to relate to:
· Graduates in highly skilled or professional roles
· Employers that think graduates are equipped with the right skills and knowledge
· Graduate wellbeing
Concerns with Graduate Outcomes Data
Over the course of the first Graduate Outcomes survey there were a number of concerns regarding the effectiveness of the collection methods and the coding of the graduates’ Standard Occupational Codes (SOC). Providers have had access to their Graduate Outcomes data since the end of March, with an update in May that had some changes to SOC codes after feedback from providers was considered. This data was provided in a spreadsheet that could be downloaded from the HESA portal, but is quite a large file containing a lot of different codes and detail. It is possible that some providers have not yet fully engaged with this dataset.
HESA have published guidelines on the acceptable usage of Graduate Outcomes data, including a piece about how the data should not be directly compared to the DLHE dataset, stating clearing that “one must not compare, for example, the proportion of graduates in employment or further study at the two reference points”. How this will change the access & participation dashboard is not yet clear, as adding in the data to the same charts as DLHE data would do just this.
Longitudinal Education Outcomes Data (LEO)
The longitudinal education outcomes (LEO) data brings together information from the Department for Education with employment, benefits and earnings information. This dataset is experimental as although there are some interesting aspects of it, there are concerns about the validity of the findings. The data does not include whether a graduate is working full-time or part-time, or the details of job they are doing, as these are not required by HMRC. The tax data for part-time workers, for example, does not indicate that they are working part-time. This means certain types of workers may have a lower income recorded in LEO. Additionally, it can be difficult to take into account differences in salaries that are caused by region, the background of the student or career aspirations. In particular, there are concerns that the dataset could be used to put pressure on providers to get rid of what some call low value degrees.
Teaching ExcellenCE and Student Outcomes Framework (TEF)
In the last TEF assessment, DLHE data was used to provide metrics for graduates in employment or further study, and those in highly skilled employment or further study. These were part of the core metrics (along with NSS data) that determined the initial award assessment. LEO data was provided as a supplementary metric, but many small providers may not have had this data available as they would need sufficient eligible graduates who graduated at least 5 years prior.
Splits of the data were available based on age, POLAR, IMD, Ethnicity, Disability, and Sex. These were provided not as raw data, but as a calculated indicator to show how far above or below they are to the benchmark. It was expected that differences from the whole provider population would be addressed in the submission. The data for subject level TEF was similar, but with problems for smaller providers if they did not have enough students in a subject to warrant much analysis.
The review of TEF conducted last year is still pending release, so we do not yet know how it will change in future, or even if it will run at all. If it does, there are unlikely to be many changes to the underlying datasets as they already use what is available, but the calculations of the metrics and indicators may change.