MAT summary evaluations

Please see our recent update on MAT summary evaluations.

Summary inspections of multi-academy trusts

The scrutiny of multi-academy trust (MAT) performance by Ofsted has undergone some operational changes. The regime previously known as 'focused inspections' has been replaced by 'MAT summary evaluations'. In this note, we comment on the new approach and identify a number of key elements for MATs to reflect on.

Background

The practice of MATs themselves being subject to scrutiny by the inspectorate is not a new one. In 2015, 'a concerted programme of action to establish the effectiveness of multi-academy trusts in supporting and challenging academy schools within individual chains', known as focused inspections, was launched. Under that regime, Ofsted mainly concentrated on the largest MATs at the weakest and strongest ends of the spectrum in terms of performance but it was never entirely clear how those inspections should run or exactly how MATs would be selected for them because no guidance was ever issued.

Ofsted’s position now is that, having trialled the new approach, the revised system of summary inspections will improve and clarify the experience for all.

Interestingly, Ofsted has released an operational note for inspectors to support this new process. Whilst Ofsted has dropped the phrase 'inspection' the operational note will, in high level terms, look broadly familiar to the approach set out in the main inspections framework.

As with the predecessor regime, the Secretary of State has written to HM Chief Inspector to give the nod to the new process (available here), including emphasising the point that MATs are not being inspected in the strict sense of the word – it remains the case that there is no legal basis for Ofsted inspecting and grading the MAT entity independently under the inspection framework.

Summary evaluation – overview of process

As mentioned above, Ofsted has taken the opportunity to publish an 'operational note for inspectors' to support the new system (available here). Rather than summarise and repeat that note’s contents here, we share some key observations below.

There was no such guidance under the previous era of focused inspections and it is fair to say that the new document is welcome and does suggest greater transparency on process. It looks, perhaps not accidentally, like a mini framework and there is a sense that Ofsted is attempting to ease the continuing tension between its role in the formal inspection of academies and reviewing the direct part that MATs play in the quality of education.

At the core of the revised regime is a two-stage process. Under stage one, Ofsted will carry out 'batched inspections', inspecting under section five or section eight, as applicable, some of the individual academies within the MAT. Importantly, the new approach offers more flexibility around scheduling so that, although in some cases the inspections could take place within the same week, it is also possible for them to be stretched out over as long as two terms. Stage two occurs after the publication of those inspection reports – again, this is seen as a key improvement as it will enable the MAT to reflect properly on the findings from the batched inspections. The second stage will include meeting with MAT personnel, site visits to academies and telephone surveys with any academies not inspected under stage one. Ofsted draws on those findings to ultimately produce the summary evaluation letter which will follow a prescribed format and highlight areas of strength and improvement. The outcome is solely narrative as opposed to any graded judgement but will be published on Ofsted’s website.

Some observations

Away from the detail of the process which can be found in the 17 pages of the operational note, some interesting points are:

• Although Ofsted clearly retains its legal powers to inspect individual academies as part of stage one, Ofsted has no power to insist that MATs engage with summary evaluations under stage two. Similarly, the ability for inspectors to carry out site visits under stage two is at the sole discretion of the MAT (this was reinforced in the Secretary of State’s letter). MATs are also able to submit a deferral request if they have genuine concerns about timing. Refusing to co-operate with a summary evaluation without any sound basis would not be advisable but it is a reminder of the lack of legal status that the process does require MAT engagement.

• A MAT will not know if they will be asked to engage in the summary evaluation process until the end of stage one. Ofsted will take an initial decision to carry out the batched inspections but will then use its discretion and likely refer to the criteria listed in paragraph seven of the operational note to decide if they are proceeding with stage two.

• If a MAT is unhappy with any aspect of the summary evaluation process, it is able to complain in accordance with Ofsted’s usual complaints process. The fact that there is now a published process for these reviews perhaps makes the complaint procedure more tangible in this context as the operational note suggests certain standards and expectations. In addition, the MAT will have five days to review and comment on a draft of the summary evaluation letter before it is published, with the operational note suggesting that such comments do not have to be limited to points of factual accuracy.

• Ofsted plans to select a 'broad range' of MATs and will not solely be focusing on large underperforming groups. One of the common questions regarding focused inspections was how Ofsted made its selections but now the factors Ofsted will typically take into account are listed at paragraph 13 of the operational note. The list doesn’t give too much away; for example, there is no prescribed minimum size of MAT which is subject to summary evaluation. However, it seems clear that cause for concern (whether due to volume of complaints, significant decline in results, warning notices or safeguarding concerns) is likely to bump up the odds of a summary evaluation and local intelligence will still be relevant. In reality, the numbers of MATs that are subject to a summary evaluation will be small, we expect about 12 a year, but the low numbers also add gravitas to the process and bring into focus the value of self-scrutiny at MAT level. Interestingly, Ofsted’s letter on the first summary evaluation published made clear that the trust was selected because of the high number of its schools that were performing well. A focus on all sizes and performance levels should be welcomed and if delivered on should provide trusts with more opportunities to gain insights from this new regime.

How can you prepare?

Paragraph 39 of the operational note sets out a list of areas upon which inspectors are likely to concentrate and is certainly worth referring to. Paragraph 35 also highlights the type of evidence that will be relevant. Neither list is exhaustive. There is no expectation that evidence is prepared in a certain format for the purpose of the process but this is very clearly the MAT’s opportunity to present evidence beyond that which has been obtained by inspectors under stage one in order to demonstrate areas of focus, the impact the MAT is having across its academies and the overall quality of education.

Notwithstanding the new name and 'framework', there is still merit in looking back at Ofsted’s findings under the focused inspections regime. We know, for example, that Ofsted sees strength where MATs are able to identify their 'weaknesses', the solutions they have in place to tackle areas of improvement, how these solutions are being implemented and then subsequently monitored. When it comes to strategy, performance, structure and impact, there are various trigger questions which MATs may want to ask themselves in preparation. For example:

• What impact you are having on your schools – both in terms of promoting strong performance and tackling underperformance?
• Can you explain your governance structure and demonstrate that it is effective?
• How are you avoiding common governance pitfalls, such as poor communication and unclear relationships between the board, the executive and local governing bodies?
• How do you identify and deploy leadership expertise within your trust?
• What is your school improvement strategy?
• Do you understand the strengths and weaknesses of all your schools?
• Do your local governing bodies (or equivalent) value the trust and the support it provides?
• Are you a learning organisation? Can you demonstrate 'double-loop learning'?
• How do the MAT’s values translate meaningfully into practices at all levels?

These are merely a sample of considerations which you would encourage MATs to reflect on but demonstrate the point that ultimately Ofsted is continuing to review whether your MAT is an effective organisation.

Learning the lessons

As summary evaluations occur, we will continue to monitor the outcome letters to ensure any trends feed into our advice, consultancy services and training for MATs.

To date, only one summary evaluation has been published. In that case, the MAT was credited, amongst other areas, for:

• embedding its vision so that it guided decision-making at all levels of the MAT consistently. It was also praised for its work being firmly grounded in sustainable improvement.
• having a thorough understanding of the strengths and weaknesses of each its schools and using this understanding to focus improvement activity.
• having very effective HR, finance and estate management functions that closely supported schools to allow school leaders to retain focus on school improvement.
• being a MAT that schools want to join.
• consistency of approach on safeguarding across the schools with trustees regularly discussing safeguarding (analysing audits and considering any trends or areas of concern across the trust).

Areas of challenge were focussed on:

• pupil attendance across the MAT.
• ensuring that existing strong practice on providing support to disadvantaged pupils impacted right across the MAT.
• further embedding school improvement strategies to increase the rate of improvement in secondary schools (the MAT had 25 schools at the time of the evaluation, of which 22 were primaries).