Progress 8 in September 2017

Progress 8 in 2017

We use the Department for Educations Progress 8 to measure progress.  This measures the actual attainment 8 score against an estimated attainment 8 score.  The estimate is based on the pupils’ Key Stage 2 reading and maths scores combined. 

Attainment 8

Attainment 8 is made up from Maths and English (Highest from English Language or Literature), the three highest Ebacc exam grades and the three highest non Ebacc subjects count. The non Ebacc bucket can contain English Language or Literature or remaining Ebacc qualifications.  If a bucket/qualification is not filled it counts as zero. English and Maths are double weighted. It effectively becomes 10 buckets/qualifications.

All qualifications have to be approved by Ofqual. English and Maths have to be the new reformed qualifications.

In 2017 new points have been introduced.  The system has been designed to accommodate the transitional two years when GCSEs are awarded in both grades and numbers.

There is a significant difference in points that can be awarded.

In 2016 the highest attainment 8 score was 80

In 2017 the highest attainment 8 score can be 87.5

In 2018 it will go up to 90 for most pupils  but be 89 (approx.) for pupils who have unreformed subjects.

A pupil achieving E grades will have a lower attainment 8 outcome in 2017 form 2016.

In 2016 the score would be 30 points

In 2017 the score could be 25 points.  (The pupil needs to achieve a 3 in reformed GCSEs to achieve this.)

The outcomes for 2017 for reformed subjects were in line with 2016. 

Ofqual ensured that at G+/1+, C+/4+ and A+/7+ benchmarks the proportions achieving each based on prior attainment remained in line with 2016.

Progress 8

Progress 8 is the difference between a pupils estimated Attainment 8 and actual Attainment 8 divided by 10 (English and Maths is double weighted). The Attainment 8 estimates are based on the Key Stage 2 fine level.  In 2017 pupils took the Key Stage 2 in 2012 and the fine level is based on the average from their Reading and Maths Key Stage 2 Scores. This score is used to give attainment 8 estimates.

In the above example Gillian’s Key Stage 2 score gave her an estimated attainment 8 of 59.92.  She actually scored 67 points.  The difference is then divided by 10.

A progress 8 score of 0 is in line with national progress of pupils from a similar starting point. A score of +1 in 2016 represented progress on average for all the 8 subjects at one grade higher than expected.  As shown on page 1 the scoring has changed and 1 point does not now represent a grade for non-reformed subjects.  A pupil who achieved a B rather than a C scores 1.5 points, however a pupil who achieved an E rather than an F scores 0.5 points. This means that the impact of gaining a grade higher or lower at the top end is greater than at the lower end. 

Additional Impacts on Progress 8 2017

1.    2017 was the first of the Reformed GCSEs.  This has made predictions difficult as it is a new syllabus with a new points system.  In English the IGCSE was not approved by Ofqual,   therefor over 200,000 leaners moved to the reformed GCSE from this qualification.  The vast majority moved to AQA who saw there entries double in 2017. They were busy.

2.    To fit in line with Attainment 8 the number of entries for Science and Geography increased in 2017.  This will means the number of Ebacc buckets populated has increased and will impact the estimates.

3.    2017 is the last year that Ofqual have approved the ECDL qualification.  Ofsted have raised concerns about entering whole cohorts. Schools with Lower Prior attaining cohorts that have done this can add up to 0.2 to the progress 8 score.  The issue is if you have not done this then your scores will go down. The Attainment 8 estimates are based on comparisons to others. We do not have the number of entries but in 2016 it was 90,000.

4.    It is worth noting that in 2012 Key stage 2 scores were half a sub level higher than in 2011. This will also negatively impact any estimates calculated.

5.    SISRA and 4Matrix have provided schools with estimates based on the results that they received from schools.  These must be used with caution. Provisional attainment 8 estimates will be released at the end of September.

6.    The Department for Education’s Floor target is -0.5.  The Coasting score is -0.25, this is over a three year period.

IGCSE 2016

Last year the impact of IGCSE English results without any reporting tolerance added to them was significant on the overall outcomes for all learners.  In the summer action was taken by the great and good to ensure that IGCSE English outcomes were kept sensible though this had varying consequences for pupils and schools.  This year the problem has started early and our friends at the Dfe need to take a wee look at it.

Ofqual have helpfully as ever released this 
 https://www.gov.uk/government/statistics/november-gcse-and-level-1level-2-certificates-entries-in-England

I have asked Ofqual to provide the number of entries for the last 3 years for November IGCSE.  The Cambridge International Examinations Board refused to give me this information.  The claim at a PIXL conference was that schools were reporting an 80% pass rate from November IGCSEs.  It actually came in at 71.1%   These results were not compared to Keystage 2 results and were certainly not assessed against the new national reference tests.

Over 41,000 pupils from this Cohort have taken this exam in November.  This means almost 30,000 pupils have already passed the exam.  This will be approximately 8% of the cohort.  I do not know if the IGCSE will this year be subject to reporting tolerance in the summer of 2016 - it has to be if the system is to retain any hint of fairness.

You may be thinking that the November Series GCSE is for re-sits only but as Ofqual point out in the document.  "The re-sit only rule in November does not apply to Level 1/ Level 2 certificates, and students may still enter for a Level 1/ Level 2 certificate for the first time in November."

So what happens from here.  The pupils who sit GCSE English in the summer 2016 should be treated fairly.  They were not in 2015.  All exams should be made to go through the reported tolerance exercise.  This year the progress 8 calculation for schools has to be done twice -once including IGCSEs and once without.  It was so unfair that Raise on Line was published last year with absolutely no reference to the English GCSE farce of the Summer of 2015.

I await the response from the great and good of Education.

Chris Beeden

Ofqual 2015 Summer Exam Series report

Thank you to Ofqual for their latest report on the issues that took place last summer, published at the end of December.

https://www.gov.uk/government/publications/ofqual-2015-summer-exam-series-report

The following two paragraphs go some way to show the issues that took place,

"We asked Cambridge International to provide us with the evidence for how they had come to its awarding decisions in 2015. In setting standards in IGCSEs, Cambridge International use very similar evidence to the GCSE exam boards. Cambridge International considered predictions based on KS2 prior attainment and comparisons of the results for 'benchmark centres' - schools with stable entries for this syllabus in 2014 and 2015. Evidence from the benchmark centres suggested that to tighten grade standards as far as Cambridge International had intended (in relation to KS2 predictions) would have been too severe. 

The other factor that made awarding more challenging was the bunching of marks, particularly around the C/D boundaries. On one paper, the difference between C and A was only 4 marks. We concluded that Cambridge International had carried out its grading appropriately. Cambridge International has carried out further analysis since the summer and is confident that their grading was appropriate." 

Next I will look at the November 2015 entry pattern and results.

I presume but do not yet know that Ofqual will insist that Cambridge International have to provide progress tolerance statistics in 2016.

Ofqual's Response to English GCSE / IGCSE 2015

Many thanks to Ofual and here is the pass rates against expected for each board for English  2015

	Foundation		Higher
	Expected	Actual	Expected	Actual
AQA	38.5	38.5	82.5	81.1
OCR	35.6	33.6	83.89	84.36
WJEC	36	36.8	79.9	78.8
PEARSON	39	39.9	80.6	81.5
IGCSE	No reporting Tolerances Applied

Why has Pearson with a lower expected outcome come out with a higher pass rate than AQA?

To allow one set of Qualifications to have no reporting tolerance is just remarkable.  Schools have reported the Coursework for IGCSE was marked down but without the above information there is no way I can show what this blog hopped to achieve.  They did have a slight drop in the pass rate but I am as convinced as before that this would not have been reflected in the expected progress figure.

The new Ofsted data dashboard certainly looks like it will take prior attainment into consideration but I feel the table above still shows the results that it is based on can not be used to show a true reflection of all pupils progress.

Thanks for reading
Chris Beeden

The GCSE results are now public and thank you to JCQ for the following information.

http://www.jcq.org.uk/media-centre/news-releases/gcse-press-notice---uk

It states there has been a 2.5% increase in the pass rate for GCSE English for 16 Year olds,  This is higher than the 1% I was worried would be the case but not near the 7% that could have been expected given the 7% swap to IGCSE English. As stated I have asked Oqual to release the progress for each board for both GCSE and IGCSE.  I am still convinced there is no standard across the two types of exam.   I will publish this data when I receive it.

The Dfe must now check and then publish in Raise on Line all the progress measures including and excluding IGCSEs.  

Thank you for reading

Chris

English GCSE Results 2015 update

Having created my blog a couple of months ago the Cambridge IGCSE results have been published.  This shows a 1.1% drop in A* - C pass rate.  This is with a completely different cohort from 2014 as the numbers below show.  I am now very worried that the impact on the GCSE results next week will be significant.

	Number of candidates	Cumulative* percentages by grade
		A*	A	B	C	D	E	F	G
June 2015	201,858	3.4	12.4	31.2	63.0	85.7	94.7	97.3	98.2
June 2014	121,530	5.7	15.5	32.5	64.1	86.7	95.4	98.0	98.6

*Percentages are cumulative. For example, in June 2015, 63% of students achieved a grade C or above

http://www.cie.org.uk/news/news-details/view/cambridge-appeal-grows-for-schools-in-the-uk-11aug15/

I have made a Freedom of Information request to Ofqual for the KS2 to KS4 expected progress against actual outcomes for IGCSE and GCSE  English for all the individual exam Boards 2015 and 2014.  This will highlight any issues that may have taken place.  There are many many implications that then follow from this that I will return to.

Chris Beeden

GCSE English 2015

I have now analysed GCSE results for the best part of 20 years.  This was a fairly predictable job, generally a slight increase year on year (the headline was usually "Best Results Ever!"), until August 2012 when suddenly it all changed and without warning GSCE English went down by two or three percent.  Since that incredible day I have kept an eye on this situation.  In 2014 I believe that there was a one or two percent drop in GCSE English when looked at by progress from Keystage 2.  These may sound small percentages but a 1% drop is over 3,000 pupils.  

I have during this year become increasingly concerned about the 2015 GCSE results and the effect on the system that a substantial number of learners having switched to English IGCSE would have. Are we facing a potential 7% drop in the pass rate this year for GCSE English?  This Ofqual document shows that there has been a 7% drop in GCSE English entries for 2015 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/429275/2015-05-22-summer-exam-entries-gcses-level-1-2-certificates-AS-and-a-lev___.pdf

IGCSE’s it can be argued are more accessible for Pupils with lower prior attainments and 60% of the qualification is made up of speaking and listening and coursework. They are really well suited for many learners and it is fantastic that they provide a route to success for many pupils.

The Government sets Floor standard for Schools as the link shows https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/408259/Impact_indicator_17.pdf  I believe that the majority of learners that have switched to the IGCSE come from schools where this floor standard is a significant issue.  Put this another way I suspect that the majority of learners that have switched will have a lower prior attainment. I am not as yet able to prove this point.

The GCSE pass rate is approximately 70%. Exam boards are under huge political pressure to maintain a high standard and will be reluctant to increase the pass rate.  (The 2018 approved qualification list is yet to be published.)  If it remains at 70% then many learners who without the national switch over would have achieved a C grade will drop to D or lower.  Upscale this to all boards and it is 20,000 young lives hampered.   If the cohort that sit the exam have a higher prior attainment the expectancy is that the pass rate will go up by I am worried that exam boards will be very reluctant to do this. It will be a brave exam board that increases its pass rate by 7%.  Looking at it from the other point of view, if the IGCSE remains with a pass rate of 66% with a cohort of lower prior attainment is that fair?

The above has been on my mind, mentioned on tweets and mentioned to colleagues for the last few weeks.  I was so glad to see this Ofqual blog - 
https://ofqual.blog.gov.uk/2015/06/26/summer-2015-gcses-igcses-as-and-a-levels/
 The worries that I have had are being looked at by Mrs Stacey at Ofqual.  Having read this I then went through the links and came across the Data Exchange link.  This is a fascinating insight into what is happening to regulate GCSE outcomes – it is all very impressive.  I was though then worried when on page 14 it states for IGCSE “There are no reporting tolerances applied”. Is this fair?  I am new to looking at all this so please look yourself at how tolerances are applied or not. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/434414/Summer_2015_Data_Exchange_Procedures.pdf

Ofqual are certainly fully aware of this issue and the information that they are publishing is fantastic. This issue will be with us this year and next year but will not be there in 2017 when the new GCSE are taken.  I expect in 2016 the entry for IGCSE to go up even higher. How this though will be resolved is what now needs to be closely watched. Do the exam boards feel they can announce such large changes to pass rates?   For schools the solution is straightforward – publish two lots of progress figures one for GCSE and one for IGCSEs.  The main issue is fairness to the learners and it is how Ofqual ensure this that we need to keep our eye on.  Thank you to Ofqual for being so open and I really hope that you successfully resolve this situation in the next month.

Chris Beeden