Q&A: Will students be treated fairly under the State exams marking process?

Q. Are all students being treated fairly in the State exams marking process?
The State exams are generally seen as inherently fair in that every candidate is treated the same.

There have also been very positive changes towards greater transparency in recent times such as allowing students to see their marked exam papers.

However, an internal report produced by the State Examinations Commission (SEC) highlights key aspects of marking process where the issue of fairness is seriously questioned.

It flags concerns over the way marking schemes for individual exams are altered during the marking process to ensure consistency of grades.

Questions raised over State examinations marking process

---

Twenty women-only professorships to be established this year

---

‘First step’ towards free books for primary school children

---

Hundreds of childcare providers may shut for ‘day of protest’

---

This effect of this – known as the “bell-curve” – is to ensure a similar proportion of candidates achieve As, Bs and Cs (or H1, H2 and H3s, as they are now known) year after year.

This leads to marks being added to some selected questions or taken away from others, depending on whether the exam is too hard or too easy.

This affects students in different ways, benefiting some and penalising others.

For example, if your strongest area in an exam is the area targeted for fewer marks under the marking scheme, you may lose out more than others.

Q. So how, exactly, is this marking process flawed?
As the SEC's research paper states, this "standard setting" process is unfair or flawed because it:

(a) changes the overall placement of students and their grades, which "reduces fairness";
(b) diverts attention from a focus on accuracy to a focus on achieving a similar proportion of candidates get similar grades each year;
(c) renders the marking scheme less valid by awarding marks in the wrong proportions when compared to the original marking scheme.

Q. Why are grades manipulated in the first place? Why not just use candidates' actual results?
Exam authorities seek to ensure the level of difficulty of each exam paper remains the same every year.

However, this is impossible to do precisely without pre-testing exam questions (which we don't do in Ireland for reasons of security and cost).

If the exam was of the same level of difficulty, it is accepted that the proportion of H1s, H2s, etc, would be very similar year after year.

As a result, a “standard-setting” process is used in most jurisdictions to ensure results are similar year on year.

The method used in Ireland is to adjust the marking scheme for exam papers while they are being marked.

Q. How do I know if my grades have been altered by changes to the marking scheme?
In all likelihood you won't. Only the final marking scheme counts. The SEC has refused requests by The Irish Times to see how marking schemes have been altered in the past.

It has argued that giving public access to these changes would undermine public confidence in the exams.

(A small number of exam papers – about 5 per cent – that were used during a sampling exercise at the start of marking process will be marked under the old scheme – but this will be over-written by the final marking scheme).

Q. How often are marking schemes altered for exams?
Invariably, marking schemes are changed to ensure the "right" proportion of students score H1s, H2s and H3s, etc, to fit the bell curve.

The extent of these changes depends on how easy or hard the exam is compared to previous years. Sometimes major changes are needed; other times the changes are smaller.

Q. How many students are affected by changes to marking schemes each year?
In essence, every candidate taking a State exam is affected either directly or indirectly.

Let’s say there are two students, a and b. Under the original marking scheme, a student scores a H1 and student b scores a H2.

However, the altered marking scheme may divert points away from some questions if the exam is deemed too easy, or towards some if it is deemed to hard.

This may result in the two students switching positions, so student b ranks above student a.

Anything which changes this rank ordering of students has implications for every student taking the exam.

It is difficult to say exactly how many actual grades are changed, as the SEC does not release details of how marking schemes are altered. There’s no doubt, however, that significant numbers of grades are change (as this is the very reason that marking schemes are altered in the first place).

Q. If most countries alter students' grades, why should we concerned about what's happening in Ireland?
The Irish system is very unusual. The weighting attached to questions on exam papers is changed during the marking process in order to achieve the "right" proportion of grades (also known as the bell curve). This is a source of concern due to issues such as fairness and accuracy (see above).

Q. What approaches do other countries take to ensure grades are consistent?
In other countries, grade boundaries – in other words, the minimum marks needed to secure a H1, for example – may be shifted after the marking process is finished.

So, if the exam has proved too easy, authorities can nudge upwards or downwards the minimum marks needed to secure a H1.

This is simpler and affects all candidates.

Q. Are there any benefits to the current system we use in Ireland?
The SEC's report says the only benefits of maintaining the current process are that it has "the appearance of objectivity", is "familiar" and "may contribute to keeping the standard of the paper consistent".