Theatre from the cracks in the psyche of culture










No. 69 : December 2014

Is it ethical to inflate a student's score in relation to another student's score so as to ensure a statistical equilibrium; thus artificially enhancing one student's future prospects while disadvantaging another?

In a much praised and vaunted education system in the ACT, there is statistical insanity that makes gross assumptions that would shock most people. When your son or your daughter receives a score on a report card in years eleven or twelve, one might have assumed quite wrongly that it was a direct reflection of their work and ability in relation to the subject matter and in relation to other students' performances.

... And you would be wrong! 

It is not the result of the above, but rather the result of "z score weighting" dependent on the mean score of a group or cohort; such scores being dependent on the size of a group and vulnerable to the initial balanced eco-system of estimated student capabilities. In other words, while achievement is necessary to gain a footing on the page, one's worth is measured solely in terms of a rank order; determined less by actual grading by teachers than by statistical sorting by the statistician who seeks probable and likely outcomes measured by complex formulas.

Knowing how the system works, teachers are encouraged to "ensure" that top students are protected. Somehow, a spread of students is to be gained by differentiating within assessment tasks. The problem then arises when a number of students display similar levels of ability at the top end of a small cohort which often can happen in the Arts faculties. The statistician will argue that anomalies will be sorted with the moderating influence of the AST exam. However, no statistic can give an accurate indication of a particular person's score; just as no political polls can say exactly how any particular person voted. So with this in mind, schools encourage teachers to fairly early on identify who is the top student and to then make sure that person is statistically immune to variations in the mean that might occur due to moderating influences. But what if the status quo is upset and another student emerges from seemingly no-where to converge on the top place and even over-take it. What if a particularly strong student is doing a double major within the same cohort and is ostensibly competing with him / her self. Such events risk the easy statistical values given to student achievement and leave the school or cohort open to accusations of not having tasks with differentiated levels embedded within them.

But perhaps the worst and most despicable element in the whole process is the rank order competition and the divisive nature of setting student against student in achieving supposed educational outcomes. The notion of co-operative learning between teachers and students makes no sense at all in a system that builds in rank order scaling of all scores to determine student results. This is especially the case when that rank order is NOT determined by skill or knowledge within subject areas but rather by statistical manipulation which is claimed to be beyond challenge. So the temptation in a school and within a department or a cohort is then to artificially inflate the score of the top student or students (in a larger cohort) thus advantaging one student over the other.

And if my analysis of the situation is to be challenged, I posit that a student in a cohort of 25 students who achieves a score of 85% might well achieve a score of 95% in a larger cohort of students where a better z score might be achieved. Can the statistician in the ACT system tell me with a straight face and with certainty that this is NOT the case?

 Is this fair?

Well, to be fair to a system one might consider reasons why the formulators of it consider it the most fair in an imperfect world. To think about this, let us look at The Rosenham Study on Being Sane in Insane Places also for a critical look check it out here. You may wonder how a psychology experiment used to expose the fallacy of judgement in deciding what was sanity and insanity has to do with our much flawed statistical methods of judging rank orders in student achievement. Go back to 'game theories' and dominant ideologies instructing US and UK governments with total belief in statistical data as the defining means for determining progress. Certainly within an Educational context, teacher judgement, like that of the psychiatrists in the Rosenham Study, is seen as less reliable than statistical formulas. The use of extensive check-lists in psychology has seen the invention of mental disorders to replace normal human emotions while seemingly objectifying the data used for the diagnosis of illness; all to the greatest benefit of large drug companies. Flawed subjective diagnosis has made way for a wider and potentially even more flawed system of labelling behaviours as "disorders". This is done on the basis of a check-list of behaviours and invented names for a seemingly endless list of mental disorders. The result is potentially socially devastating.  In Education, the use of "league tables", NAPLAN (in Australia) and the reduction of everything to seemingly measurable outcomes has led to a narrow focus on literacy and numeracy as being the determinant of success in educational instruction and management. In the ACT in Australia, this focus has led to an exam being used to moderate a cohort of students against another. It's value is actually dependent on the notion of an equilibrium existing around a mean score within an individual subject/or subject cluster cohort then within school and then within schools drawn from the whole of the Australian Capital Territory. While movement of student scores is possible within this equilibrium, the over-all intellectual eco-system remains the same. In many respects it bears the hallmark of the Nash Equilibrium while ignoring the knowledge ecology within educational organizations, communities and circumstances.

Some application of recent work on "knowledge ecology" within a national, community and school based application might indicate some of the findings and observations made by Pasi Sahlberg, the Finnish Educator who points to the dangers of the Global Education Reform Movement (GERM) which began somewhere in the 1980s.

A key element in this has been in the development of consequential accountability through standardised tests using primarily maths and literacy as the starting point.  Sahlberg gives a good illustration of how this has affected policy in the United States :

"Perhaps the best-known practical illustration of large-scale education reform driven by the notion of standardization and related consequential accountability is found in the USA, where controversial federal legislation termed No Child Left Behind (Public Law 107-110) links school and teacher performance to Adequate Yearly Progress and to financial and resource allocations to schools (Popham, 2004; Centre on Education Policy, 2006). Recent research, however, suggests that 'the ability of standardized tests to accurately reflect school performance remains in doubt' (Lemke et al., 2006, p. 246). Furthermore, Amrein and Berliner (2002) concluded, on the basis of their analysis across 18 states in the USA, that since clear evidence was not found for the positive impact of high-stakes testing policies upon increased student learning and because there are numerous reports of unintended consequences associated with these policies, such as increased student drop-out rates, teacher and school cheating on exams, and teacher defection from the profession, there is need for transforming existing high-stakes testing policies."

(Also quoted in: New Directions for school leadership and the teaching profession: DEECD discussion paper, June 2012 -  A Response by Martina Golding, Performing Arts at South Geelong Primary School.

This point is also made by Adam Curtis in his documentary THE TRAP part 2)

The ACT began a system in the 1970s that was the opposite of that proposed by GERM. In the past decade it has been slowly drawn into the statistical games theories reflecting the supposed isolation and personal self-interest theories of John Nash and the resultant discrediting of teacher judgement as an indicator of student achievement ranked against other students.

The reliance on statistical data as an indicator has led to the opposite of what it sought to achieve. Instead of removing teacher judgement from the equation, it has artificially led to the increase in teacher/school manipulation of statistical data to get the results they wanted. This is exactly what schools do by limiting those who sit for the moderating exams (the AST) or NAPLAN (where record numbers of students are being exempted from sitting) and where inaccurate pictures of school achievements are being created. It leads to further inaccuracies by selective schools determining who will actually be accepted into the school. It reinforces those areas where wealth determines the composition of school communities. And worst of all, the statistical determination of success reinforces the teaching to exams and the consequent alienation of students from effective and meaningful learning.

Just as the proliferation of  Mental "disorders" has become significant in psychology, so too has the effective wrangling of the system by newly skilled applications of  such "disorder" thinking to gain rank order for University entrance become the big phenomenon of 2014/15 Education. Rather than encouraging intrinsic learning the application of skill to life and creativity, the statistical nightmare of our contemporary system is doing the exact opposite: forcing schools, students and communities into a cynical games play where everything is on the table to assist favourable rank order.

Questions of fairness no longer come into play; rather we focus on what might "advantage" or "disadvantage" students as an "accountability" issue rather than as a question for learning.

. . . And a question I pose to those who dispute this thesis: "Two students in two different schools with very different sized cohorts in a given subject like Maths! Both score 100% on all of the tests and assessment tasks which were the same for both schools. Can the ACT system guarantee that both students would score the same in the final outcome?"



References are generally linked in the article on 19 - 20 December 2014

PS: Have a look at an alternative to the "pathologizing of childhood" model used in Australia and eminating from America HERE.


Trinculo's Shadow

Receive our Trinculo's Shadow newsletter directly by subscribing to us. Read information about issues and events from local, national and international groups who:

  • use theatre to explore the human condition

  • venture into the very psyche of cultural and personal expression, and

  • are interested in exploring the mysteries of life and social/cultural interaction.

To receive your copy and read provocative theatre articles and receive information about scripts, workshops and performances from Shadow House PITS, please sign up here as a subscriber.

To subscribe, click here and complete the simple form.