The Hidden Bias Of Artificial Intelligence In Indonesian Elt Assessment: A Mixed-Methods Study Of Grammarly And Turnitin Scores On Student Writing

Abstract

The growing use of AI tools in ELT assessment, like Grammarly and Turnitin, poses risks of algorithmic bias against non-native varieties of English. Although there is extensive research to show bias against Indian, Nigerian and Chinese English writing, no empirical studies about Indonesian English have been investigated nor isolated, comparing AI scores versus human rubric marking which allows culturally situated features like code-switching, politeness markers and more indirect rhetorical structures. This sequential explanatory mixed-methods study examined how Indonesian EFL writing features that are grounded in local culture were penalized by Grammarly and Turnitin, and what lecturers perceived about this bias. Presumably, Grammarly, Turnitin and the three lecturers were invited to score sixteen essays by six university students, then subsequently interviewed semi-structured style, a qualitative design. In the quantitative data, some strong negative bias was revealed: essays were penalized by Grammarly for larger -11.8 points (93.75%) of essays or Turnitin scores -6.9 points (87.5% of essays). The harshest penalties were levied for code-switching (Grammarly bias -15.2), while the moderate ones were reserved for politeness markers (-12.6) and indirect rhetorical structures (-8.9). Lecturers were in agreement against AI-only assessment, devising human-led, AI-assisted models and advocating for intelligibility as the standard reference point instead of native speaker reflective competence. The study concludes that existing AI writing assessment tools will not be appropriate for high-stakes Indonesian ELT assessment without adaptation because they devalue local linguistic identities through systematic ‘cultural erasure,’ as one lecturer called it.