Each year, around one million students sit the University Entrance Test in Indonesia in order to be accepted into courses at public universities across the country.
Students sit the computer-based assessment online, and are provided with their results 10 days after their test date.
The test is run by the University Entrance Test Institution under the Ministry of Research, Technology and Higher Education. We spoke to psychometrician Dr Urip Purwono from the Centre for the Study of Physchometrics, Assessment, and Evaluation, Padjadjaran University, Indonesia, to find out more about how they overcome the challenges of delivering such a large-scale online test.
Developing a large-scale assessment
When developing the University Entrance Test, Purwono says they looked at what the best predictors of college success would be to decide what the test would assess.
‘So, we delved into the literature and we believe that there are two elements that would make a best predictor for college success – the cognitive ability and achievement,' he tells Teacher.
The breakdown of the assessment into these two elements means students are encouraged to study for the achievement section of the test – which Purwono describes as a test of their knowledge of the subjects they've studied at school – but do not need to study for the cognitive ability section. Explaining the reason why they don't need to study this section, Purwono likens the cognitive ability element of the assessment to an IQ test.
Once the students sit the test, they only need to wait 10 days to receive their results. Purwono explains that students are given eight or nine separate scores: four to five scores for subject achievement (Mathematics, Physics, Biology and Chemistry if the student sits the Science and Technology test; and Mathematics, Geography, Sociology, History and Economics for the Social and Humanities test) and four scores for cognitive ability, which is broken down into fluid reasoning, mathematical knowledge, comprehension and reading and writing.
Assessing millions of children online
The University Entrance Test underwent significant changes between the 2018 and 2019 test cycles, such as giving students the opportunity to choose when they sit the test.
‘In 2018, everybody who applied to a public university had to take this test on the same day at the same hours, regardless of where they were located,' Purwono says. ‘In 2019… they had an option to choose [from] 24 sessions available.'
The test is now rolled out across these 24 separate sittings, which take place in the morning and afternoon each weekend for six weeks. Because of this, Purwono says test developers need to be constantly accounting for a particular type of cheating that may arise – students memorising test questions and passing this information on to those students yet to sit the test.
‘Some people may be taking the test not because … they really want to get into college, but because they want to memorise some of the items so they can rebuild the test,' he shares, adding these people might then go on to sell the test, or tutor students before they sit it.
Test developers are aware of this issue and are taking active measures to prevent this form of cheating.
To best ensure test security, the assessments aren't available to be completely downloaded onto the computers at the test centres until one hour before the test is due to begin. ‘Before that – several days before, for example – they can download only the operating system and the application,' Purwono explains.
In 2019 the test was rolled out without any major challenges. In the future, there are plans to open up the test to be used for student entrance into private universities, and to allow students to choose which year in high school they sit the test.
Editor's note: This article was updated on 15 October, 2019 to clarify the details of the University Entrance Test Institution.
