Some Results of the Main NMT Session

Participants of the main sessions of the National Multi-subject Test (NMT) have already been able to create electronic offices and apply for admission to their desired higher education institutions, so it's time to summarize the first results of the tests.

The main NMT sessions were held from June 07 to June 20. In addition to Ukraine, the tests were held in almost all European countries, as well as in Azerbaijan, Turkey, Canada and the United States - a total of 47 cities in 30 countries. To ensure 10 days of assessments, 670 temporary testing centers were used in Ukraine and 62 abroad. 260,381 participants received their results, which is more than 93% of those registered.

What are these results? We will analyze the NMT-2023 test forms in detail in a report to be published on our website in the fall, but today we can announce the results of the main NMT sessions in compulsory subjects.

As we can see, only 253 applicants failed the Ukrainian language test (less than 0.1% of 260,381 participants).  In contrast, 13,760 participants (more than 5%) received results above 180 points on a scale of 100-200.

10,755 applicants (approximately 4%) did not provide correct answers to the required number of math tasks, while 8,475 (slightly more than 3%) received high results (181-200 points).

Thus, the testing within the main sessions ensured proper distribution of participants by rating scores, which will facilitate access to the desired universities depending on their objective results. Unfortunately, there are isolated cases when students who were successful at school received low NMT results. This could be due to excessive psychological stress and anxiety. Such examples cannot be used as a systemic sign, but instead we should try to study the reasons for each situation to help applicants in the following years.

As for the test materials used to determine the results of the participants, it should be noted that despite the fact that the Ukrainian Center for Educational Quality Assessment had to develop a significant number of test forms in a short time, and despite the impossibility of conducting full-fledged testing of test tasks in the current difficult circumstances, the various test forms used during the main sessions are as close as possible in complexity. This is evidenced, in particular, by the distributions of results of participants who were tested in compulsory subjects on different days of the main sessions.

 

Undoubtedly, many educators would like to see all the test forms used during the main sessions of the NMT, as they used to do when the tests were administered in paper format. However, as we wrote earlier, this is not possible, because high-quality test items that can be used in different test years to ensure the accuracy and reliability of measurements are an extremely valuable resource, and even more so if there is a need to use a large number of test forms within one test year. Some may argue that a significant number of this year's test materials are already available online, so what's to stop them from being officially published? Here it is important to understand that the vast majority of tasks posted online are tasks that have been restored from drafts, and thus they do not always reflect the exact content of the questions and answer options. At the same time, understanding the desire of teachers to know what their students worked with this year, we will publish two test collections for each subject in the report on the results of the NMT-2023. We hope that this information will help teachers better understand the specifics of the current entrance assessment tools and, if necessary, will be used rationally in the educational process, for example, to prepare students for next year's testing or as part of the tools for launching work to make up for learning gaps.

At the same time, we emphasize that NMT tests are tools that were developed to ensure a fair distribution of applicants for selection to higher education institutions, and therefore no other conclusions can be drawn based on the results obtained from the use of the relevant tools. Therefore, despite the fact that both certificates and information cards of this year's NMT and last year's NMT and EIE 2020 and 2021 are accepted for admission to higher/professional higher education institutions in 2023, it is not worth comparing the overall results of NMT-2023 and NMT-2022 or EIE. This is primarily due to the fact that this year's test forms have a larger number of tasks than last year's, but fewer than were usually offered at the EIT. We also caution against using the results of NMT-2023 as an indicator of the level of educational achievement or the degree of educational losses/gaps, as the main purpose of NMT tools, we emphasize once again, is to ensure the quality distribution of applicants, not to assess the quality of education or measure educational losses in 2023 compared to other years.

By the way, this year was the second time that entrance tests were held in the form of mass computerized assessment. Of course, the experience gained last year helped us prepare more thoroughly for this year's NMT, but it did not guarantee that we could foresee all potential difficulties, especially force majeure. And they did occur.

The greatest challenge was the constant air raid alerts: participants had to interrupt the test, go to shelters, which caused additional anxiety, and instructors had to respond quickly to the situation, monitor more closely the participants' compliance with the rules, in particular regarding the use of mobile phones, and stay in temporary testing centers to allow all applicants to complete the testing procedure. Unfortunately, despite all the measures taken, more than 500 applicants were unable to complete their test tasks due to prolonged air raids and will now take the NMT during additional sessions. Some parents rightly pointed out that it would have been better to conduct the testing in shelters. We can partially agree with this, as we have positive experience in some cities where some of the testing centers operated in shelters. However, at the same time, it is important to emphasize that this practice is extremely difficult to repeat, because temporary testing centers are not only premises, but also equipment, uninterrupted Internet connection, access to restrooms, etc.

Another, no less difficult challenge of the main session of NMT-2023 was the technical difficulties that arose during the tests. In particular, constant hacker attacks led to technical failures in the online service: it was not possible to log in immediately, the system did not always produce correct images, and the tester had to be rebooted frequently. We are indeed sorry that the applicants had to work in these somewhat complicated conditions, but another thing is very important in such circumstances: despite all the problems, there was no interference with the assessment results database. The system's servers were overloaded, but all the answers entered were saved. This was confirmed by all those participants who contacted the Ukrainian Center and received detailed information about their answers upon request. We hope that the applicants who had difficulties with the tester's work that could have affected their results submitted their applications in time and will be able to take part in additional NMT sessions.

At the same time, those participants who did not properly familiarize themselves with the requirements and tips for taking the NMT beforehand will not have such an opportunity, which led to sad consequences for them. For example, during the entire period of the main sessions, at the request of the TC staff, 77 participants left the temporary testing centers for using mobile phones during testing, 6 for using handwritten or printed materials, 5 for not fulfilling the requirements and instructions on the NMT procedure, and 1 for using the Internet.  It was some of these participants who, in particular, took pictures of test tasks and then distributed them on the Internet. There were more of these "activists" in the first days of testing, but gradually their number decreased significantly. This was facilitated both by measures taken by the TC staff on the instructions of the Ukrainian and regional centers for educational quality assessment and by the Ukrainian Center's targeted outreach on social media and on its website. We kept asking NMT participants not to risk their future and not to use any digital devices during the test. And the fact that only a few of the many thousands of applicants managed to take pictures of some test tasks, taking advantage of the fact that the instructors did not see any violations, shows the effectiveness of the measures we took.

It should also be noted that during the main session, there was a case of another person taking the test instead of the participant who was in the TC at the time.  The IP addresses used to log in to the testing system from the outside were identified, so this person's results were canceled, and therefore this person will not be able to use them for admission.

To summarize. The main NMT sessions have taken place. They have become a test of strength for many participants and organizers, both in Ukraine and abroad. Lessons have been learned and all possible measures have been taken to improve the situation.

We would like to thank those who set up temporary testing centers in Ukraine and abroad, as well as those who worked around the clock in them. You have kept the educational front going, making sure that Ukraine has the future we all deserve. We would also like to thank the participants who have studied hard, received their test results and now have the opportunity to enter Ukrainian universities. You have shown remarkable resilience and endurance, so no new challenges will stop you on your way to your dreams.

 

We thank the Armed Forces for life!

 

We thank teachers for willpower!