Flawed licensing exam led to half of Quebec nursing students failing, report finds

Doctors, nurses react to health-care reform Quebec's College of Physicians and Nurses reacted today to the government’s new bill to overhaul health and social services.

Hundreds of nursing candidates were deprived of their right to practice because their professional order made them artificially fail the exam.

That's what the commissioner with Quebec's office for the professions revealed in the second progress report stemming from the investigation into the fiasco that was the Order of Nurses (OIIQ) exam in the fall of 2022.

In this second part of his analysis, Commissioner André Gariépy questions the reliability and validity of the exam, but offers recommendations to remedy it.

Gariépy focuses on the methodology behind the design of the exam and harshly criticizes the OIIQ for its lack of rigor. The highlights of his second progress report state that "for several years, the statistical reliability coefficient of the exam has been low."

He goes on to say that "the documentation typically used to construct the exam and ensure its validity has not been reviewed in over a decade," despite the fact that significant changes have been made to the exam itself.

Even worse, the commissioner states that "due to confidence issues" in its own exam, the OIIQ decided, in 2021, "to systematically raise the passing score beyond the score established by the agreed-upon method."

This increase, which Gariépy considers unjustified, appears to have caused some 500 candidates to fail the September 2022 exam. These aspiring nurses were deprived of their right to practice without the order being able to truly judge their competence.

"That's as many future nurses who could have had their license to practice in the fall of 2022 and contributed to the health network," he asserts in his report.

To remedy the situation, the commissioner asks "that the September 2022 results be recalculated based on a revised and fully justified passing score."

The commissioner is also interested in the order's inability to maintain a fair level of difficulty from exam to exam. He is also interested in the appropriateness of the exam questions and what they are really trying to assess.

In September 2022, only 51.4 per cent of first-time candidates achieved a passing score of 55 per cent on the nurse licensing examination. When candidates who had failed in the past are included, the overall pass rate dropped to 45.4 per cent.

Faced with this disproportionate failure rate, Gariépy launched an investigation. In his first progress report, the commissioner indicated that he had noted "worrying elements regarding both the exam and the training of candidates."

The investigation into the quality of training and preparation of candidates is ongoing. The results will be published in a third progress report.

At the time the exam results were released, the OIIQ blamed the context of the pandemic for an inadequate framework for learning or exam preparation for students in the various nursing programs.

The Canadian Press health content receives funding through a partnership with the Canadian Medical Association. The Canadian Press is solely responsible for editorial choices.

This report was published by The Canadian Press on May 9, 2023.