Comparing Block Assessments of 3rd Professional MBBS Annual Examination 2022 at Rawalpindi Medical University Pakistan

Introduction

Assessment serves as one of the building blocks of any curriculum. It is not possible to ascertain the progress in learning without assessments [1]. Both formative and summative assessments are of paramount significance to foster learning and to grade them for promotion respectively [2]. All frameworks and standards of courses specified by the regulatory bodies worldwide have emphasized the need for comprehensiveness of assessment [3,4].

Being regulatory body for medical education in Pakistan, Pakistan Medical & Dental Council (PM&DC) has specified the guidelines for both traditional and integrated undergraduate curriculum. These guidelines specify the requirement for year-wise allocation of teaching hours for each discipline. In addition to indicating the assessments methods for each learning domain, tools to be employed for summative assessment of undergraduate medical students have also been evident [5].

Although Miller’s pyramid of competences serves as a guide to choose an assessment tool in alignment with the learning domain tested the selection of appropriate assessment method to evaluate its effect on students’ learning is still laborious [6,7]. Single best answer Multiple Choice Questions (MCQs) are developed in all medical institutes globally to verify objectivity of assessment. Some qualitative studies have also proven their positive impact in terms of testing higher order thinking skills among students [8]. They can conveniently be developed to measure understanding and application of knowledge by the students in limited time span by using Optical Mark Reader (OMR) machine [9].

Item analysis of MCQs have considerably been done in various institutes for evaluating their quality in terms of difficulty index, discrimination index and distractor effectiveness [10]. Measuring these indices is useful in critical appraisal of questions. The present study is intended to compare the results of block assessments primarily MCQs-based assessment among 3rd year MBBS students during their professional exam 2022 that was multidisciplinary as these students have been subjected to modular curriculum since 2017 following upgradation of the Rawalpindi Medical College to university level [11]. Rawalpindi Medical Univresity pioneered in launching integrated modular curriculum that was meant to ensure comprehensive and experiential learning of the students [12]. Comparing the structured objectives-based results of block assessments in current study along with subject-wise analysis would enable stakeholders to scrutinize the underlying reasons for the difference of inter-block assessments or inter-subject score variations and to do rational planning for curricular revision or upgradation in future.

Methods

A comparative analytical study was done to compare the results of block assessments in 3rd professional MBBS annual examination 2022 and to evaluate the results in depth by examining inter¬disciplinary mean score differences. Results of total 330 students who appeared in professional exam were collected from the examination department of Rawalpindi Medical University through informed consent. The data was entered and analyzed by using Microsoft Excel 2016. Descriptive statistics were applied. 95% Confidence Interval (CI) for each block assessment was also calculated. Statistical differences in the results of block assessments were determined by applying paired t-test. P < 0.05 was taken as significant. Subject-wise mean percent score was also computed.

Results

A total of 330 third year MBBS students appeared in the 3rd year MBBS Annual examination 2022 at Rawalpindi Medical University. Of the total 60 Multiple-Choice Questions (MCQs) incorporated in each block assessment, most of the MCQs were from Pharmacology and Pathology as illustrated below in Table 1:

                                     Table 1: No. of MCQs Incorporated Subject-Wise in Block Assessments

The mean percent score attained by the students in objective block assessments is depicted below in Figure 1.

Maximum and minimum scores got by students in each block assessment are presented below in Figure 2.

Figure 2: Maximum and Minimum Scores Recorded in Block Assessments

Students achieved relatively higher mean scores in block-II assessment as illustrated below in Table 2.

Table 2: Mean Scores of Block Assessments of 3rd Professional MBBS Annual Exam 2022

The differences in the mean scores attained by 3rd year students in Block-I and Block-III objective assessments were statistically insignificant as shown below in Table 3.

Table 3: Statistical Difference Between the Results of 3 Blocks of 3rd Professional MBBS Annual Examination 2022

The mean percent score of individual subjects in theory paper pertaining to each block is shown below in Table 4.

                              Table 4: Subject-Wise Mean Percent Score in 3rd Year MBBS Professional Annual Exam

Discussion

Multiple Choice Questions (MCQs) pertaining to Pharmacology, Pathology, Forensic Medicine and Behavioral Sciences were developed in accordance with the teaching hours allocated to each subject in respective block as illustrated in Table 1. The maximum score was achieved by 3rd year students in block-III that was followed by the mean % of block-I and block-II (Fig 1). The mean score of the students in objective assessments remained in forties out of 60 that reflects inadequate preparation of the students. Although pass percentage for MBBS students in medical colleges of Punjab is 50%, being high stake assessment, grading of the medical students while promoting them to next level should be finalized in alignment with the community healthcare needs and patients’ safety. Likewise, a study carried out by Hernandez S et al among 3rd year MBBS students revealed deficiencies in surgical knowledge due to their online classes amidst COVID-19 pandemic. They were not well-equipped with certain hidden curriculum competencies like ethics, professionalism, communication skills etc. that can considerably be acquired with direct patient interaction in real clinical settings [13]. During COVID pandemic, this 3rd year class was enrolled in 1st year, so they were primarily subjected to campus classes with interactive sessions, practical, small group teaching sessions etc with minimal coverage of clinical sciences topics. Their knowledge might be compromised during first 2 years of their medical sessions due to on and off virtual sessions during pandemic, gaps in cognitive domain of learning can appropriately be addressed by counseling them or paying head to their academic queries. Another study by Smith RS et al among 3rd year medical students concluded that educational interventions can be done and prove useful if students perceive their knowledge about healthcare of the community and concomitant disparities [14]. Many other studies done worldwide during 2020-2022 emphasized the impact of COVID-19 on academic performance of the students [15]. However, analyzing the reasons for academic lacunae of our medical students would enable the institutional administration to take necessary steps for overcoming the existing deficiencies.

As compared to statistically significant differences in mean scores of 3rd year block-I and block-II, block-II and block-III; variations between block-I and block-III were statistically insignificant (P > 0.10) as evident from Table 3. Less score of the students in block-II is quite grievous and upsetting that requires attention and hence scrutiny of the facts for variations of intermodular assessments among same group of students. Although minimum score was attained by 3rd year students in block-III (Fig 2); yet the means score in block-II was found to be the least (Table 2 & 3). This is illustrative of the achievement of low score in block-III by fewer students. Reviewing subject-wise mean percent score as shown in Table 4, students got 57.3% score in block-II and 51.91% score in block-I that reflected achieving below 60% in the respective subjects. A similar descriptive study done among 3rd year medical students and respective teaches of HBS Medical and Dental College Islamabad in 2019 revealed that one of the reservations of the students were non-alignment of their assessments with recent revisions of their curriculum. They opined to have their Pathology lectures over the span of four years to have minimal academic burden as they are now being subjected to integrated modular curriculum [16]. Another study by Atta I et al recommended the implementation of quadriphasic model for mapping Pathology curriculum particularly by the medical institutions that are shifting from traditional to integrated teaching [17]. Another correlational study on assessing the results of Hematology module encompassing the content from both basic, clinical and paraclinical sciences showed remarkable improvement due to utilization of diverse assessment modalities to measure the knowledge and skills of the students [18]. It is imperative for the medical students to gain knowledge and practical competencies pertaining to both basic and special Pathology. It is necessary to incorporate them realistically in modular curriculum in accordance with level of training. The component of Pathology assessed among 3rd year students undoubtedly had rationally been planned and finalized in their respective curriculum at Rawalpindi Medical University; the underlying causes for less scores should intelligently be reviewed and discussed by the curriculum committee in compliance with the feedback of the faculty members and the respective students.

The result of Forensic Medicine in block-I assessment was 51.91%. The students showed remarkable improvement in their Forensic Medicine knowledge in block-II and block-III results (Table 4). A study carried out by Ahmad M et al among 3rd year medical students from Armed Forces medical college of Dhaka revealed awareness among 93.91% of students regarding the need for autopsy in case of unnatural death; however, 49.56% of the students were uncertain about the types of autopsy [19]. Another study among undergraduate medical students of Haryana concluded that knowledge of the students about concerning medico-legal autopsies can substantially be enhanced by arranged more problem based learning sessions and making their rotations to mortuaries mandatory [20]. Ali D et al in his descriptive study among medical students from Nawabshah illustrated the experiences pertaining to autopsies should thoroughly be incorporated in medical curriculum as it is necessary for the medical students to carry out their future official responsibilities [21]. Feeback from the students as well as faculty might have been gathered on Forensic Medicine result notification for block-I assessment, however, one-on-one counseling of the students can also prove beneficial in boosting the academic performance in block-I for the next batch of the students. On analyzing the average of all subjects’ scores, the scores of Forensic Medicine and Pharmacology acquired by 3rd year students in block assessments of their annual examination were comparatively lower than those of Pathology and Behavioral Sciences (table 4). A latest study by Fasinu PS et al highlights the importance of Small Group Interactive Sessions (SGIS) in active and hence long-term learning pertaining to complex subject like Pharmacology. It was also recommended to integrate this subject at all levels from 1st to final year of medicine for better understanding and to distribute the academic burden of the said subject over the span of five years [22]. Contrary to this, a study by Sharp K et al among two different batches of medical students, one subjected to traditional and other one to competency-based curriculum revealed marked difference in MCQs based assessment. The students undergoing traditional learning attained higher mean score (94.24 ± 5.26) than those of another batch (59.68 ± 11.3) [23]. No doubt, curriculum is a dynamic document that should be revised annually in compliance with the feedback gathered from the students, faculty and administration. Studying the curriculum thoroughly and re-arranging the topics in each module of in order of complexity can help a great deal in adjustment of the existing gaps deemed necessary to improve the academic performance of the students.

Conclusion & Recommendations

3rd year medical students got maximum score in block-III assessment during their professional annual examination. The mean percentage achieved in Behavioral Sciences was the greatest, followed by that of Pathology, Pharmacology and Forensic Medicine. The modular study guides for 3rd year MBBS students need revision with respect to teaching strategies and assessment methods for enhancement of their academic performance.

References

1. Update, C. S. A. I. (2018). Standards alignment to curriculum and assessment.
2. Shepard, L. A. (2019). Classroom assessment to support teaching and learning. The ANNALS of the American Academy of Political and Social Science, 683(1), 183-200.
3. General Medical Council. Assessment in undergraduate medical education.
4. Pakistan Medical & Dental Council. Standards for undergraduate medical education.
5. Pakistan Medical & Dental Council. PM&DC Medical and Dental Undergraduate Education (Admission, Curriculum Conduct) Policy and Regulations 2023. Last Updated 13th July 2023.
6. Witheridge, A., Ferns, G., & Scott-Smith, W. (2019). Revisiting Miller’s pyramid in medical education: the gap between traditional assessment and diagnostic reasoning. International Journal of Medical Education, 10, 191.
7. Preston, R., Gratani, M., Owens, K., Roche, P., Zimanyi, M., & Malau-Aduli, B. (2020). Exploring the impact of assessment on medical students’ learning. Assessment & Evaluation in Higher Education, 45(1), 109-124.
8. Liu, Q., Wald, N., Daskon, C., & Harland, T. (2024). Multiple-choice questions (MCQs) for higher-order cognition: Perspectives of university teachers. Innovations in Education and Teaching International, 61(4), 802-814.
9. Multiple Choice Questions: Benefits, Debates and best practices.
10. Shahid, R., Zeb, S., Hayat, U., Yasmeen, S., & Khalid, M. (2021). Item analysis of Pathology Assessment of 4th year MBBS at Rawalpindi Medical University Pakistan. J Comm Med Pub Health Rep, 2(5), 1-5.
11. University of Health Sciences. Prospectus Government Medical and Dental Colleges in Punjab.
12. MBBS: Introduction.
13. Hernandez, S., Song, S., Silva, O. N. N., Anderson, C., Kim,A. S., Campbell, A. R., ... & Roman, S. A. (2022). Third year medical student knowledge gaps after a virtual surgical rotation. The American Journal of Surgery, 224(1), 366-370.
14. Smith, R. S., Silverio, A., Casola, A. R., Kelly, E. L., & de la Cruz, M. S. (2021). Third-year medical students’ self-perceived knowledge about health disparities and community medicine. PRiMER: Peer-Review Reports in Medical Education Research, 5.
15. Kelly, E. L., Casola, A. R., Smith, K., Kelly, S., & de la Cruz,M. S. D. (2021). A qualitative analysis of third-year medical students’ reflection essays regarding the impact of COVID-19 on their education. BMC medical education, 21, 1-9.
16. Janjua, R. S., Riaz, J., & Omer, W. (2020). Special Pathology in the course content of Third Year MBBS: Views of students and teachers. Pakistan Journal of Medical Sciences, 36(5), 987.
17. Atta, I. S., & AlQahtani, F. N. (2018). Mapping of pathology curriculum as quadriphasic model in an integrated medical school: how to put into practice?. Advances in Medical Education and Practice, 549-557.
18. Vuma, S., Sa, B., & Ramsewak, S. (2015). A retrospective co-relational analysis of third year MBBS students' performance in different modalities of assessment in haematology and final integrated examinations. The Caribbean Teaching Scholar, 5(1).
19. Ahmad, M., Rahman, M. F., Rahman, F. N., & Rahman, M. A. (2016). Knowledge and Attitude of 3rd year Medical Students regarding Postmortem Examination. Journal of Armed Forces Medical College, Bangladesh, 2016l12 (2), 9-14.
20. Chawla, H., Tyagi, A., Kumar, R., Malhotra, R., Kumar, N., & Shankar, S. (2019). A study evaluating effectual of knowledge, attitude and perception of undergraduate medical students towards medico-legal autopsy. Indian Journal of Forensic and Community Medicine, 6(3), 138-143.
21. Ali D, Mal S, Magsi I, Qayyum SA, Imran S, Muhayudin G, Awan A. Assessment of student’s knowledge, attitude and perceptions about autopsy. PJMHS Oct 2021; 15(10): 2853-2855.
22. Fasinu, P. S., & Wilborn, T. W. (2024). Pharmacology education in the medical curriculum: Challenges and opportunities for improvement. Pharmacology Research & Perspectives, 12(1), e1178.
23. Sharp, K., Teli, S. E. S. I., Mahajan, H., & Sonawane, R. (2023). Comparative study between academic performances of traditional versus competency-based medical education MBBS curriculum students in Pharmacology. National Journal of Physiology, Pharmacy and Pharmacology, 13(5), 971-971.