Background

Cognitive engagement is the readiness of a student to participate in a purposeful task requiring an active mental state to experience learning in the zone of proximal development and sustain connectivity to the activity (Bowden, Tickle, & Naumann, 2021). 

First-year undergraduate nursing students at the Aga Khan University School of Nursing and Midwifery (AKUSONAM) learn numerous nursing skills, including vital signs, height/weight measurement, hygiene care, glucose monitoring, and the administration of oxygen therapy during skills labs on nursing mannikins or a simulator. Skill-based assessment (SBA) is often performed after required skills have been taught to learners and before starting clinical rotation in a hospital setting. This paper aims to address the design and execution of a skills-based assessment for first-year undergraduate nursing students. The innovative digital evaluation technique that we used in assessment can be replicated in other skill-based courses with intent modification. ​

Designing Skills-based Assessment

Skills-based assessment is considered a meaningful and valid tool to assess competency and clinical judgement for undergraduate nursing students (Koster & Soffler, 2021). Innovation in assessing nursing competency frequently refers to a difficult component that provides the information for instructors to deploy concrete and foster students’ cognitive engagement in assessment (Park, Kwon, & Chung, 2021). Cognitive involvement in skill refers to students’ active mental involvement in a conducive learning environment so they can comprehend the challenging clinical situations, implement their clinical judgment, and strengthen their clinical competency and acquire new knowledge (Ladino Nocua et al., 2021).

Our goal is to develop SBA as a tool to enhance students' cognitive involvement by using digital modality to assess their competency in administering oxygen therapy using an appropriate mode of oxygen device to a simulated patient. This approach captured the students' cognitive attention to the clinical data presented on the screen, prompted them to reflect on the information, used critical reasoning, made the correct choice, and provided safe care to the client. This initiative can be replicated in other skills-based courses with intended modifications to bridge the gap between the skills learned in a simulated environment and the clinical setting. 

Planning Phase

The SBA planned to accommodate 147 learners in-person in four clinical ward areas at the Centre for Innovation in Medical Education (CIME). We divided the large group into three sub-groups with four facilitators over the course of three days. We set up two mandatory stations. In the first station, each student was given a scenario about the intake and output of a patient in 24 hours, and they were instructed to document the required calculation on a given flowsheet. The second station was designed to administer prescribed oxygen therapy to the patient using an appropriate oxygen device. A nursing mannequin is used to simulate a patient suffering from desaturation termed “hypoxia”. The team constructed multiple clinical scenarios depicting desaturation in a variety of settings. They also assured that student learned concepts of patient’s deteriorating conditions depicted in the scenarios. Subsequently, with the collaboration of a CIME analyst, the oxygenation data set was fed into software and translated into a series of videos. These videos had two phases, beginning with the deterioration of patients' conditions, and concluding with the recovery stage. Multiple laptops were used for the dry run to assure accessibility and enable all faculties to get hands-on. A day before the exam, a few logistical arrangements for station setup, equipment accessibility, and operations, as well as scheduling for both students and faculties have been verified. To unify the grading standards, the team turned detailed standard checklists for skills into a liker scale ranging from 0 to 6 for each step of the skills. We determined the fundamental concepts required to pass the skills prior to the assessment, ensuring consistency in grading across faculties throughout the assessment. During the week of skills practice, the team briefed all groups an overview of several skills stations and checklists, familiarity with equipment, venues, format, and grading requirements for the completion of the course’ clinical. 

Execution phase

On examination day, students were expected to start their assigned stations as per schedule. For station one, they randomly selected the scenario for intake and output and submitted the documentation in ten minutes. Later, they moved to the second station, and after selecting the scenario and interpreting the oxygenation data displayed on the laptop's screen, they were supposed to select the appropriate oxygen devices, such as a nasal prong, face mask, or venturi mask for administering oxygen therapy according to the prescription and complete all required steps according to the checklist and learned concepts. In addition, they were also expected to document desaturation and improved saturation readings on a given flowsheet in another ten minutes. 

Navigate the challenges 

We encountered a few obstacles in constructing higher-order thinking tasks and tailoring them to the cognitive level of first-year nursing graduates. Students’ autonomy to select their own scenarios enhanced their self-confidence to perform their best and increased their cognitive engagement at the beginning of the task. It was challenging to establish a safe learning environment where all equipment was accessible to each student, accommodate a large class in the venues with consistent scheduling, and maintained uniformity over the course of three days. A few months prior, the CIME department was informed about the advance booking of venues and the quantity of equipment. Each student’s clinical performance evaluation, which involved marking them on each step and taking notes within ten minutes was also a challenge for all facilitators, for which the Liker scale and overall facilitators’ comments were helpful. Lack of technical competency to transform clinical situations into visual data sets, availability of laptops for each faculty, and visibility of data sets to students were challenges for us. Moreover, we observed that exam-related anxiety created trouble for certain students to interpretate the oxygen saturation data sets and they were also confused with the pulse rate displayed on the laptop's screen simultaneously. However, facilitators ensured that heart rate (HR) and oxygen saturation (SPO2) labels, which were taught during skill practice, were shown on screen to support students throughout the exam.

Students’ Outcome

We recognized that a skills-based assessment format was crucial for determining students’ competency in performing skills within a certain time period. A clinically conducive environment has given each student the opportunity to demonstrate his or her best clinical decision-making skills in a comfortable zone. Moreover, the interactive simulated videos and scenarios increased the complexity of the assessment and fostered uniformity across the examination process for all learners. This cognitive engagement in assessment helped students-built confidence, strengthened their motor dexterity, and enhanced their critical thinking. All faculties witnessed the progression in students’ clinical capacity to recognize early warning signs of hypoxic patients and assisted them in responding promptly with best practices while supervised in the clinical setting. In addition, the structure of the stations and a pre-briefing on the desired learning outcomes thrive students in focusing on constant monitoring of the patient's progress and administering oxygen therapy according to the standardized guidelines.  The SBA strategy enabled us to effectively connect students' learning with the programmed outcomes of providing safe and competent patient care. Few students shared feedback about SBA:

“The video strategy which was used to deliver the concept of oxygenation therapy was very fruitful for me as the video was very easy to comprehend & it gave us a quick snapshot of the whole procedure. The video strategy was very interactive & it help the students to grasp the concept more effectively” (Student BSCN Year I)."

Way Forward 

The interactive, simulated video series pedagogy for evaluating skill competence reflects new learning and insight to encourage students' cognitive engagement in skills-based assessment. We recommend this digital modality in SBA for other clinical skills, including wound management, Modify Early Warning Sign (MEWS) scoring, code management, and central venous or arterial line pressure monitoring, so that high-order thinking, and clinical judgement can be reinforced and become an integral element of skill-based assessment. We believe that students can confidently apply theoretical concepts and skills of oxygenation in their future clinicals with more competency and independently with safe practices. A student shared:

“The video-based strategy used for oxygenation in skill-based assessment was very helpful for us. I think such strategies should be implemented in other skills as well for experiencing a real situation and excellent understanding of the skills which are going to be applied on real patients in hospital” (Student BSCN Year I). ​

Acknowledgement: Zohra Jetha, Farzana Yameen, CIME team and Shahnaz Cassum