Translate this page into:
Using virtual reality simulation for training practical skills in musculoskeletal wrist X-ray - A pilot study
, Rune Overgaard Jensen2, Signe K. K. Gade2, Maria Grabau Thielsen2, Winnie Most3, Pia Iben Pietersen2
*Corresponding author: Janni Jensen, Department of Radiology, Odense University Hospital, Odense, Denmark. janni.jensen@rsyd.dk
-
Received: ,
Accepted: ,
How to cite this article: Jensen J, Graumann O, Jensen RO, Gade SK, Thielsen MG, Most W, et al. Using virtual reality simulation for training practical skills in musculoskeletal wrist X-ray - A pilot study. J Clin Imaging Sci 2023;13:20.
Abstract
Objectives:
Using virtual reality (VR), students of radiography can practice acquisition and positioning of musculoskeletal radiographs and get immediate feedback on their performance within the simulator. The purpose of this study was to assess usability of a newly developed VR simulator and to explore self-perceived clinical readiness (SPCR) of radiography students before and after training acquisition of wrist radiographs in the VR simulator.
Material and Methods:
A prospective methodology was applied where the students (n = 10) estimated their own SPCR in regard to acquisition of wrist radiographs pre- and post-VR training. A questionnaire on usability, realism, and educational value of the simulator was answered post-VR training. Usability and SPCR scores were calculated. The student’s paired t-test was applied to explore the impact of VR training on SPCR.
Results:
The students (90%) reported that the simulator was realistic and they thought that it could contribute to learning. The pre- and post-SPCR scores were 75 (95% confidence interval [CI]: 54–96) and 77 (95% CI: 59–95), respectively. There was no significant difference (P = 0.4574) between the pre- and post-SPCR scores.
Conclusion:
Results indicated that the concept of training acquisition and positioning of wrist radiographs in a VR simulator is feasible with positive feedback from the students. The SPCR scores improved slightly, although not statistically significant, after completion of the training session.
Keywords
Virtual reality
Wrist radiography
Patient positioning
INTRODUCTION
Distal radius fractures (DRFs) are one of the most commonly occurring fractures and account for 15–20% of all fractures. The DRF is typically diagnosed on radiographs taken in the acute setting.[1] Treatment of a DRF is, in part, based on quantification of fracture displacement as measured in radiographs.[2,3] Improper patient positioning during the radiographic procedure can affect the measured values of fracture displacement; in other words, correct patient positioning is directly linked to diagnostic value.[4,5] Correct radiographic positioning is, however, a skill that requires practice and knowledge.[6]
Until recently, radiographic positioning has typically been taught in the classroom or trained using phantoms. With the advent of new technologies such as virtual reality (VR), students of radiography are given the opportunity to practice and simulate musculoskeletal radiography acquisition skills using a VR headset and controllers while in a safe and limitless environment with neither radiation nor involvement of patients.[7] The risk of radiation makes it challenging for radiographers to train radiographic positioning involving patients; this risk is eliminated using VR. Another challenge is that it may be time consuming to get real-time feedback on the radiographs from an expert. A recent study involving 1st year students of radiography showed positive correlation between VR simulation and their subsequent clinical performance across a range of parameters.[8]
The virtual three-dimensional VR universe has also been used in educational contexts within other health domains.[9-12] Studies have already explored the effect of VR learning in the areas of ultrasound, anatomy, and surgery and have found significant positive effects although further research is suggested in previous studies.[11,13]
The use of VR has also been seen within the field of X-ray training where students of radiography reported a positive attitude toward training procedures in VR. The biggest challenge they faced was lack of tactile feedback when palpating anatomical landmarks during positioning. Moreover, a general lack of interaction and communication with the patient was missing as well as feedback on positioning from the system. The students mentioned that they would like feedback on their radiographs so they could learn how to improve their positioning and improve diagnostic quality of the images.[7]
There is, to the best of our knowledge, a relatively limited amount of literature on the topic - especially on students training in a virtual simulation where the focus is positioning (rotation, deviation, and flexion/extension) of the wrist involving direct feedback and information based on their own radiographs as taken in the VR simulator. Accordingly, the primary aim of this study was to conduct a pilot study and evaluate the effect of an immersive VR simulator for training musculoskeletal X-ray of the wrist. The objectives of this study were to explore: (i) Self-perceived clinical readiness (SPCR) before and after training in the VR simulator and (ii) VR system usability.
MATERIAL AND METHODS
Study design
A single group study design was used to assess feasibility of training acquisition of wrist radiographs using a VR-based simulation. The study was approved by the Local Committee for Ethics in Research (RA2008003). All students of radiography, currently attending University College, i.e., not in clinical placement were invited to participate. Ten students of radiography at varying stages of their education were recruited voluntarily. All participants were given a unique ID and neither name nor any other person-sensitive data were collected. The tests took place over a 3-day period in April 2022. The participants were informed that they could withdraw from the study at any time without having to provide a reason. They were also informed that they could take a break during the simulation if needed. An oral introduction was prepared to ensure that all participants received the same information.
Data collection
Data were collected using moderated usability testing, a real-time test with the presence of a trained facilitator with an in-depth knowledge of the VR system.[14] Each test had a duration of approximately 1 h and consisted of three questionnaires, VR introduction videos, and the acquisition of wrist radiographs in the VR simulator. All tests were performed at a classroom at University College of Radiography.
Questionnaires
Before the VR simulation, the participants answered two questionnaires - an initial questionnaire on demographics, VR- and/or gaming experience, and with wrist X-rays and next, a questionnaire on SPCR in relation to obtaining wrist radiographs was answered. The SPCR questionnaire was answered twice, a pre-SPCR before entering the VR simulator and a post-SPCR after completing the VR simulation. The SPCR questionnaire contained ten statements on the participant’s self-perceived ability when obtaining wrist X-rays using a five-point Likert scale (strongly disagree, disagree, neutral, agree, and strongly agree) with statements such as “I feel insecure on positioning the wrist of the patient when performing musculoskeletal X-ray” [Appendix 1]. The same SPCR questionnaire was answered after completion of the VR simulation. Finally, a modified questionnaire on system usability, system usability scale (SUS),[15,16] was answered by all participants containing statements such as “I felt very confident using the system.” The SUS was expanded with 2 statements (1) “I think the X-ray room was realistic” and (2) “I think the simulator can contribute with learning to the education of radiography,” totaling 2 questions [Appendix 2].
The SUS scores were calculated using the method suggested by Alathas[16] by adding the scores (from 1 = strongly disagree to 5 = strongly agree) of all the odd-numbered questions and subtracting 5 from the total to achieve a scoreodd. The total of all even-numbered questions was subtracted from 25 and this sum was multiplied by 2.5 to achieve the scoreeven. The sum of scoreodd and scoreeven was the final total SUS score. Thus, a SUS score from 0 to 100 could be achieved, with higher SUS scores indicating higher usability.[16] The SPCR score was calculated using the same method as described for the SUS scores, therefore the higher the SPCR-score the better, with 100 being the maximum score obtainable.
VR simulator
The VR simulation was developed in collaboration with a company specializing in VR software for medical education (VitaSim ApS, Odense, Denmark). In the VR simulator, a short immersive VR tutorial was given to the participants, before positioning the patient and taking posterior-anterior (PA) and lateral wrist radiographs. The tutorial consisted of an oral introduction to the VR simulator and the basic VR interactions. Hereafter, the students were led into the X-ray simulation room where an avatar introduced the tasks in the room, i.e., positioning the virtual patient, taking the wrist radiographs and how they could get feedback on radiographic positioning. In Figure 1, the test flow is presented.
- Flowchart of the virtual reality simulation test.
The virtual patient had the following anatomical movement of the shoulder (abduction, adduction, and rotation), elbow (flexion and extension), and wrist (supination, pronation, flexion, extension, radial, and ulnar deviation). Moreover, the participants had the opportunity to remove the skin to expose the bones. Figures 2 and 3, respectively, depict the X-ray room as seen in the VR simulator and the VR patient with the bones exposed.
- The virtual X-ray room with the patient seated and positioned for a wrist radiograph.
- The patient is positioned for a posterior-anterior wrist radiograph and the bones have been exposed to help with correct positioning for a retake.
After positioning the patient and collimating the image, PA and lateral wrist radiographs were obtained of the VR patient, the participants were shown the corresponding radiographs and given immediate feedback on positioning, i.e., deviation, flexion, extension, and rotation. The feedback was visually depicted on the wall behind the patient. The user could choose between feedback following each single radiograph or after acquisition of both PA and lateral radiographs. An avatar appeared in the X-ray room and gave further information on how to interpret the feedback as well as theoretical background knowledge on positioning criteria of PA and lateral wrist radiographs. Following the feedback, the participants could reposition the patient and obtain as many retakes as needed. Updated feedback was available following each new wrist radiograph.
Technical VR setup
An Acer Predator Helios 300 with RTX 3070 GPU and 16 GB RAM (Compal Electronics Inc., Taiwan) was used for developing and running the VR simulator tethered. An Oculus Quest headset (Meta Technologies, Meta Inc., Menlo Park, CA, USA) was used for immersive VR.
Data analysis
Demographics of participants were presented descriptively with mean and range. The SUS scores were presented by mean and standard deviation (SD). The modified questions on simulator realism and educational usability were reported in percentage.
The SPCR score was presented by mean and SD with corresponding 95% confidence intervals (CIs). Assuming normality of data, a student’s paired t-test was applied to assess differences between pre- and post-SPCR scores. Results were visually depicted in a box plot. Stata version 16 (StataCorp. 2019, TX) was used for statistical analyses.
RESULTS
Ten participants (6 females and 4 males) with a mean age of 23.5 years (range: 20–27) completed the VR simulation. The students were at different stages of their education (1st semester n = 1; 3rd semester n = 5; 5th semester n = 4). The 3rd and 5th semester students had educational and practical experience with musculoskeletal radiographs of the wrist. Demographics on experience with video games, VR, and wrist X-rays are shown in Figure 4.
- Demographics. n = 10 (6 female; 4 male).
Based on subjective assessment of how confident the participants felt obtaining a radiograph of the wrist, the mean pre-SPCR score was 75 (95% CI: 54–96) and SD: 29.8. Average post-SPCR score was 77 (95% CI: 59–95), SD (25) [Figure 5]. Thus, no statistically significant differences were shown between the pre- and post-SPCR score, P = 0.4574 [Figure 6].
- Depicting individual pre- and post-self-perceived clinical readiness (SPCR) scores (n = 10). Cases with only one red dot signify equal pre- and post-SPCR scores, as seen in participants 1, 5, and 10.
- Box plot with medians, quartiles, and ranges comparing the pre- and post-self-perceived clinical readiness scores.
The average SUS score was 77.5 (SD: 19.1) out of 100. Most of the participants, 90% (n = 9), strongly agreed that the X-ray room in the VR simulator was realistic whereas one participant was neutral on this question. Regarding learning potential, 90% (n = 9) of the participants strongly agreed that the VR simulator could contribute to learning at the radiographer education.
DISCUSSION
This study piloted feasibility of using a VR simulator as an educational tool when training correct positioning of wrist radiographs with specific objectives of exploring: (i) SPCR before and after training in the VR simulator and (ii) system usability. In the simulator, feedback was offered based on patient positioning along with educational pointers on wrist radiographs and positioning regarding anatomical landmarks and how to assess anatomical positioning of the wrist and hand in a radiograph.
Despite positive feedback on the use of the simulator, the SPCR score was improved only by an average of 2 points after the completion of the VR sessions. This finding could indicate what psychologists Kruger and Dunning described in 1999 – the Dunning–Kruger effect. They described after four experiments that students who objectively performed well often subjectively underestimated their own performances.[17] On the other hand, students that performed objectively poor overestimated their own performances.[17] Considering that all participants in the current study were students, they had a rather high SPCR regarding obtaining wrist radiographs. This may in part be because of the relatively low esteem of musculoskeletal radiography. However, while the radiography skills and competencies have not altered overtime, the importance of accuracy and recognition of the skill may have become downgraded by some.[6] Aforementioned Dunning–Kruger[17] effect may also explain the student’s trust in own competences, or perhaps, more likely, a combination of the two.
As technologies and imaging modalities have developed, musculoskeletal radiography has been increasingly seen as less complex compared to magnetic resonance imaging and computed tomography.[6] A hypothesis could be that the students overestimated their own ability to obtain wrist radiographs assuming that it is a common and therefore simple procedure. In a recent rejection analysis study, it was, however, uncovered that improper positioning was one of the most common reasons for retake of digital musculoskeletal radiographs.[18] Hence, in a profession such as radiography, it is critical that clinical competency assessments are part of the curriculum both as formative assessments for identifying the strengths and weaknesses and for highlighting deficiencies, or as summative pass/fail tests, e.g., for high-risk or high-stake procedures. By implementing tests, trainees, in this case, radiography students can train the procedure repeatedly until sufficient proficiency. Some students might need more training, guidance, and supervision than others due to, e.g., different previous experiences and learning paces. This principle is called mastery learning and is a frequent and well-described method for health-care education.[19] VR supports the principles of mastery learning, because it is possible to incorporate assessment tools within the virtual simulation. It is possible to make the training and learning period more flexible to the students and decrease the number of instructors needed compared to, e.g., classroom-based education and training. Mastery learning, however, requires clear learning objectives, a competence level to meet or exceed, and the opportunity for continuous feedback. To our knowledge, no studies have been published presenting VR tests for assessing radiography positioning skills but solid validity evidence have been published for VR tests in ultrasound and for preparing students of radiography for clinical practice.[8,13]
Particularly, regarding positioning and musculoskeletal radiography, even small degrees of malpositioning can influence the diagnostic value derived from the radiograph. In the case of wrist radiographs, forearm rotation away from the mid-prone position can depict the apparent angulation of the articular surface of the radius differently as seen in the lateral image. Studies have shown that forearm supination can decrease the radiographically measured value of dorsal angulation and conversely, pronation may increase the apparent dorsal angulation.[5,20,21] Patient positioning is also important in other anatomical regions, such as the pelvis, where pelvic rotation during the radiographic procedure will impact the appearance of radiographic signs of acetabular retroversion such as the crossover sign.[22] Hence, the value of VR training and positioning in musculoskeletal radiography is important in, but not limited to, wrist radiographs.
Limitations
An inherent limitation in the current study is the low number of participants (n = 10) combined with the fact that the participants were at various stages of their education. A larger and more homogenous cohort of participants might have changed the results. The path from educational research to clinical outcomes is long with multiple factors and variables affecting the pathway. The first step could be to investigate the clinical effect of the VR training, e.g., exploring the transfer to a clinical setting by comparing the image quality (positioning) of radiographers or students who have trained using VR compared to those who have trained using normal methods. An alternative approach could be to gather validity evidence for a VR-based simulation test in radiography competences such as the wrist X-ray. Moreover, this VR simulator allows for training in patient positioning only, and thus, factors such as radiation protection, radiation dose, and subsequent technical image quality are not assessed in this study.
CONCLUSION
This pilot study showed no difference in SPCR before and after the use of VR. The feasibility of using VR simulation for training correct positioning of wrist radiographs was identified though with positive feedback from the participants. Additional studies are recommended to further elaborate on this initial research.
Declaration of patient consent
The Institutional Review Board (IRB) permission was obtained for the study.
Conflicts of interest
R.O.J. is a co-founder and co-owner of VitaSim. R.O.J. advised the authors M.G.T. and S.K.K.G. on technical solutions to developing the X-ray training module in virtual reality. R.O.J. had no influence on data collection, data analysis, interpretation, or publishing decisions.
Financial support and sponsorship
This research was funded by “Innovationspuljen” at Odense University Hospital. The funding body had no role in study design; data collection, analyses, or interpretation; in the writing of the manuscript; or in the decision to publish the results.
References
- The epidemiology of distal radius fractures. Hand Clin. 2012;28:113-25.
- [CrossRef] [PubMed] [Google Scholar]
- A structured review addressing the use of radiographic measures of alignment and the definition of acceptability in patients with distal radius fractures. Hand. 2015;10:621-38.
- [CrossRef] [PubMed] [Google Scholar]
- American Academy of Orthopaedic Surgeons clinical practice guideline on: The treatment of distal radius fractures. J Bone Joint Surg Am. 2011;93:775-8.
- [CrossRef] [PubMed] [Google Scholar]
- The effect of forearm rotation on radiographic measurements of the wrist: An experimental study using radiostereometric analyses on cadavers. Eur Radiol Exp. 2021;5:15.
- [CrossRef] [PubMed] [Google Scholar]
- Dorsal tilt of the distal radius fracture changes with forearm rotation when measured on radiographs. J Hand Surg Glob Online. 2021;3:182-9.
- [CrossRef] [PubMed] [Google Scholar]
- There's nothing plain about projection radiography! A discussion paper. Radiography. 2021;27:1227-30.
- [CrossRef] [PubMed] [Google Scholar]
- 3D virtual reality simulation in radiography education: The students' experience. Radiography (Lond). 2021;27:208-14.
- [CrossRef] [PubMed] [Google Scholar]
- The impact of 3D virtual reality radiography practice on student performance in clinical practice. Radiography (Lond). 2022;29:159-64.
- [CrossRef] [PubMed] [Google Scholar]
- Education in focused lung ultrasound using gamified immersive virtual reality: A randomized controlled study. Ultrasound Med Biol. 2023;49:841-52.
- [CrossRef] [PubMed] [Google Scholar]
- Using immersive virtual reality simulation to ensure competence in contrast-enhanced ultrasound. Ultrasound Med Biol. 2022;48:912-23.
- [CrossRef] [PubMed] [Google Scholar]
- Immersive virtual reality in basic point-of-care ultrasound training: A randomized controlled trial. Ultrasound Med Biol. 2023;49:178-85.
- [CrossRef] [PubMed] [Google Scholar]
- Clinical ultrasound education for medical students: Virtual reality versus e-learning, a randomized controlled pilot trial. Ultrasound Q. 2021;37:292-6.
- [CrossRef] [PubMed] [Google Scholar]
- Can virtual-reality simulation ensure transthoracic echocardiography skills before trainees examine patients? Int J Med Educ. 2022;13:267-73.
- [CrossRef] [PubMed] [Google Scholar]
- Moderated vs Unmoderated Usability Testing: Which Do You Need? Available from: https://www.justinmind.com/blog/moderated-vs-unmoderated-usability-testing-which-do-you-need [Last accessed on 2022 Dec 18]
- [Google Scholar]
- How to Measure Product Usability with the System Usability Scale (SUS) Score. 2008. Available from: https://uxplanet.org/how-to-measure-product-usability-with-the-system-usability-scale-sus-score-69f3875b858f [Last accessed on 2022 Dec 18]
- [Google Scholar]
- Unskilled and unaware of it: How difficulties in recognizing ones own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;88:1121-34.
- [CrossRef] [PubMed] [Google Scholar]
- Reject analysis in digital radiography: A local study on radiographers and students' attitude in Iran. Med J Islam Repub Iran. 2019;33:49.
- [CrossRef] [PubMed] [Google Scholar]
- Mastery learning: It is time for medical education to join the 21st century. Acad Med. 2015;90:1438-41.
- [CrossRef] [PubMed] [Google Scholar]
- The effects of forearm rotation on three wrist measurements: Radial inclination, radial height and palmar tilt. Hand Surg. 2005;10:17-22.
- [CrossRef] [PubMed] [Google Scholar]
- Palmar tilt of the distal radius: Influence of off-lateral projection initial observations. Radiology. 2001;220:594-600.
- [CrossRef] [PubMed] [Google Scholar]
- Effect of pelvic tilt on acetabular retroversion: A study of pelves from cadavers. Clin Orthop Relat Res. 2003;407:241-8.
- [CrossRef] [PubMed] [Google Scholar]
APPENDICES
| S. No. | SPCR |
|---|---|
| 1. | I feel theoretically confident about how to perform musculoskeletal X-rays of the wrist |
| 2. | I feel confident in being able to carry out musculoskeletal X-rays of wrists in the clinic |
| 3. | I feel uncertain about how to use an X-ray machine |
| 4. | I feel uncertain about positioning the patient’s wrist when performing a musculoskeletal X-ray |
| 5. | I feel confident about how to position for the good PA wrist X-ray |
| 6 | I feel confident about how to position for the good lateral wrist X-ray |
| 7. | I feel uncertain about assessing whether an x-ray of a wrist is diagnostically useful |
| 8. | I feel confident in what anatomical structures to look for to assess whether a PA wrist X-ray is appropriate |
| 9. | I feel confident in what anatomical structures to look for to assess whether a lateral wrist X-ray is appropriate |
| 10. | I feel confident in how to fine tune the wrist position based on the X-ray. |
All questions were answered on a scale from 1 to 5. (1: Strongly disagree, 2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). SPCR: Self-perceived clinical readiness, PA: Posterior-anterior
| 1. I think that I would like to use this VR simulator frequently |
| 2. I found the VR simulator unnecessarily complex |
| 3. I thought that the VR simulator was easy to use |
| 4. I think that I would need the support of a technical person to be able to use this VR simulator |
| 5. I found that the various functions in this VR simulator were well integrated |
| 6. I thought that there was too much inconsistency in this VR simulator |
| 7. I would imagine that most people would learn to use this VR simulator very quickly |
| 8. I found the VR simulator very cumbersome to use |
| 9. I felt very confident using the VR simulator |
| 10. I needed to learn a lot of things before I could get going with this VR simulator |
| 11. I think the X-ray room was realistic |
| 12. I think the VR simulator can contribute with learning to the education of radiography. |
All questions were answered on a scale from 1 to 5. (1: Strongly disagree, 2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). VR: Virtual reality
