How does ChatGPT 4omni perform in consenting for common orthopedic and musculoskeletal interventional procedures?

INTRODUCTION

A fundamental component of patient-centered care and good medical practice is obtaining informed consent before medical treatments, tests, or interventions.^[1,2] Although consent should be sought regardless of the procedure or examination being undertaken, it is of particular importance when discussing the objective, benefits, risks, and alternative available options for patients undergoing invasive or major procedures, such as orthopedic surgery. Conventionally, this occurs in the form of a direct discussion between the clinician and patient supplemented by written consent forms and information leaflets. Where this information is inherently complex, an exhaustive explanation of the specifics of surgery, its benefits and the exact risk (by percentages, for example) are not possible by virtue of finite clinic appointment times and varying degrees of patient comprehension.^[2,3] The new ChatGPT 4omni (ChatGPT 4o) (note, preceded by “ChatGPT 4.0”), an artificial intelligence (AI) language model, is able to hold real-world human conversations and offer individualized yet comprehensive information on complex topics. In this study, we examined the role of the new ChatGPT 4o, for the 1^st time, to enhance the consenting process for ten commonly performed orthopedic and musculoskeletal interventional procedures at our tertiary cold orthopedic hospital.

MATERIALS AND METHODS

Ten commonly performed elective orthopedic and musculoskeletal interventional procedures at the Royal Orthopedic Hospital, a tertiary referral cold orthopedic center, were selected by fellowship-trained consultant musculoskeletal radiologists and orthopedic surgeons, respectively. ChatGPT-4o was asked to consent patients on these procedures [Tables 1 and 2]. The procedures selected were as follows: total hip replacement (THR), total knee replacement (TKR), knee arthroscopy, computed tomography (CT)-guided bone biopsy, ultrasound-guided soft-tissue biopsy, radiofrequency ablation of osteoid osteoma, vertebroplasty, image-guided joint injection, image-guided cervical nerve root injection, and trochanteric bursal injection.

The information provided was scored against existing consent guidance from the British Orthopedic Association consent form templates and a Royal College of Radiologists audit questionnaire on consent for interventional radiological procedures, together with statistical values and consent points used in the score’s clinical practice, by the consultant musculoskeletal radiologist and orthopedic surgeon. These were graded according to four parameters: (i) description of the procedure, (ii) benefits, (iii) risks, and (iv) overall impression of the document. The grading was performed using a Likert scale of 1–5 (1 = bad and inaccurate, 2 = poor, 3 = average, 4 = good, and 5 = excellent and accurate). Each section was scored independently by both the radiologist and orthopedic surgeon. Comments were documented regarding each consenting response and if the articles were totally incorrect or different.

Microsoft Excel data sheet was used for data collection. The Statistical Package for the Social Sciences (SPSS) 24.0 software (SPSS Inc., Chicago, Illinois, USA) was used for statistical analysis. Descriptive statistics analysis was undertaken and statistical probability values were calculated for scores given by scorer, in each of the parameter.

RESULTS

The Likert scores for each domain of (i) description of the procedure, (ii) benefits, (iii) risks, and (iv) overall impression of the document were reviewed with a score graded from 1 (worst) to 5 (best) by the two independent assessors which have been displayed in the form of graphs [Figures 1-4]. Of the orthopedic procedures reviewed by an orthopedic surgeon, both THR and TKR received a Likert score of 3 for description of procedure and 5 for each of the risks and benefits and an overall impression score of 4. Comments on the score pertaining to description of procedure were that there was a lack of description on skin incision, surgical scar, approach (posterior vs. anterior, minimally invasive vs. otherwise), positioning of patient, cemented versus uncemented arthroplasty, and details on skin closure. TKR and knee arthroscopy were scored as 5 in all four domains. The radiologist Likert scoring of all of THR, TKR, and knee arthroscopy revealed a score of 5 across all parameters in each of these procedures.

Close

Figure 1:

Likert scores given by clinicians for description of a procedure in consenting performed by ChatGPT 4omni. THR: Total hip replacement, TKR: Total knee replacement, CT: Computed tomography, US-G: Ultrasound-guided, RFA: Radiofrequency ablation, Image-G: Image-guided, Trochanteric B injection: Trochanteric bursal injection, Cervical NR injection: Cervical nerve root injection, ChatGPT: Chat generative pre-trained transformer.

Export to PPT

Close

Figure 2:

Likert scores given by clinicians for benefits in consenting performed by ChatGPT 4omni. THR: Total hip replacement, TKR: Total knee replacement, CT: Computed tomography, USG: Ultrasound-guided, RFA: Radiofrequency ablation, Image-G: Image-guided, Trochanteric B injection: Trochanteric bursal injection, Cervical NR injection: Cervical nerve root injection, ChatGPT: Chat generative pre-trained transformer.

Export to PPT

Close

Figure 3:

Likert scores given by clinicians for risks in consenting performed by ChatGPT 4omni. THR: Total hip replacement, TKR: Total knee replacement, CT: Computed tomography, USG: Ultrasound-guided, RFA: Radiofrequency ablation, Image-G: Image-guided, Trochanteric B injection: Trochanteric bursal injection, Cervical NR injection: Cervical nerve root injection, ChatGPT: Chat generative pre-trained transformer.

Export to PPT

Close

Figure 4:

Likert scores given by clinicians for overall impression of the document in consenting performed by ChatGPT 4omni. THR: Total hip replacement, TKR: Total knee replacement, CT: Computed tomography, US-G: Ultrasound-guided, RFA: Radiofrequency ablation, Image-G: Image-guided, Trochanteric B injection: Trochanteric bursal injection, Cervical NR injection: Cervical nerve root injection, ChatGPT: Chat generative pre-trained transformer.

Export to PPT

For the musculoskeletal interventional procedures review by an orthopedic surgeon, CT-guided bone biopsy, ultrasound-guided biopsy, radiofrequency ablation of osteoid osteoma, vertebroplasty, and cervical nerve root injections demonstrated a Likert score of 5 across all parameters. For image-guided joint injection and trochanteric bursal injection, a score of 4 for description of procedure and 5 for both risk and benefits were given. Comments on the description of the procedure were that it could have been described in a more concise manner with reduced medical jargon. In contrast, the radiologist gave a score of 5 across all domains in each of the procedures with no additional comments made.

The mean Likert score for each parameter was (i) 4.7 for description of the procedure, (ii) 5 for benefits, (iii) 5 for risks, and (iv) 4.9 for overall impression. Statistical comparison in the form of probability values comparing Likert scores in each of the four parameters between those scored by the orthopedic surgeon and radiologist revealed P-value for (i) description of the procedure of 0.03, (ii) benefits as “perfect agreement,” (iii) risks as “perfect agreement,” and (iv) overall impression of the document as 0.02. All but one (overall impression of the document) showed statistical significance.

DISCUSSION

ChatGPT 4o was rolled out on May 13, 2024, and represents OpenAI’s newest multimodal model, featuring improvements in speed, cost-efficiency, and capability over previous versions such as GPT-4 and GPT-4 Turbo. This model combines text, vision, and audio functionalities, making it a versatile tool for applications such as image recognition, real-time voice communication, and video analysis. Artificial intelligence in healthcare, in general, has been subject to significant developments in the past decade. The availability of open-source AI software, such as ChatGPT, has led to a rise in its awareness and adoption across virtually all medical disciplines, with research on AI advancing to the phase of randomized controlled trials.^[4] It has played a part in enhancing operational efficiency, accelerating research or drug development and recently as a diagnostic tool, especially in imaging.^[5-9] Several studies have also explored its role in medical education, some benchmarking the performance of ChatGPT software against international examinations such as the United States Medical Licensing Examination and the Fellowship of the Royal College of Radiologists final examination; results have shown varying efficacy with some promising outcomes although complex procedural knowledge or “non-interpretable” data pose some limitations.^[10-12] Furthermore, authors have explored its role in predicting clinical risk and healthcare costs, allowing automated risk adjustment algorithms to support hospital processes.^[13,14]

In the UK, informed consent is a fundamental principle in healthcare, ensuring that patients are fully informed about and actively participate in decisions regarding their treatment. It is rooted in the ethical and legal framework that prioritizes patient autonomy, informed decision-making, and the right to consent to or refuse treatment.^[15,16] The General Medical Council (GMC) provides detailed guidance on obtaining consent, emphasizing the importance of communication, respect for patient autonomy, and the provision of adequate information.^[1] The key principles of consent are that it is: informed, patient is aware of what the treatment involves, including benefits, risks, and alternatives; it is voluntary, the decision made by the patient and not influenced by third parties; and patients have the mental capacity to understand, retain, weigh up, and communicate their decisions. Over the past few decades, technological and scientific advancements alongside cultural and social shifts have influenced the nature of the doctor-patient relationship.^[2] This has moved from a paternalistic model, where the doctor held exclusive decision-making power, to a more therapeutic partnership. In this modern relationship, the patient’s autonomy in decision-making is crucial, making them an active participant in their healthcare decisions. Accordingly, this necessitates relatively more time to ensure that the consent process adequately addresses the patient’s queries in a comprehensive manner. Failing to adequately inform patients about medical procedures can lead to costly outcomes, such as reduced patient satisfaction rates, poor compliance to treatment plans, and both the underuse and overuse of the healthcare system.^[17]

Current practice necessitates clinicians to engage in direct discussions with patients for consent. However, appointment times, intricate procedural details, and the comprehensive discussion of risks, benefits, and complications may not always be exhaustively covered, particularly for complex procedures. Consequently, patients may sometimes need additional clinic visits, perhaps to dedicated “consent clinics.” While previous studies have explored the potential of ChatGPT software in the consent process, its limitations in facilitating reciprocal conversation and addressing complex areas are evident.^[18,19] With the advent of ChatGPT 4o, which can be primed with relevant treatment information, risks, benefits, and alternatives, a genuine discussion akin to traditional clinic consultations becomes conceivable. The results of this study reveal a Likert scale average of 4.9 for the overall impression of the ChatGPT 4o consent document against standards used in routine clinical practice; the domains of risks and benefits showed perfect agreement and this may reflect the same and recent evidence base being used by both ChatGPT and clinical guidelines. It was noted that certain descriptions of procedural elements of an intervention were lacking; Likert scores of 3 and 4 for THR and image-guided joint injection, respectively. For THR, descriptions did not encompass skin incision, scar, and surgical approach used. It is recognized that these factors may vary between surgeons and institutions and although this affected the results of this study, it is possible for institutions to customize these inputs to meet their local practice.

Adoption of ChatGPT 4o for consenting carries numerous advantages for clinicians, patients, and the healthcare system as a whole. First, while it may not entirely replace conventional consent methods, it could serve as a supplementary tool, especially for patients seeking further information post-appointment. Second, it may assume the role of a triage tool where unanswered queries could be directed to the appropriate clinical team. Overall, this could reduce clinician burden, enhance patient accessibility to information, and streamline processes within an already strained healthcare system.^[20] Third, there is an advantage in the consistency of consent information provided. For example, information relayed by physicians is susceptible to variation and human error, particularly concerning details related to risks or complications, which may differ among clinical practitioners. In addition, consenting is often conducted by junior members of the medical team who, while possessing a working understanding of the procedure, may lack the extensive knowledge of a consultant responsible for the patient’s care (by virtue of experience).^[21] Although it is expected that such discussions pertaining to such complex cases are performed by a senior clinician, AI offers the potential for a consistent response supported by evidence-based guidelines and statistics, ensuring uniformity in the information provided. Finally, the repeated interaction and ability to review prior queries and answers that patients have when utilizing ChatGPT may improve recollection and understanding of their procedure up to and after it has been performed.^[22]

The ethical implications of integrating AI into direct clinical care prompt scrutiny.^[23] The World Health Organization has addressed this issue through a global report, offering a framework to foster ethical development in this rapidly evolving space.^[24] First, AI software requires rigorous trials and randomized control studies to validate its efficacy against established gold standards (which typically take the form of existing clinical guidelines or clinician judgment). Our study was compared against a standard of established guidelines that are currently followed and were respectively compared by experienced consultants. Furthermore, establishing a requisite level of performance for AI software is important. While a “purist” stance may advocate for absolute accuracy and consistency, it is inevitable that in actual clinical practice mistakes, omissions, and human error will occur.^[25] Consequently, this expectation may not align with the existing standard of care observed in non-AI settings. Third, it is imperative to address the medicolegal implications of AI.^[26] While clinicians typically bear responsibility for errors that they had been involved in during the consent process, it is somewhat unclear who would be held accountable if AI were to provide misinformation; this includes developers of the AI software, the vendor, the clinician utilizing the technology, and the department responsible for its procurement. Governance in this domain necessitates meticulous attention, particularly during AI implementation in healthcare. Finally, despite promising study outcomes, successful integration of AI software hinges on patient partnership. The perception of AI as a computerized system without real human interaction may impact patient acceptance, particularly regarding its role as a comprehensive consenting tool.

CONCLUSION

ChatGPT 4o is a new AI tool featuring improvements in speed and capability over previous versions such as GPT-4 and GPT-4 Turbo. This model combines text, vision, and audio functionalities, making it a versatile tool for more complex. When comparing consent for radiological and surgical orthopedic procedures facilitated by ChatGPT 4o against established clinical consent guidelines, it demonstrates high levels of accuracy, especially in communicating the parameters of risks and benefits of a procedure. Future prospective studies are necessary to further explore the effectiveness of ChatGPT in obtaining medical procedure consent, with careful consideration of issues surrounding adoption and governance in any future implementation.