Nurses’ Evaluations of the Feasibility and Clinical Utility of the Use of the Critical-Care Pain Observation Tool-Neuro in Critically Ill Brain-Injured Patients

Richard-Lalonde, M., Bérubé, M., Williams, V., Bernard, F., Tsoller, D. & Gélinas, C. (2019). Nurses’ Evaluations of the Feasibility and Clinical Utility of the Use of the Critical-Care Pain Observation Tool-Neuro in Critically Ill Brain-Injured Patients. Science of Nursing and Health Practices / Science infirmière et pratiques en santé, 2(2), 1–18. https://doi.org/10.31770/2561-7516.1045 Résumé de l'article


INTRODUCTION
The intensive care unit (ICU) admits a wide variety of patients with critical illnesses, all at risk to experience pain either at rest (Chanques et al., 2007) or during standard care procedures (Puntillo et al., 2014). In Canada, the Canadian Institute for Health Information (CIHI, 2007) reports that there are on average 50,000 stroke-related and 23,000 traumatic brain injury (TBI)-related hospitalizations annually. These patients may require to be admitted to the ICU if their condition is unstable and require intensive monitoring (CIHI, 2016).
Many ICU patients, and especially those with a non-traumatic and traumatic brain injury, are unable to self-report during their ICU stay due to several factors affecting their ability to communicate such as an altered level of consciousness, use of sedative agents, mechanical ventilation, and the severity of their condition. Therefore, the patient's self-report may not be possible to obtain, so behavioral pain scales are alternative measures. Few tools were developed and validated for use in brain-injured patients. The Nociception Coma Scale-Revised was tested for use in acutely ill brain-injured patients (n= 60) including the ICU (Chatelle et al., 2012(Chatelle et al., , 2016, however it would require some adaptation as the item related to verbal response is not applicable to mechanically ventilated patients. Also, further validation with larger samples representative of brain-injured patients hospitalized in the ICU is necessary. Specifically developed for ICU patients unable to self-report, the Behavioral Pain Scale or BPS (Payen et al., 2001) and the Critical-Care Pain Observation Tool (CPOT) by Gélinas, Fillion, Puntillo, Viens, & Fortier (2006) were tested in brain-injured ICU patients . A low effect size was found for the facial expression item of the BPS (Ribeiro et al., 2019) and a grimace score of 2 and muscle rigidity score of 2 on the CPOT were not frequently observed (Joffe, McNulty, Boitor, Marsh, & Gélinas, 2016) raising attention about suitability of these items in this specific ICU population. In alignment with these findings, recent evidence has shown that braininjured ICU patients express unique behaviors (Arbour et al., 2014;Roulin & Ramelet, 2014), and guidelines highlighted the need to revise behavioral pain scales to make their content more suitable to these patients (Barr et al., 2013;Devlin et al., 2018).
The CPOT (Gélinas et al., 2006), recommended in these guidelines, was recently revised into the CPOT-Neuro based on braininjured ICU patients' observations, as well as clinicians' and family members' perspectives Gélinas et al., 2018a;Vanderbyl & Gélinas, 2017). Observation data informed us on common and predictive pain behaviors in brain-injured ICU patients. Brow lowering was commonly observed during standard care procedures (i.e., turning, endotracheal suctioning) known to be painful, and many facial expressions (i.e., grimace, orbit tightening, eyes tightly closed, mouth opening) as well as tearing were predictive of self-reported pain intensity in this patient group . Clinicians' perspectives on the relevance of pain behaviors in brain-injured ICU patients supported our observations and the patient's level of consciousness was identified to influence some behaviors (e.g., verbal complaints of pain are more likely to be observed in conscious patients compared to those who are unconscious) (Gélinas et al., 2018a). Interestingly, seven family members of brain-injured ICU patients described similar behaviors (e.g., brow lowering, tearing) they thought were indicative of pain in their loved one (Vanderbyl & Gélinas, 2017).
Feasibility and clinical utility are relevant concepts to evaluate in relation to the use of assessment tools (Duhn & Medves, 2004;Smart, 2006). More specifically, feasibility refers to: duration of training, quickness of use, clarity on the use of the tool, simplicity to understand, and easiness to complete the tool. Clinical utility refers to the usefulness of the tool in practice (Smart, 2006): recommendation of the tool's use, the helpfulness of the tool, and its influence on practice from the perspectives of the nurses were evaluated.

OBJECTIVE
This study aimed to describe the nurses' evaluations of the feasibility and clinical utility of the CPOT-Neuro for pain assessment in brain-injured ICU patients in the context of the validation of its use at the bedside. Based on similar evaluations related to the use of the original CPOT, it was expected that the CPOT-Neuro would be feasible and useful from the point of view of ICU nurses who used it at the bedside in the context of this study (Gélinas, 2010;Gélinas et al., 2014).

DESIGN, SAMPLE AND SETTING
This descriptive study took place as part of a larger research project on the validation of the CPOT-Neuro, prior to its implementation in ICU clinical practice. The study was conducted in two university-affiliated trauma centres in Montreal (Quebec, Canada), each of which provided ethical approval (English site: UHC-15-994 and French site: 2015-1164). The inclusion criteria of ICU patients cared for by the nurse participants were: >18 years old, admitted for brain injury (including traumatic brain injury, stroke, or any other causes) for less than 4 weeks and a Glasgow Coma Score >3/15. Nurse participants had to have used the CPOT-Neuro at least for one patient in order to be eligible to evaluate its feasibility and clinical utility.
All ICU nurses who worked day and/or evening shifts from both settings were invited to participate, but not those from the night shift because the validation of the tool was only conducted during the day and the evening shifts. Lunch information sessions about the study were organized in each site in collaboration with nursing managers. Participation to this study was voluntary. Those who agreed to participate provided their written informed consent.
A total of 26 nurses were recruited in the site designated to validate the French version of the CPOT-Neuro, and 33 nurses were recruited in the site designated to validate the English version of the CPOT-Neuro, for a total of 59 nurses who were trained to use the CPOT-Neuro. However, only 27 (46 %) nurses actually used the CPOT-Neuro in the ICU setting with participating patients during the study period (12 from the French site and 15 from the English site). The main reason why other nurses didn't use the CPOT-Neuro was because they were not assigned to enrolled patients. Four ICU nurses from the French site no longer worked in the ICU after the launching of the study, and one withdrew from the study. Three ICU nurses from the English site were on maternity leave after the study was launched. All of the nurse participants completed the feasibility and clinical utility questionnaire at the end of the study.

Critical-Care Pain Observation Tool-Neuro.
The CPOT-Neuro has recently been derived from the original CPOT (Gélinas et al., 2006;Gélinas et al., 2018a). It includes five behavioural items: a) facial expression, b) autonomic responses, c) body movement, d) ventilator compliance/vocalization, and e) muscle tension. Except for autonomic responses which were newly added in the CPOT-Neuro, all other items were part of the original CPOT and were modified to be more suitable to brain-injured ICU patients based on previous patients' observations, as well as clinicians' and families' perspectives. In summary, facial expression, body movement and ventilator compliance/vocalization are scored on a scale from 0 to 2, whereas autonomic responses and muscle tension are scored from 0 to 1, for a total CPOT-Neuro score ranging from 0 to 8. The CPOT-Neuro was initially created in French (Appendix 1) and then back-translated into English (Appendix 2). Details related to its development are described in another paper (Gélinas et al., 2018a).
Feasibility and Clinical Utility Questionnaire. The self-administered questionnaire on the CPOT-Neuro feasibility and clinical utility consisted of eight closed-ended questions or statements to be rated on a Likert scale from 1 (totally disagree) to 4 (totally agree). The feasibility items included: duration of training, quickness of use, clarity on the use of the tool, simplicity to understand, and ease to complete. The clinical utility items included: recommendation to use the tool, helpfulness of the tool, and influence on practice. All questions had space for short answers, allowing further elaboration to any given rating (Appendix 3). Two open-ended questions were also included for participants to describe what supported them in the use of the CPOT-Neuro and how the tool could be improved. The questionnaire was inspired by the version developed by Puntillo and colleagues (2002) which was adapted from a previous version used to evaluate the feasibility and clinical utility of the original CPOT (Gélinas, 2010). In addition, nurses were asked about their sociodemographic information, including age, sex, level of education, years of experience in the ICU and as a nurse.

PROCEDURES
ICU nurses were trained in small groups to use the CPOT-Neuro in French or English during the validation testing period (June 2015 to October 2016). The training session lasted 45-minute and was given by the principal investigator or a clinical coordinator and included the description of the CPOT-Neuro items and the scoring methods. Nurses practiced scoring levels of pain with the CPOT-Neuro using three patient videos, and scores were discussed within the group. The goal was to obtain scores with no more than one-point difference. When a difference in two points or more were found, the scoring methods were clarified before moving to the next patient video. Overall, 15 (25.4 %) nurses had appropriate scores for all three patient videos, 29 (49.2 %) nurses had difference in scores >2 points for one patient video, and 15 (25.4 %) nurses had difference in scores >2 points for two patient videos. No nurse had difference in scores >2 points for the three patient videos. A total of 19 (73.1 %) nurses from the French site and 25 (75.8 %) nurses from the English site had appropriate scores for all three patient videos or had difference in scores >2 points for only one patient video.

DATA COLLECTION
Use of the CPOT-Neuro occurred with braininjured ICU patients from whom written informed consent was obtained. For patients unable to consent, the family representative provided written consent on their behalf. The trained nurses, if assigned to these recruited patients (or assigned to a nearby patient and available to participate), were then invited to participate as inter-raters with the research personnel to assess their patients' pain at rest, during a nonnociceptive procedure (one-minute gentle touch on the arm or non-invasive blood pressure) as well as during a painful procedure such as turning, endotracheal suctioning, intravenous or arterial line insertion, tube or drain removal, and wound care (Gélinas et al., 2018b). If the patient was administered an opioid, the nurse assessed the patient's pain prior to as well as 15 minutes after the opioid administration. Nurses and research personnel were blinded to one another's CPOT-Neuro scoring. The observations of the patients at rest and post-opioid administration were completed over one minute, whereas the observations of the painful procedures lasted for the duration of the procedure in order to allow for the detection of any relevant behavior included in the CPOT-Neuro.
At the end of the validation testing period (October 2016), nurses who had used the CPOT-Neuro were invited to evaluate the tool's use by completing the feasibility and clinical utility questionnaire, and those who did received $20 in compensation for their time. The questionnaire was handed in person or distributed in work mailboxes and collected in person by research personnel at both sites. Nurses were identified by numeric codes in order to keep the answers confidential.

DATA ANALYSIS
Descriptive statistics (i.e., frequencies and medians) were calculated for each question using SPSS 23.0 to describe the nurse sample and their responses on the feasibility and clinical utility questionnaire. Mann-Whitney U tests, Fisher's exact test and likelihood ratios were performed (as Chi-square assumptions were violated) to compare responses of nurses who used the CPOT-Neuro French or English version. Written responses were also compiled by topics and presented in order of frequency of occurrence.

NURSE PARTICIPANTS
Sociodemographic information of nurse participants is described in Table 1.  (80) 3 (20) 12 (100) 0 (0) 24 (88.9) 3 (11.1) .231 Of the 59 ICU nurses who were trained to use the CPOT-Neuro, 27 (46 %) used the CPOT-Neuro at the patient's bedside during the validation testing period. However, since three nurses left on maternity leave, one withdrew from the study and four left the unit over the course of the study, it may be more accurate to report the participation rate as 27 (52 %) out of 51. Nurses from the English site used the CPOT-Neuro for 5.4 pain assessments on average (median=4.0, min=1, max=12). Nurses from the French site used the CPOT-Neuro for 6.5 pain assessments on average (median=5.5, min=1, max=17). At the English and the French sites, two nurses and one nurse used the tool only for one pain assessment, respectively. The ages of the nurse participants from both sites ranged from 26 to 60 years old. The reported years working in the ICU ranged from 2 to 34 years and the years spent working as a nurse ranged between 4 and 37 years. Most of the participants were female nurses and most held a permanent full-time ICU position (significantly more permanent full-time positions at the English site than at the French site: p = .008). Regarding education levels of participants, most reported having completed at least a bachelor's degree with the majority (59.3 %) holding a bachelor's degree in nursing. There was no significant difference between both sites in terms of age, years of experience as a nurse or in the ICU, sex, or highest level of education.

NURSE PARTICIPANTS
Patients (n=54) cared for by nurse participants over the course of the study were aged between 19 and 95 years old (mean = 56.37, SD = 22.89). They were admitted for brain injury to the ICU: traumatic brain injury with or without other trauma (n=41), ischemic stroke (n=5), hemorrhagic stroke (n=4) or brain injury from other causes (n=4), including subdural hematoma, subarachnoid hematoma, and hydrocephalus. Twenty of the patients had no alteration of consciousness (GCS 13-15), and 34 had an altered level of consciousness (GCS 9-12).

CPOT-NEURO FEASIBILITY AND CLINICAL UTILITY
Results from the evaluation of feasibility and clinical utility of the CPOT-Neuro are presented in Table 2 and some illustrative quotes from the written comments are described in Table 3. Regarding feasibility, all nurses agreed (scores of 3 and 4/4) that the training duration was sufficient (median=4.0), that the directives of use were clear (median=4.0), and that the tool content was simple to understand (median=3.0). Although medians of 3.0 were obtained for other questions, a few nurses disagreed on the feasibility items that the tool was quick to use (11.1 %) and easy to complete (7.7 %). Regarding clinical utility, a few nurses disagreed that the tool was helpful for practice (7.7 %), that it influenced their practice (15.4 %) and that they would recommend it routinely (11.1 %). Of those who disagreed with certain aspects of feasibility or clinical utility, two did not elaborate on the reason for their rating. Responses did not differ between nurses from the two trauma centres who used either the CPOT-Neuro French or English version (Mann-Whitney U tests, p>.05). Table 3 presents comments provided by the participating nurses about the feasibility and clinical utility of the CPOT-Neuro based on their use of it at the bedside. The most frequently reported comment was about being more attentive to their patients' nonverbal cues. Over half of the nurses who used the CPOT-Neuro also reported now having access to an adequate tool to assess pain in their patients with brain injury unable to selfreport. Almost half of the nurses expressed a desire to get more training and practice using the CPOT-Neuro. Eleven nurses also wrote about how beneficial they found it to have support from and access to resource persons (i.e., research staff) who knew the CPOT-Neuro tool very well. Some nurses expressed that the tool was too lengthy or reported difficulty to assess pain in agitated patients.

DISCUSSION
The feasibility and clinical utility of the CPOT-Neuro were evaluated for the first time in this study. Overall, most nurse participants reported either agreeing or totally agreeing with all aspects of feasibility and clinical utility criteria that were assessed. Findings were similar to those obtained for the validation of the original CPOT (English and French version) using the same questionnaire with feasibility and clinical utility endorsed by more than 80 % of nurse participants in three Canadian trauma ICUs (n=33 in Gélinas, 2010;n=35 in Gélinas et al., 2015;n=12 in Topolovec-Vranic et al., 2013). The CPOT was also found to be easy to learn, accurate, and useful with a median rating of 8/10 (10 being the highest score) for all three items by 20 nurses in an American medical ICU (Chanques et al., 2014). In a pilot implementation study of the CPOT in two Canadian medical-surgical ICUs, 23 nurses rated the CPOT as feasible and clinically useful, and expressed a desire to have the tool integrated in their flowsheet (Bourbonnais, Malone-Tucker, & Dalton-Kischei, 2016), as highlighted by nurse participants in our study.
For those who did not agree with certain aspects of the feasibility or clinical utility (n=10, including 5 from the French site and 5 from the English site), written comments were helpful to provide further enlightenment. Regarding feasibility and consistent with findings related to the training of the original CPOT, the use of patient videos was identified to be the most useful learning strategy by ICU nurses . Similarly to studies with the original CPOT (Gélinas, 2010;Gélinas et al., 2015), some participants disagreed with the CPOT-Neuro being quick and/or easy to use. Nurses expressed the need for more practice with its use. For those who disagreed with its ease to complete, the high number of items in the CPOT-Neuro was challenging to assess at the same time. It is worth mentioning that for the purpose of the validation study, nurses were asked to check all behaviors ("descriptors") they observed in addition to their CPOT-Neuro scores. However, in the implementation of the tool into practice, only CPOT-Neuro scores would need to be provided. Furthermore, because the assessment of the feasibility and clinical utility of the CPOT-Neuro was conducted prior to the tool being implemented in clinical practice, the results of the study could have been influenced by the nurses not having fully integrated the tool into practice, as we would expect to occur over time. Indeed, clinical utility ratings are expected to change from the preimplementation stage, and through the various implementation stages, as nurses acquire more experience using a tool (Proctor et al., 2011). Thus, it would be relevant to re-evaluate these outcomes once the tool has been implemented in clinical settings.
According to clinical utility, few nurse participants disagreed with the CPOT-Neuro being helpful for practice or that it had influenced their practice (in a pre-implementation context) because it was not yet integrated into practice, and it was challenging to use in agitated patients. Nurses suggested to include the CPOT-Neuro on their flowsheet, to involve champions in the use of the tool to support them, and to have booster training sessions if needed.
As previously reported with the original CPOT Topolovec-Vranic et al., 2013), there was some confusion between the CPOT-Neuro, which was developed to evaluate painrelated behaviors, and the Richmond Agitation Sedation Scale (RASS) (Sessler et al., 2002), which evaluates behaviors related to the level of agitation and sedation.
Co-management of agitation/sedation and pain should be included in educational training of the ICU care team (Payen & Chanques, 2012). The use of standardized assessment tool such as the CPOT-Neuro could provide common language within the ICU team and optimize pain assessment in critically ill braininjured patients.

LIMITATIONS
This study had some limitations. First, the participation rate was lower than expected considering the total number of nurses who initially received the training. This was mainly because nurses who were trained were not assigned to enrolled patients or were not present on the unit when data collection occurred. Second, some participants who disagreed with feasibility and clinical utility items did not mention a reason for their disagreement. Thus, it was not always possible to understand the reasons behind the ratings given, which could have been better elucidated with qualitative methods (e.g., semistructured interviews). Third, the questionnaire was based on some aspects of feasibility and clinical utility as described in the literature (Duhn & Medves, 2004;Puntillo et al., 2002;Smart, 2006), but its development did not include a content validation process with experts. We used the same version as the one used for the evaluation of the original CPOT (Gélinas, 2010) to allow us to compare findings of both versions of the tool. However, a revised version of the questionnaire inspired by a conceptual framework and content validation with experts (Voepel-Lewis et al., 2008) could be developed and used in future implementation studies. Finally, nurses were not specifically asked about which contextual factors such as resources, culture, and leadership might have influenced their evaluation.

CONCLUSION
This was the first study to evaluate the feasibility and clinical utility of the CPOT-Neuro for pain assessment in critically ill brain injured ICU adults. The findings are consistent with previous studies on the feasibility and clinical utility of the original CPOT (Chanques et al., 2014;Gélinas, 2010;Gélinas et al., 2015;Topolovec-Vranic et al., 2013). Indeed, the feasibility and clinical utility of the CPOT-Neuro were positively evaluated by ICU nurses. Directives on the use of the tool could include specific information regarding the assessment of patients with complex conditions such as agitation and how to consider sedation along with pain management. Future research on clinical implementation of the CPOT-Neuro is necessary to evaluate the feasibility and clinical utility of the tool's use in daily practice, and its impact on ICU pain management practices and patient outcomes.
Authors' contribution: MRL: substantial contributions to the analysis and interpretation of data for the work, drafting the paper and revising it critically, final approval of the version to be published. and agreement to be accountable for all aspects of the work. MB: substantial contributions to the design of the work and interpretation of the data for the work, revising the paper critically, final approval of the version to be published, and agreement to be accountable for all aspects of the work. VW: substantial contributions to the acquisition of data for the work, revising the work critically, final approval of the version to be published, and agreement to be accountable for all aspects of the work. FB: substantial contributions to the design of the work, revising the paper critically, final approval of the version to be published, and agreement to be accountable for all aspects of the work. DT: substantial contributions to the acquisition of data for the work, revising the paper critically, final approval of the version to be published, and agreement to be accountable for all aspects of the work. CG: substantial contributions to the conception/design of the work and interpretation of the results for the work, drafting the paper and revising it critically, final approval of the version to be published, and agreement to be accountable for all aspects of the work.