Development, validation, and reliability testing of the standard treatment workflow checklist

Rangisetti Naga Prudhvi Teja; Arjun Thimmiah; Kavita Kachroo; Syed Abdul Khader Moinudeen; Vanjavakam Sahithya; Jitendra Sharma; Ashoo Grover

doi:10.18203/2394-6040.ijcmph20243289

Authors

Rangisetti Naga Prudhvi Teja Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Arjun Thimmiah Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Kavita Kachroo Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Syed Abdul Khader Moinudeen Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Vanjavakam Sahithya Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Jitendra Sharma Department of Health Research, Kalam Institute of Health Technology, Visakhapatanam, Andhra Pradesh, India
Ashoo Grover Indian Council of Medical Research, New Delhi, Delhi, India

DOI:

https://doi.org/10.18203/2394-6040.ijcmph20243289

Keywords:

Cronbach’s alpha, Validation, Item analysis, Reliability, Checklist, Content validation

Abstract

Background: Standard Treatment Workflows (STWs) are systematically developed documents designed to enhance clinical care and patient outcomes. Validating these workflows is essential for their adoption across diverse clinical settings. This study aimed to develop and validate a checklist for evaluating STWs.

Methods: The study was conducted in three phases. Initially, items were generated through a comprehensive literature review and expert consultations, followed by pre-testing (N=50). In the second phase, content validation (N=43) was performed using the Content Validity Index (CVI) and Content Validity Ratio (CVR) to assess the relevancy and essentiality of each item. Item-level (I-CVI) and scale-level (S-CVI) indexes were calculated. Items were accepted, modified, or rejected based on these scores. In the third phase, pilot testing (N=30) assessed the checklist's reliability and calculated Cronbach's alpha.

Results: The final checklist comprised 33 close-ended and one open-ended items. Each item was deemed relevant and essential, with I-CVIs ranging from 0.96 to 1.00 and S-CVI/UA and S-CVI/Ave values of 0.83 and 0.99, respectively. CVR values ranged from 0.85 to 1.00, with an average CVR of 0.97. Reliability testing yielded Cronbach’s alpha values of 0.863 and 0.961 for the two parts of the checklist.

Conclusions: The developed checklist is valid and reliable, with satisfactory CVI, CVR, and Cronbach’s alpha values. It can be used as a standard tool for assessing STW validation and effectively capturing essential aspects of STW evaluation.

Metrics

Metrics Loading ...

References

STWs. Indian Council of Medical Research | Government of India. Available at: https://www.icmr.gov.in/stws. Accessed on 15 May 2024.

Grover A, Bhargava B, Srivastava S, Sharma LK, Cherian JJ, Tandon N, et al. Developing Standard Treatment Workflows-way to universal healthcare in India. Front Public Health. 2023;11:1178160.

EQUATOR Network. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Available at: https://www.equator-network.org/reporting-guidelines/right-statement/. Accessed 12 July 2024.

EQUATOR Network. The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines. Available at: https://www.equator-network.org/reporting-guidelines/the-agree-reporting-checklist-a-tool-to-improve-reporting-of-clinical-practice-guidelines/. Accessed 24 June 2024.

Taherdoost H. Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. Int J Acad Res Manage. 2016;5:28-36.

Shi J, Mo X, Sun Z. Content validity index in scale development. Zhong Nan Da Xue Xue Bao Yi Xue Ban. 2012;37(2):152-5.

Wynd C, Schmidt B, Schaefer M. Two quantitative approaches for estimating content validity. Western J Nurs Res. 2003;25:508-18.

Yusoff MS. ABC of content validation and content validity index calculation. Educat Medi J. 2019;11(2):49-54.

Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382-5.

Davis LL. Instrument review: Getting the most from a panel of experts. Applied Nursing Research. 1992;5(4):194-7.

Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 200;30(4):459-67.

Bujang MA, Omar ED, Foo DHP, Hon YK. Sample size determination for conducting a pilot study to assess reliability of a questionnaire. Restor Dent Endod. 2024;49(1):e3.

Jansen M, Doornebosch AJ, de Waal MW, Wattel EM, Visser D, Spek B, et al. Psychometrics of the observational scales of the Utrecht Scale for Evaluation of Rehabilitation (USER): Content and structural validity, internal consistency and reliability. Arch Gerontol Geriatr. 2021;97:104509.

Gottems LBD, Carvalho EMPD, Guilhem D, Pires MRGM. Good practices in normal childbirth: reliability analysis of an instrument by Cronbach’s Alpha 1. Rev Lat Am Enfermagem. 2018;26:e3000.

Ayre C, Scally A. Critical Values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development. 2013;47:79-86.

Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4(2):165-78.

Sharma S, Pandit A, Tabassum F. Potential facilitators and barriers to adopting standard treatment guidelines in clinical practice: An Indian context. Int J Heal Care Quality Assur. 2017 ;30(3):285-98.

Bano R, Gupta S, Shekhar C. Translational research in biomedical sciences in India: Challenges, observations & national perspectives. Ind Med Res. 2020;152(4):335.

Barile G, de Rosa G, Papadia P, Muci G, Capodiferro S, Corsalini M. Clinical comparative study for validation of digital impression reliability with the gypsum check: a simple and fast way to evaluate the trueness and accuracy of implant-supported rehabilitation. Appli Sci. 2023;13(13):7358.

Lahariya C, Sharma S, Agnani M, de Graeve H, Srivastava JN, Bekedam H. Attributes of standard treatment guidelines in clinical settings and public health facilities in India. Indian J Community Med. 2022;47(3):336-42.