Revista Americana de Medicina Respiratoria - Volumen 21, Número 4 - Diciembre 2021

Artículos Originales

Validation and reproducibility of the COPD-PS questionnaire for the screening of COPD: Bogotá, Colombia

Abstract

Introduction: Chronic obstructive pulmonary disease is a condition with high prevalence worldwide. It is preventable and treatable, but with very high levels of underdiagnosis. The use of screening tools is imperative. These tools are easily applied, interpreted, and validated in different populations and help not only the clinician to confirm the diagnostic suspicion, but also the patients to become aware of their disease. The objective is to validate the COPD-Population Screener Questionnaire (COPD-PS) in one Colombian population.

Materials and Methods: A prospective cohort study was carried out. Participants had to be older than 40 years, show a good quality spirometry, and have completed the COPD-PS questionnaire twice. COPD was defined as FEV1/FVC < 0.7 and with a history of exposure to tobacco smoke. A reproducibility and validation analysis has been conducted.

Result: Out of a total of 2.199 potential subjects, 1.662 entered the final analysis; the prevalence of COPD in the study was 21.1%. With the COPD-PS questionnaire cut-off point of four, the sensitivity was 77.2% and the specificity was 46.3%, with an area under the receiver operating characteristic curve of: 0.66 (95% CI: 0.63-0.69) (p<0.01). An intraclass correlation coefficient of 0.817 (95% CI: 0.79-0.84) and a kappa coefficient of: 0.45 (95% CI: 0.31-0.59) (p<0.01) were obtained.

Conclusion: The COPD-PS questionnaire is a tool with high sensitivity and good reproducibility for the screening of COPD, and could suggest the use of a spirometry in subjects not diagnosed with this disease.

Key words: Chronic obstructive pulmonary disease; Questionnaire; Reproducibility of results; Lung function test.

Received: 01/18/2021

Accepted: 08/12/2021

Introduction

Chronic obstructive pulmonary disease (COPD) is the fourth leading cause of mortality throughout the world. It is estimated that it affects more than 5 percent of the world population¹, and 8-10% people older than 40 years^2-4. However, despite the fact of being a very common and well-known disease in the medical practice, it is highly underdiagnosed. The reason for this underdiagnosis is that patients do not attend their consultation because they have become used to the symptoms or they don’t know for sure if COPD can be treated. Also, the spirometry isn’t easily interpreted by some medical care providers and sometimes COPD is mistaken for asthma, because both conditions may show fixed airflow obstruction^5-7.

One of the questionnaires to be used was the Chronic Obstructive Pulmonary Disease Population Screener (COPD-PS)⁵. It consists of five items: three items related to symptoms (dyspnea, produc­tive cough and activity limitation), a fourth item related to smoking history (100 or more cigarettes smoked throughout the patient’s life) and the last item associated with age. It has a maximum score of 10 points¹⁰. In its initial validation study done in the United States in 2008, with a cut-off point ≥ 5, it showed 84.4% sensitivity and 60.7% specificity. Then it was validated in 2021 and 2014 in Spain and Japan with a cut-off point of 4, and obtained the following results: 93.6% sensitivity and 64.8% specificity in the Iberian country.

In Colombia, COPD is a prevalent disease, where the medical record or the use of screening question­naires may be useful for searching patients with this disease^{4, 15, 16}; however, in order to use screening instruments, it is necessary first to conduct validation studies ideally in the populations where they are intended to be used. In the country there is lack of data about the diagnostic performance of specific questionnaires such as the COPD-PS. The objective of this study is to determine the reproducibility and validation of the COPD-PS questionnaire in a Colombian population.

Methodology

We carried out a prospective cohort with the objective of determining the reproducibility and valida­tion of the COPD-PS questionnaire in a Colombian population. The participants attended an external consultation at the Clínica Universidad de la Sabana, Chía, Colombia. Patients were enrolled between 2015-2020.

Population

Individuals older than 40 years who were asked to undergo a spirometry to be performed in the lung function laboratory of a third level clinic, regardless of their indication. Individuals needed to have enough time available for this study and had to give their consent to answer a lung function question­naire. We excluded subjects whose spirometry didn’t fulfill the acceptability or reproducibility criteria of the American Thoracic Society (ATS) guidelines, or subjects with some kind of limitation on their communication that would complicate the development of clinical questionnaires. The spirometry was performed by the duly qualified and trained staff of the lung function laboratory, with previously cali­brated equipment. COPD was defined as the presence of fixed airflow obstruction with a FEV1/FVC ratio of less than 0.7 after the administration of the bronchodilator, according to the ATS definition, and with a smoking history of more than 10 packs per year.

Study variables and data gathering

Data gathering included a first visit to obtain demographic information: age, sex, race, weight, height, level of education, respiratory symptoms, history of exposure to tobacco smoke, wood smoke or other occupational exposure to smoke, history of medical diagnosis of COPD or asthma (confirmed or not confirmed) and lung function values. The COPD-PS questionnaire was applied at the end of the spi­rometry and subsequently during a second visit with a minimum difference of 15 days between them, where the questionnaire was repeated for the reproducibility analysis.

COPD-PS questionnaire

The Spanish version of the COPD-PS questionnaire consists of the following 5 questions: 1. For the past 4 weeks, how many times did you feel breathless? 2. Do you sometimes expel something such as mucus or sputum when you cough? 3. During the last year, did you reduce your daily activities due to your respiratory problems? 4. Did you smoke at least 100 cigarettes throughout your life? 5. How old are you? Each question has multiple answers with a score that goes from 0 to 2, with a maximum total score of 10 points. A value ≥ 4 is considered high risk of having chronic obstructive pulmonary disease (COPD) and need to do additional tests.

Sample size

In order to calculate the sample size, we used data from the studies of Martínez FJ, which showed 84.4% sensitivity and 60.7% specificity (5) for the COPD-PS questionnaire; also, the studies of Miravitles M, who reported 93.6% sensitivity and 64.8% specificity10, and of Tsukuya G, who found 67.1% sensitivity and 72.9% specificity for the same questionnaire11. With this information, for a COPD prevalence of 8.9%4, 95% confidence level and 2% precision we required a minimum of 2012 subjects. The subjects entered the study in a sequential manner, and those who didn’t fulfill the inclusion criteria were re­placed by others until the sample size was exceeded.

Data analysis

Data were obtained through the REDCap platform (web platform for building and managing online surveys and databases), and then they were analyzed with the statistical program SPSS, version 25. An initial description of qualitative variables was made in frequencies and percentages, and a description of quantitative variables was carried out using mean and standard deviation if their distribution was normal, or median and interquartile range if it wasn’t. Then, we compared and analyzed quantitative variables through the Student T Test or the Mann-Withney U Test according to their distribution and also compared qualitative variables through the Chi-Square Test. We calculated the sensitivity, specific­ity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR-), number needed to screen and number needed to harm. To evaluate reproducibility, we calculated the kappa coefficient and the intraclass correlation coefficient (ICC). Finally, we calculated the area under the receiver operating characteristic curve (AUROC) with the values of the COPD-PS questionnaire. We estimated the 95% confidence intervals for the reproduc­ibility measures obtained and considered p <0.05 as statistically significant.

Ethical considerations

The research protocol followed the international ethical guidelines of the Declaration of Helsinki, the country’s ethical considerations of the 8430 resolution of 1993 and the Data Protection Law 1581. It was presented and approved by the Research Committee of the Universidad de la Sabana and by the Ethics Committee of the Clínica Universidad de La Sabana.

Results

Of a total of 2199 potentially eligible subjects for the study, 1662 participated in the final analysis. Figure 1 shows the flow of subjects through admission to the study and the respective exclusions.

General characteristics of the population

The mean age was 70.04 years (SD: 10.8); 86.1% were mixed-race; 56.8% were males and there was a COPD prevalence of 21.2%. 89.04% had indicated some respiratory symptom and 51.9% of the popu­lation had completed basic primary education. Table 1 shows the characteristics of the population, respiratory symptoms, their background and the results of the lung function tests.

COPD: chronic obstructive pulmonary disease; PYI: packs-year index; X: average; SD: standard deviation; N: number or frequency; FVC: forced vital capacity; FEV1: forced expiratory volume in the first second.

Results of the COPDS-PS questionnaire

Table 2 shows the answers to the COPD-PS questionnaire, classified by each one of the questions and with the total score, both in subjects with and without COPD. It is evidenced that each question and the average score of the questionnaires in both groups have a statistically significant p value for the diagnosis of COPD. The average response time for the questionnaire was calculated at 1 minute.

X: average; SD: standard deviation; N: number or frequency.

Reproducibility of results and validation of the COPD/PS questionnaire

Table 3 shows the analysis of sensitivity and specificity characteristics of the COPD-PS questionnaire at different cut-off points. The Youden index that determined the highest sensitivity together with the specificity (0.77-0.46, respectively) used a cut-off point ≥ 4. Also, for that cut-off point we found a posi­tive predictive value of 2.279 and a negative predictive value of 0.883, and a positive likelihood ratio of 1.439 and a negative likelihood ratio of 0.490.

PPV: positive predictive value; NPV: negative predictive value; LR +: positive likelihood ratio; LR -: negative likelihood ratio; NNS: number necessary to screen; NNH: number necessary to harm.

Discussion

In this study we evaluate the validation of the COPD-PS questionnaire for the screening of COPD in a Latin American population; the performance is good when compared to the value of the FEV1/FVC ratio of less than 0.7 post- bronchodilator (AUROC:0.666 95% CI:0.633-0.699). This performance was similar to that found by Tsukuya G¹¹, who found an AUROC of 0.74 in the healthy overall population; however, with this score, AUROC values of even 0.88 have been found and reported by Miravitlles M10 in a population of patients at the first level of care. The sensitivity in the last study reached 93.6%; that is 16.4 points higher than the sensitivity observed in our study. Such difference may be due to a higher degree of exposure to smoking in the Spanish population studied by Miravitlles M et al., where the mean PYI was 42.79, twice as much as the one found in our study (PYI: 22).

The specificity found in our study (46.3%) is the lowest, compared to the validation studies of the COPD-PS questionnaire mentioned before; given the fact that this questionnaire evaluates respiratory symptoms, its capacity depends on the expression of such symptoms. In our population, the respiratory symptoms may eventually be expressed in a different way, maybe due to the high frequency of chronic bronchitis (CB) and the influence of the altitude. Subjects with CB can have mild obstruction not ex­pressing tobacco exposure exclusively, thus affecting this score^{17, 18}. On the other hand, Horner found that subjects who live in geographical areas higher than 1500 MASL show less respiratory symptoms, even the ones with COPD¹⁹. We shall remember that when doing screening tests, we expect sensitivity to be higher than specificity.

Other validations prior to our study used two cut-off points. In the first validation analysis conducted in the United States a cut-off point of five was used. It reported adequate sensitivity and specificity values⁴. But subsequent validation analyses conducted in Spanish and Japanese populations used a cut-off point of four, also used in our study, because with that number a better sensitivity can be ob­tained, with an acceptable reduction in the specificity of these populations^{10, 11}. The reason for these differences is not clear, but the cut-off points of different diagnostic tests may vary depending on the characteristics of the populations being evaluated, hence the importance of conducting the respective validation analyses. In our study, the COPD-PS questionnaire showed excellent test-retest reproduc­ibility and reliability, with very good intraclass correlation coefficients and kappa coefficients for the dichotomous responses, both in the overall population and in the COPD and non-COPD groups. This favors the use of this tool.

In our study, the prevalence of COPD was 21.1%, similar to that reported in the PUMA study in a hospital environment; and obviously higher than the one reported in a general Colombian and Latin American population. The PREPOCOL study carried out in a community-based population showed that 9 out of 100 people older than 40 years had COPD, setting a prevalence of 8.9%^{4, 20, 21}. It is known that a higher prevalence affects the positive predictive value and limits the extrapolation of the study results to the overall population; however, this questionnaire may be useful for the medical evaluation of different care levels where COPD underdiagnosis rates are still high, as shown by a study conducted in Argentina which found an underdiagnosis rate of 77.4%. And, in the PLATINO study the rate was 88.7%^{19, 22-24}.

To make the diagnosis of COPD, it is necessary to have tools that are easy to use, access and under­stand by the overall population, that can be used for case detection or screening both in the community and in the hospital environment, and that aren’t used exclusively by professionals specialized in internal medicine and pulmonology. The objective of having such tools is to raise awareness among patients about their health status but also to create an alert in the healthcare personnel that makes them do tests such as a spirometry for early diagnosis, and take preventive measures such as quitting smoking and timely treatment. Despite its validation, reproducibility and potential benefits as a detection test for COPD patients in our population, some weaknesses should also be taken into account: the type of population of the study, which may limit the extrapolation of results, and the lack of cost estimates with which the findings of this study could have been strengthened. In future studies in our population, we could include in the evaluation of these scores the assessment of risk factors such as exposure to wood smoke and the estimation of the economic impact upon the use of these tools.

Conclusion

The COPD-PS questionnaire is a tool with high sensitivity and good reproducibility for the screening of COPD, and could suggest the use of a spirometry in subjects not diagnosed with this disease.

Conflicts of interest

None.

Funding

This study hasn’t received any financial support or scholarship from public or private sectors. It is a non-profit study.

Ethical responsibilities

Protection of people and animals: The authors declare that for this research no experiments have been done on human beings or animals.

Data confidentiality: The authors declare they have followed the protocols of their work center on the publication of patient’s data.

Right to privacy and informed consent. The authors declare this article doesn’t provide any informa­tion about the patients.

References

1. Kochanek KD, Murphy S, Xu J, Arias E. Mortality in the United States, 2016. NCHS Data Brief. 2017; (293): 1-8.

2. Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006; 3(11): e442.

3. Kaplan A, Thomas M. Screening for COPD: the gap between logic and evidence. Eur Respir Rev. 2017; 26(143):160113.

4. Caballero A, Torres-Duque CA, Jaramillo C, et al. Prevalence of COPD in five Colombian cities situated at low, medium, and high altitude (PREPOCOL study). Chest. 2008; 133(2): 343-9.

5. Martinez FJ, Raczek AE, Seifer FD, et al. Development and initial validation of a self-scored COPD Population Screener Questionnaire (COPD-PS). COPD. 2008; 5(2): 85-95.

6. GOLD-2020-REPORT-ver1.0wms.pdf [Internet]. [cited on July 17 2020]. Available at: https://goldcopd.org/wp-content/ uploads/2019/11/GOLD-2020- REPORT-ver1.0wms.pdf

7. Albers F, Shaikh A, Iqbal A. Design, rationale, and baseline demographics of SEARCH I: a prospective cluster-randomized study. Int J Chron Obstruct Pulmon Dis. 2012; 7: 437-45.

8. de GesEPOC G de T. Proceso de la atención inicial al paciente con EPOC. Estrategias de cribado. Arch Bronconeumol. 2017; 53: 15-21.

9. Sui CF, Ming LC, Neoh CF, Ibrahim B. VitalQPlus: a potential screening tool for early diagnosis of COPD. Int J Chron Ob­struct Pulmon Dis [Internet]. 2015; 10: 1613-22. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4541542/

10. Miravitlles M, Llor C, Calvo E, Diaz S, Diaz-Cuervo H, Gonzalez-Rojas N. [Validation of the Spanish version of the Chronic Obstructive Pulmonary Disease- Population Screener (COPD-PS). Its usefulness and that of FEV(1)/FEV(6) for the diagnosis of COPD]. Med Clin Barc. 2012; 139(12): 522-30.

11. Tsukuya G, Matsumoto K, Fukuyama S, et al. Validation of a COPD screening questionnaire and establishment of diagnostic cut-points in a Japanese general population: the Hisayama study. Allergol Int. 2015; 64(1): 49-53.

12. Chung KS, Jung JY, Park MS, et al. Cut-off value of FEV1/FEV6 as a surrogate for FEV1/FVC for detecting airway obstruction in a Korean population [Internet]. International Journal of Chronic Obstructive Pulmonary Disease. 2016 [cited on July 17 2020]. https://www.dovepress.com/cut-off-value-of-fev1fev6-as-a-surrogate-for-fev1fvc- for-detecting-air-peer-reviewed-full-text-article-COPD

13. Kobayashi S, Hanagama M, Yanai M. Early Detection of Chronic Obstructive Pulmonary Disease in Primary Care. Intern Med [Internet]. 2017; 56(23): 3153-8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5742385/

14. Hidalgo Sierra V, Hernández Mezquita M, Palomo Cobos L, et al. Usefulness of The Piko-6 Portable Device for Early COPD Detection in Primary Care. Arch Bronconeumol. 2018; 54(9): 460-6.

15. Londoño D, García OM, Celis C, et al. Guía de práctica clínica basada en la evidencia para la prevención, diagnóstico, trata­miento y seguimiento de la enfermedad pulmonar obstructiva crónica (EPOC) en población adulta. Acta Médica Colomb. 2014; 39(2 SI (2)): 5-49.

16. Bernal Pinilla J. Carga de la enfermedad pulmonar obstructiva crónica en Colombia. 2015 [cited on July 17 2020]; http://repository.javeriana.edu.co/handle/10554/19457386 387

Validation and reproducibility of the COPD-PS questionnaire for the screening of COPD

17. Torres-Duque CA, García-Rodriguez MC, González-García M. Is Chronic Obstructive Pulmonary Disease Caused by Wood Smoke a Different Phenotype or a Different Entity? Arch Bronconeumol Engl Ed [Internet]. 2016; 52(8): 425-31. Available at: http://www.sciencedirect.com/science/article/pii/S1579212916301306

18. Gonzalez-Garcia M, Caballero A, Jaramillo C, Torres-Duque CA. Chronic bronchitis: High prevalence in never smokers and underdiagnosis— A population- based study in Colombia. Chron Respir Dis [Internet]. 2019; 16: 1479972318769771. Avail­able at: https://doi.org/10.1177/1479972318769771

19. Horner A, Soriano JB, Puhan MA, et al. Altitude and COPD prevalence: analysis of the PREPOCOL PLATINO-BOLD-EPI-SCAN study. Respir Res. 2017; 18(1): 162.

20. Rojas YG, Duque CAT, Figueredo M del C, et al. Estimación de la prevalencia de EPOC en Colombia a partir del Registro Individual de Prestaciones de Servicios de Salud (RIPS). Rev Colomb Neumol [Internet]. 2019; 31(1). https://revistas.aso­neumocito.org/index.php/rcneumologia/article/view/325

21. Schiavi E, Stirbulov R, Hernández Vecino R, et al; Puma Team. COPD screening in primary care in four Latin American countries: methodology of the PUMA Study. Arch Bronconeumol. 2014; 50(11): 469-74.

22. Echazarreta AL, Arias SJ, Del Olmo R, et al; Grupo de estudio EPOC.AR. Prevalence of COPD in 6 Urban Clusters in Ar­gentina: The EPOC.AR Study. Arch Bronconeumol (Engl Ed). 2018; 54(5): 260-9.

23. López Varela MV, Montes de Oca M. Variabilidad en la EPOC. Una visión a través del estudio PLATINO. Arch Bronconeumol [Internet]. 2012; 48(4): 105-6. Available at: http://www.archbronconeumol.org/es-variabilidad-epoc-una-vision-traves-articulo- S030 0289611003309

24. Hinojosa F, C E. Enfermedad pulmonar obstructiva crónica (EPOC). Acta Médica Peru [Internet]. October, 2009 [cited on July 27 2020];26(4):188-91. Available at: http://www.scielo.org.pe/scielo.php?script=sci_abstract&pid=S1728-591720090004000 01&lng=es&nrm=iso&tlng=es.