Development and Validation of a Risk Scoring System to Predict Pneumothorax in CT-guided Percutaneous Transthoracic Needle Biopsy

Introduction

Percutaneous transthoracic needle biopsy (PTNB) has been widely used to obtain lung lesion specimens because of its high diagnostic accuracy rate (> 90%) and low complication rate [1, 2]. Pneumo- thorax and pulmonary hemorrhage are the two most common com- plications caused by PTNB [3]. Pneumothorax is a common but potentially dangerous complication. In previous studies, the aver- age rate of pneumothorax episodes was reported to range from 1.1 to 29.8%, while the development of severe pneumothorax post-bi- opsy requiring intervention occurred in 1%-14.2% of patients un- dergoing CT-guided PTNB [2]. The risk factors for pneumothorax have been less studied [1-4]. An evidence-based and easily avail- able risk scoring system for predicting pneumothorax could be valuable for interventionists who perform PTNB.

Methods

This study was designed as a retrospective single-center study and was conducted in accordance with the Health Insurance Portability and Accountability Act. This retrospective study was approved by the Institutional Review Board Committee of the hospital. Patient informed consent requirements were waived.

Patients and Definitions

Patients were enrolled if they underwent CT-guided PTNB be- tween September 11, 2019, and August 30, 2021. Repeated PTNBs were recognized as new procedures in this study.

Biopsy Procedures

The procedure was conducted under CT guidance. All PTNBs were performed using a co-axial needle system with 18-gauge au- tomated biopsy needles (Huaxing Medical Devices, China). Ba- sically, there were two steps during the procedure. First, put the outer sheath into the lesion. Second, punctured the lesion. During procedure breath-holding was not required.

Assessment of Pneumothorax

Pneumothorax was assessed by an on-site CT scan. Cases with delayed pneumothorax were also included, which was defined as pneumothorax occurring within 24 hours after the procedure. Se- vere pneumothorax was defined as 2 cm depth from apex to cupola distance or interpleural distance at level of the hilum or even not meet the standard but with the obvious breathless symptom which need for thoracic closed drainage.

Covariates

A total of 47 variables were extracted, including patient-related variables, procedure-related variables, and lesion-related variables (radiomic feature extraction). The patient-related variables includ- ed patient demographics, comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, history of tumor), smoking history (packyear), and body mass index (BMI). The lesion-related variables were the following: location (divided into right/left upper, middle, and lower zones); central or peripher- al; size (largest diameter); lesion characteristics (nodules or mass- es were defined based on the largest diameter that was larger than 3 cm; patches were defined as not qualified to be a mass or nodule); pure ground glass, solid, subsolid, or cavitary lesions; subsolid le- sions were defined as containing a mixture of discrete solid and ground-glass areas; pleural adhesions, pleural effusion, lobular le- sions; bone metastasis; pericardial effusion; lesion enhancement (mild/moderate/strong), feasibility (defined as an increase of at least 10 Hounsfield units after contrast administration for each de- gree); and the histopathological biopsy results. Procedure-related variables included the patient’s position (su- pine or prone), number of lesion puncture, number of specimens, maximum puncture depth and number of CT scanning during the procedure. After the biopsy, the CT images were taken to detect procedure-related complications, including lung tissue bleeding and pneumothorax.

Statistical Analysis

Data were analyzed using statistical software (R 4.2; IBM Corp., Armonk, NY). Data are reported as the mean and standard devia- tion (SD) or as the median and interquartile range, as appropriate. The Gower distance, partitioning around medoids (PAM), and sil- houette width were evaluated in order to cluster the data of mixed types including all kinds of variables (continuous, binary or qual- itative [5-7]. The risk model was developed with the following steps. First, univariate analysis was performed to identify variables associated with pneumothorax caused by PTNB. Second, variables with a p value of <0.2 in the first step were included in a stepwise multivariate logistic regression analysis model to identify indepen- dent risk factors associated with pneumothorax [5]. The probabil- ity used for stepwise removal was 0.05 or less for entry and 0.05 or less for removal.

Then, we obtained a risk model and a forest tree plot. The area under the receiver operating characteristic curve (AUROC) was applied to assess the ability to predict pneumothorax [8]. The Hos- mer–Lemeshow test was carried out to evaluate the goodness of fit for the predicting model (p > 0.05). Third, we followed the method proposed by Sullivan et al. and Jun Duan et al. to create the risk scoring system [9, 10]. The variables in the prediction model were classified into clinically meaningful categories, and the midpoint was calculated. In each category, we set the lowest risk for pneu- mothorax as the within-group reference, which was assigned zero points; then, the weight in each category was calculated. Finally, one point was assigned to the category with the lowest weight, which was set as the between-group reference. The value of that weight in the other category divided by the between-group refer- ence was then calculated, which was rounded off to the nearest in- teger as the assigned points. The risk scale for pneumothorax was the sum of the points. Then, the total points associated with each category of each risk factor were calculated. The risk scoring sys- tem was then applied to the training and validation data. AUROC was applied for internal and external validation. The cutoff value was determined based on the AUROC.

The sample size was calculated by Buderer’s formula [11]. Based on clinical experience and a literature review, we estimated that the risk scoring system for pneumothorax achieved a 70% sensitivity and a 90% specificity. The average prevalence of pneumothorax was approximately 15% in previous studies [2, 4, 12]. We chose α = 0.05 and maximum marginal error of estimate = 5%. Thus, a minimal sample size of 807 cases was required in derivation co- horts. The validation cohort was a prospective aspect on this retro- spective study. Fig 1 shows flow chart of this study.

Figure1: Flow chart of this study.

Results

General Clinical Data

A total of 1001 patients were enrolled in the test cohort from Sep- tember 2019 to February 2021, while another 230 patients were enrolled in the validation cohort from March 2021 to August 2021. The baseline characteristics of patients who underwent PTNB are shown in Table 1. Among the derivation cohort, 611 were men (61%), and the mean age of all patients was 63.81±12, ranging from 14 to 90. In the validation cohort, 81 patients were men (70%), and the mean age of all patients was 66.56±10.86, ranging from 41 to 87.

According to the pathology reports, 660 were malignancies, 85 were confirmed as tuberculosis (n=83) and mycosis (n=2), 224 were nonspecific inflammations and 32 were judged to be nondi- agnostic in the derivation cohort. One hundred forty were malig- nancies, 18 were confirmed as tuberculosis (n=10) and mycosis (n=8), 60 were nonspecific inflammations and 16 were judged to be nondiagnostic in the validation cohort. Of all the 169 pneumo- thorax cases among the patients, (16.9%) occurred after PTNB, while 12 cases (7%) were defined as severe pneumothorax that needed closed thoracic drainage in the derivation cohort. The cor- responding rate in the validation cohort was 36 pneumothorax cas- es (15.7%), and 9 case (3.9%) needed intervention. The overall rate of hemoptysis was 3.4% (n=34), with most cases having vol- umes of hemoptysis < 20 mL. One patient died because of massive hemoptysis in the derivation cohort. The corresponding hemop- tysis rate in the validation cohort was 3%(n=7) and no one died.

Clustering for Pneumothorax

After calculating the silhouette width for clusters ranging from 2 to 10 for the PAM algorithm, we noticed that 2 clusters yielded the highest value (Fig 2A). After running the algorithm and select- ing two clusters (Fig 2B), we interpreted the clusters by running a summary on each cluster (Fig 2 B). Based on these results, it seemed that Cluster 1 was mainly younger/more number of lesion puncture/higher puncture depth with a heavy smoking index. Clus- ter 2, on the contrary, was mainly older/fewer number of lesion puncture/lower puncture depth with a light smoking index.

Derivation of a Risk Prediction Model for Pneumothorax and Development of a Risk Scoring System.

The final risk prediction model with an AUROC of 0.673 (Fig 3A) contained seven variables, including age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion (table 1), that were all statistically significantly associated with pneumothorax (p < 0.05) in the regression analyses in the derivation cohort, which could be visualized by a forest tree plot (Fig 2C). We used these seven variables to develop a risk scale to predict pneumothorax. The cat- egories and assignment of points in each variable are summarized in Tables 2 and 3. The scale ranged from -13 to 26 points. Risks associated with point totals are demonstrated in Table 4.

Table 1: Demographics of patients with or without pneumothorax undergone CT-guided PTNB in derivation and validation cohort. Data are presented as the mean ± SD or as a number with/without the percentage in parenthesis, as appropriate. Number of lesion puncture, Puncture number. Number of specimens, Specimen. Needle path bleeding, tissue bleeding. Lobularity of the lesion, lobularity. NA, not available.

B: The plot shows the two well-separated clusters of pneumothorax that PAM was able to detect. C: Logistic regression plot of odds ratios and 95% confidence intervals. Number of specimens, needle path bleeding, and lobular of the lesion was protective factors of pneumothorax. Number of lesion puncture, Punctimes. Puncture depth, Puncdep. Number of specimens, Speci. Needle path bleeding, tissueble. Lobu- larity of the lesion, lobular. Smoking index, Smok.

Validation of the Risk Scoring System

Internal Validation

In the derivation cohorts, the risk scoring system could only achieve an AUROC of 0.601 in the evaluation of predicting pneu- mothorax (Fig 3B). However, the subgroup analysis demonstrated that with increasing age, the predictive effect of the risk scoring system was high. In the group with patients older than 80 years, the AUROC was as high as 0.766 (Fig 3C/D). To obtain a high predictive effect, we excluded protective predictors, such as the number of specimens, needle path bleeding, and lobularity of the lesion, one by one to test whether the prediction effect could be higher, but combining all predictors had the highest AUROC (Fig 3E). The predictions made from the risk model were in alignment with the observed outcomes suggested by Hosmer-Lemeshow tests (P=0.33). The cutoff score suggested by Youden’s index was 15 points in the derivation cohorts.

Figure 3: A Receiver operating characteristic curve (ROC) of step wise logistic model for predicting pneumothorax. Cut-off was esti- mated as the point where sensitivity and specificity reach their maximum values in the curve.

B: ROC of risk scoring system for predicting pneumothorax in derivation cohort. Cut-off was estimated as the point where sensitivity and specificity reach their maximum values in the curve. C: Comparison of ROC curves for different age group by using the scoring system in derivation cohort. Red, blue, green line correspon- dences to age older than 60, 70, 80 years respectively. D: ROC of risk scoring system to predict pneumothorax in age older than 80s group in derivation cohort. E: Comparison of ROC curves of excluding each protective factor (number of specimens, needle path bleeding, and lobular of the lesion) from the scoring system in derivation cohort. Green line includes all variables. F: ROC curve of using the risk scoring system to predict pneumothorax in validation cohort.

External Validation

In the validation cohort, the risk scoring system obtained an AU- ROC of 0.736 in the evaluation of predicting pneumothorax (Fig 3F). The cutoff score suggested by Youden’s index was 11 points in the validation cohort.

Discussion

This is a retrospective single-center study attempting to identify risk factors and then set up a risk score system for CT-guided PT- NB-associated pneumothorax complications in a tertiary hospital. In our study, the incidence of pneumothorax and severe pneumo- thorax were 16.9 and 1.2% in the derivation cohorts, respectively. These pneumothorax complication rates in our study were in line with the results of previous studies, implying that the results of our study can be extrapolated to other studies. Clustering allows us to better understand how a sample might be comprised of distinct subgroups given a set of variables. After calculating the silhouette width for clusters for the PAM algorithm, we observed 2 clusters of pneumothorax complications related to PTNB. To the best of our knowledge, this is the first study to use this method to cluster and visualize these clusters. This result provides us with a new angle to better understand the complications of pneumothorax.

The identified variables further served as the foundation for the de- velopment of a risk scoring system connecting CT-guided PTNB. This risk scoring system takes into account age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion, which are easily ob- tained. To the best of our knowledge, this is the first study in the development and validation of a risk score system for predicting pneumothorax. Among those factors, the number of specimens, needle path bleeding, and lobular lesion were protective factors for pneumothorax, which could not be excluded from the model for obtaining the highest AUROC. Three things interest us among those factors. First, the number of lesion puncture was a risk fac- tor, but the number of specimens was a protective factor. It is un- derstood that each puncture may not obtain one specimen because off-target puncture sometimes occurs, which results in damage to the normal lung and is not helpful for diagnosis. We hypothesize more numbers of lesion puncture mean more destruction, especial- ly puncture into normal lung. Second, more specimens in the lesion do not necessarily mean more destruction of the normal lung, and more specimens result in more tissue bleeding, which may stop the development of pneumothorax, which is why it is a protective factor for pneumothorax. Third, it was noticed that pneumothorax usually happened at two stages during the procedure. First stage was the moving forward of the outer sheath to reach the lesion, while second was the lesion puncture. We tried to use parameters including lesion location, puncture depth and number of CT scan- ning to mimic the first procedure stage, but these parameters were hard to describe the complexity such as the angle adjusting during the outer sheath moving forward.

We used seven variables to develop this scoring system to predict pneumothorax in 1001 patients and validated it in another 230 pa- tients. All the variables were readily available; among these, two were general information (age, smoking index), four were proce- dure-related variables (number of lesion puncture, puncture depth, number of specimens, needle path bleeding), and one was a ra- diologic feature (lobularity of the lesion). We found that the diag- nostic accuracy of pneumothorax was acceptable in both the test and validation cohorts, while in the older age group, the diagnostic power was better. We therefore conclude that this scoring system is a good tool to help clinical practitioners predict PTNB-related pneumothorax. We developed the scoring system using this pre- sentation format because of its straightforward use in clinical prac- tice. During the CT-guided PTNB procedure, patients were scored 11 or higher, and we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to pre- vent subsequent pneumothorax. Our study has a larger sample size than the calculated sample size that was needed but had a relative- ly low incidence in the outcome. Thus, the possibility of type II errors in this study may not be low, and the results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggest- ed an appropriate fit of the scoring system.

There were some limitations that need to be mentioned. First, due to its retrospective design, information bias might have occurred during the outcome ascertainment. Second, the sample size was small in the higher age group, such as the group of patients aged older than 70 or 80 years, and the efficacy of the scoring system in these patients may be skewed. Third, other centre may use small- er gauge needle and this may make a difference to pneumotho- rax risk. Fourth, external validation by a future prospective study would be warranted, as it would make this scoring system more relevant for clinical scenarios.

Discussion

This is a retrospective single-center study attempting to identify risk factors and then set up a risk score system for CT-guided PT- NB-associated pneumothorax complications in a tertiary hospital. In our study, the incidence of pneumothorax and severe pneumo- thorax were 16.9 and 1.2% in the derivation cohorts, respectively. These pneumothorax complication rates in our study were in line with the results of previous studies, implying that the results of our study can be extrapolated to other studies. Clustering allows us to better understand how a sample might be comprised of distinct subgroups given a set of variables. After calculating the silhouette width for clusters for the PAM algorithm, we observed 2 clusters of pneumothorax complications related to PTNB. To the best of our knowledge, this is the first study to use this method to cluster and visualize these clusters. This result provides us with a new angle to better understand the complications of pneumothorax.

The identified variables further served as the foundation for the de- velopment of a risk scoring system connecting CT-guided PTNB. This risk scoring system takes into account age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion, which are easily ob- tained. To the best of our knowledge, this is the first study in the development and validation of a risk score system for predicting pneumothorax. Among those factors, the number of specimens, needle path bleeding, and lobular lesion were protective factors for pneumothorax, which could not be excluded from the model for obtaining the highest AUROC. Three things interest us among those factors. First, the number of lesion puncture was a risk fac- tor, but the number of specimens was a protective factor. It is un- derstood that each puncture may not obtain one specimen because off-target puncture sometimes occurs, which results in damage to the normal lung and is not helpful for diagnosis. We hypothesize more numbers of lesion puncture mean more destruction, especial- ly puncture into normal lung. Second, more specimens in the lesion do not necessarily mean more destruction of the normal lung, and more specimens result in more tissue bleeding, which may stop the development of pneumothorax, which is why it is a protective factor for pneumothorax. Third, it was noticed that pneumothorax usually happened at two stages during the procedure. First stage was the moving forward of the outer sheath to reach the lesion, while second was the lesion puncture. We tried to use parameters including lesion location, puncture depth and number of CT scan- ning to mimic the first procedure stage, but these parameters were hard to describe the complexity such as the angle adjusting during the outer sheath moving forward.

We used seven variables to develop this scoring system to predict pneumothorax in 1001 patients and validated it in another 230 pa- tients. All the variables were readily available; among these, two were general information (age, smoking index), four were proce- dure-related variables (number of lesion puncture, puncture depth, number of specimens, needle path bleeding), and one was a ra- diologic feature (lobularity of the lesion). We found that the diag- nostic accuracy of pneumothorax was acceptable in both the test and validation cohorts, while in the older age group, the diagnostic power was better. We therefore conclude that this scoring system is a good tool to help clinical practitioners predict PTNB-related pneumothorax. We developed the scoring system using this pre- sentation format because of its straightforward use in clinical prac- tice. During the CT-guided PTNB procedure, patients were scored 11 or higher, and we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to pre- vent subsequent pneumothorax. Our study has a larger sample size than the calculated sample size that was needed but had a relative- ly low incidence in the outcome. Thus, the possibility of type II errors in this study may not be low, and the results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggest- ed an appropriate fit of the scoring system.

There were some limitations that need to be mentioned. First, due to its retrospective design, information bias might have occurred during the outcome ascertainment. Second, the sample size was small in the higher age group, such as the group of patients aged older than 70 or 80 years, and the efficacy of the scoring system in these patients may be skewed. Third, other centre may use small- er gauge needle and this may make a difference to pneumotho- rax risk. Fourth, external validation by a future prospective study would be warranted, as it would make this scoring system more relevant for clinical scenarios.

Conclusion

We established and validated a risk scoring system to predict pneumothorax in patients who underwent CT-guided PTNB. For patients who have high scores during the PTNB procedure, we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to lower the risk of pneumo- thorax.

References