Computer-aided Detection Fidelity of Pulmonary Nodules in Chest Radiograph

Introduction

The most ubiquitous chest diagnostic method is the chest radiograph. A common radiographic finding, often incidental, is the nodular pulmonary lesion. This term comprises nodular opacities ranging from 1 to 30 mm with a broad distribution of morphology, allocation, number, and clustering patterns. The long list of nodular lesion differential diagnosis ranges from benign granulomas to sessile or metastatic lung cancer.[1]

Detection of pulmonary nodules in routine chest radiographs is a daily clinical challenge. Not only the detection of small lesions within a complex parenchymal structure but also the ability to discriminate between true lesions and overlapping structures, surroundings or background noise quite often overwhelms the diagnostic efficacy of the observer. As a result many nodular lesions remain undiagnosed and many unspecific nodular projections are perceived as pulmonary lesions in the daily practice. The detection rate of pulmonary nodules has been estimated 81.4% by experienced radiologists and 81.2% by radiology assistants.[2] The concept of computer-aided detection (CAD) implements computational methods to improve the diagnostic efficacy. CAD software products use image analysis algorithms to reduce the background noise, increase the signal-to-noise ratio and implement image segmentation and pattern recognition techniques that automatically detect nodular lesions in limited time.

In this study, we have retrospectively tested SoftView™ 2.4A and OnGuard™ 5.2 CAD systems (Riverain Technologies, Miamisburg, OH, USA) in a collective of 100 patients with histologically proven nodular pulmonary lesions and revised the relevant literature.

Subjects and Methods

Results

The tested CAD software aims to improve image contrast using a bone suppression algorithm and automatically detect pulmonary coin lesions in conventional PA radiographs. In a sample radiograph with multiple coin lesions before [Figure 1a] and after BSI [Figure 1b], the CAD software has detected two coin lesions [Figure 1c]. CT has confirmed the validity of detection [Figure 1d and e]. Note that lesions are easier to spot in the BSI. Lack of bone artifacts helps focusing the attention span in the lung parenchymal pattern, consequently reducing the probability of a lesion undergoing radiologist's attention.

Close

Figure 1

Sample images for bone suppression imaging and computer-aided detection lesion identification (a) unprocessed posteroanterior chest radiograph (CTL), (b) posteroanterior chest radiograph after bone suppression imaging, (c) posteroanterior chest radiograph with lesions as detected by computer-aided detection (white arrows and circles). (d and e) Spiral computed tomography scans in axial plane, lung-window reconstruction, confirmation of the detected lesions in a-c (white arrows).

Export to PPT

To quantify the contrast-to-background improvement on radiograph images by bone subtraction, we compared image pairs before (CTL) and after processing (BSI). Gathering all lesion sizes [Figure 2a] returns no statistical significance (CTL = 1.34 ± 0.30, BSI = 1.34 ± 0.31, P = 0.735, n = 100, Wilcoxon Rank-sum test). Due to the three-dimensional character of pulmonary lesions, the X-ray attenuation and consequently the lesion's contrast are linearly dependent on the size [Figure 2b] in both CTL and BSI with minimal differences (P = 0.45 × 10⁻³, R = 0.35 for CTL and P = 0.017 × 10⁻³, R = 0.42 for BSI, n = 100, Pearson's test). This suggests that the bone suppression algorithm does not drastically modify the basic radiograph properties and attenuation proportions. Size-based lesion grouping [Figure 2c and Table 1] showed no significant differences in lesion contrast for groups S1-S5 [Table 1].

Close

Figure 2

(a) Contrast after bone suppression imaging; P = 0.735, n = 100, Wilcoxon Rank-sum test. (b) Contrast is positively correlated with size in control (CTL) and bone-suppression imaging; (P, R) = (0.45 × 10⁻³, 0.35) for CTL and (0.017 × 10⁻³, 0.42) for bone-suppression imaging, n = 100, Pearson's test. (c) No significance for all nodule sizes, detailed statistics shown in Table 1. (d) Contrast in association to surrounding structures, P = 0.115 for CTL and P = 0.099 for bone-suppression imaging, one-way ANOVA. Contrast is unaffected by bone-suppression imaging: bone n = 95, P = 0.676, Wilcoxon; soft tissue n = 10, P = 0.508, paired t-test; no neighbor n = 5, P = 0.631, paired t-test.

Export to PPT

Table 1 Bone suppression imaging effect on the nodular pulmonary lesion's contrast-to-background

This result indicates that BSI does not achieve any relevant improvement of the lesions’ contrast-to-background opacity.

An important factor obscuring nodular lesion detection is the vicinity to nonparenchymal structure projections. We questioned the effect of various structure vicinities on the measured lesion's contrast with and without BSI [Figure 2d]. Three subgroups were created according to the nature of the neighboring structure: (a) bone (95%), (b) soft tissue (10%; this subgroup includes mediastinum, pleura, and diaphragm), and (c) no neighbor (5%). From n = 100 lesions, many displayed vicinity to more than one structure and have been counted accordingly. The lesion's contrast appears to be irrelevant to the neighboring structure (P = 0.115 for CTL and P = 0.099 for BSI, one-way ANOVA) and unaffected by BSI: bone n = 95, P = 0.676, Wilcoxon; soft tissue n = 10, P = 0.508, paired t-test; no neighbor n = 5, P = 0.631, paired t-test.

Summarizing the above, bone suppression algorithm seems to subjectively improve radiograph images by artifact removal, however without proving any significant objective contrast improvement, regardless of the lesion size and vicinity associations.

CAD analysis automatically marked 210 lesions in 100 patients (total positive lesions; to 210). All detected lesions were optically assessed for validity; 88 of them were FP (88) and 197 were true positive (TP = 197). The character of the FP lesions comprised mostly hilar structures (n = 22 lesions, 25%), followed by bones (n = 20 lesions, 23%) and the rest was summarized as unspecific [Figure 3]. CAD analysis showed 62% sensitivity and 58% specificity. The FP rate (FPR) was 0.88/image and the false negative rate was (FNR) 0.35/image.

Close

Figure 3

Identity of false positive lesions.

Export to PPT

Optically detected lesions (n = 100) were histologically proved as 75% malignant (n = 75 patients) and 25% benign (n = 25 patients). The majority of malignant lesions were pulmonary metastasis (n = 48), followed by bronchial cancer. About 13 lesions were histologically characterized as benign and 10 as granulomas [Figure 4].

Close

Figure 4

Identity of optically detected lesions.

Export to PPT

From the histologically proven lesions (n = 100, the largest lesion per patient from n = 100 patients), TP lesions as detected by the CAD software were 20% benign and 80% malignant. FN lesions were 47% benign and 53% malignant, respectively. The Chi-square test returned a significantly positive correlation between the detection probability and malignancy (P = 0.012, n = 100) [Table 2]. Thus, malignant lesions seem to have a significantly higher probability to be detected by the CAD software compared to benign ones. Since the detection algorithm is contrast-sensitive, this result indicates that malignant lesions might possess different contrast values compared to benign ones. However, paired analysis [Figure 5] demonstrates no contrast differences between benign and malignant lesions with or without BSI (CTL: P= 0.39, t-test; BSI: P =0.781, Mann–Whitney U-test).

Table 2 Computer-aided detection in association to lesion dignity

Close

Figure 5

Contrast of benign and malignant lesions before (CTL) and after bone suppression (bone-suppression imaging). No difference was observed between benign and malignant lesions in either CTL or bone-suppression imaging images; n = 25 benign and n = 75 malignant nodules; CTL: P =0.39, t-test; bone-suppression imaging P = 0.781, Mann–Whitney U-test.

Export to PPT

Conclusively, CAD has shown 62% sensitivity and 58% specificity for the detection of pulmonary nodular lesions in chest radiographs. Moreover, the software significantly favored the detection of malignant lesions as opposed to benign ones, without this results being explainable with dignity-dependent contrast differences.

Discussion

Independently published clinical studies, summarized in Table 3 showed that the diagnostic chest radiograph reading of pulmonary malignant lesions was significantly improved with software assistance.[1011] In the aforementioned studies, experienced radiologists independently evaluated randomized and blinded chest radiographs with and without BSI, concluding that BSI improves radiologist's performance by 4% and 17%, respectively. Similar results verifying the diagnostic value of bone suppression software have been demonstrated for other than malignant lung lesions such as pulmonary aspergillosis and focal pneumonia.[121314] In those studies, detection sensitivity was significantly improved by 17% and 4%, respectively. Another study investigated the value of BSI in the detection of nodular bronchial carcinoma lesions with average diameter 20 mm in standard versus dual-energy chest radiographs.[14] Consistent to previous results, BSI improved the detection rate of nodular lesions in standard radiographs by 10%. A further, albeit small improvement of the detection rate by 18% could be achieved by the implementation of dual energy subtraction. The authors, however, critically concluded that the additional equipment requirements and radiation exposure of the dual energy imaging do not sufficiently compensate for the small improvement in detection rate.[14]

Table 3 Literature review of computer-aided detection of pulmonary nodules in chest radiographs

All in all, previous studies agree that BSI improves the diagnostic detection rates by experienced radiologists. However, none of them report on the objective image improvement, i.e., on the lesion contrast-to-background enhancement, which could have explained the improved detection rates after BSI. Our study uses a large patient sample to solidly demonstrate that BSI does not affect the objective lesion-to-background contrast. Thus, evidence-based assessment of our results suggests that BSI improves diagnosis independent of lesion-to-background contrast, most likely due to a subjective attention span improvement resulting from pattern simplification after ablation of bone artifacts.

Several CAD software applications have become commercially available in the last years, aiming to accelerate, standardize, and maybe improve the detection rate of pulmonary lesions in chest radiographs compared to observational diagnosis. Others and we have investigated the reliability of CAD and whether this can really (totally or partially) substitute observational diagnosis by experienced radiology specialists.

In accordance with previous results, our CAD analysis showed a sensitivity of 62% and a specificity of 58%. The FPR was 0.88/image and the FNR was 0.35/image. In previous studies, CAD stand-alone sensitivity was 59% with FPR 1.9 FP/image.[5] The study of Schalekamp et al., not only showed a higher sensitivity of 74% but also in turn a higher FPR of 1/image.[2] In another study comparing different versions of CAD systems, the sensitivity of the newest tested version OnGuard 5 was 64.4% and the FPR was 2/image.[8] Moreover, the stand-alone performance of a CAD system on preprocessed chest radiographies with two different bone suppression algorithms revealed a sensitivity of <80% and a specificity of <50%, both independent of the bone suppression algorithm applied. Therefore, the reliability of CAD software as a stand-alone standard of clinical diagnosis was severely doubted by the authors.[9]

Abandoning the stand-alone CAD usage, more studies have focused on defining the value of CAD software packages as a diagnostic adjuvant. The questioned parameters of CAD diagnosis are in this case not only the sensitivity and the specificity in performance but also the associated time costs in the clinical routine. A retrospective study using the OnGuard™ CAD software package (Riverain) has been designed to question the potential advantage of a CAD-assisted observational diagnosis of lung nodules compared to nonassisted observation.[6] For this purpose, 297 chest radiographs with 81 FN diagnoses underwent independent and blinded reevaluation without and with CAD assistance. Interestingly, OnGuard increased the detection sensitivity from 44% to 50%, thus supporting the value of a CAD-assisted rather than that of a CAD-based diagnosis.[6] However, authors do not report on the associated time costs. The same group has extended the question and further investigated whether CAD-assisted lung nodule detection could be indeed superior to the observational diagnosis on BSI.[2] CAD has substantially improved radiologists’ performance for the detection of lung nodules on chest radiographs from 73.1% to 79.3%, even when baseline performance was optimized by providing lateral radiographs and BSIs.[2] In line with this, De Boo et al., showed that CAD improves the sensitivity of inexperienced readers (39% vs. 45%) for the detection of small nodules at the expense of loss of sensitivity.[4] Another study suggested that the CAD system may help improve the observer's performance in detecting malignant lung nodules on chest radiographs. The sensitivities with and without CAD were 87% and 84%, whereas the FP/case with and without CAD was 0.19 and 0.17, respectively.[7] Similar results have been published by an independent group, where the CAD-assisted diagnosis has not only improved the sensitivity from 63.8% to 92.7% at the cost of a small decrease in specificity from 98.1% to 96.2% but also injected a non negligible increase in FPR to 2.7 FP/case.[3]

Conclusion

Summarizing the above, others and our study suggest that CAD sensitivity <80%, specificity <50%, and FPR >1/image do not qualify for a stand-alone standard of diagnosis. The “hybrid” approach of CAD-assisted observational diagnosis, i.e., CAD implementation accompanied with a critical radiological assessment has been proven advantageous for the detection of lung nodules so far, nevertheless with additional time costs. It is essential to focus future study design on the cost-effect assessment of the time costs by real-time (as opposed to retrospective) CAD implementation in clinical diagnostics.