27  First Phase Results Text

Cases had a median of 13.5 blocks (min:8 - max:22) and a total of 829 biopsy cores in the Phase I analytical cohort (851 slides as uploaded, 22 excluded per the canonical exclude-list — 19 duplicate rescans, 2 accidentally uploaded IHC stain slides, and 1 slide where the Paige website did not run). After excluding cores that could not be re-scored in the second-reader phase, 810 cores were re-read by all four pathologists with and without AI assistance in Phase II; of these, 138 had a parseable Gleason score from every interpreter and constitute the inter-rater complete-cases subset used for the Fleiss kappa. 601 (72.5%) cores were labeled as benign by both AI and the pathology report. 199 (24.0%) cores were labeled as malignant by both AI and the pathology report. 1 core was labeled as benign by AI but malignant in the pathology report (Dx_Research = adenocarcinoma; AI miss). 28 cores were labeled as malignant by AI but benign in the pathology report. Reference diagnosis after expert review with IHC: 18 benign, 1 ASAP, and 9 adenocarcinoma. These adenocarcinoma foci were minute and low grade; the other cores of these cases had abundant tumor.

When evaluated on a case basis with final IHC confirmation, AI had over diagnosis in 4 cases.

Overall AI had 92.1% and 90% positive predictive value, 99.8% and 100% negative predictive value on core and case based analysis respectively.

Materials Methods:

c17 c45

tanı sırasında immün yapılan vakalar’a paige ne demiş?,

• 60 cases whose digital images are available were included.

• Slides were scanned by Leica Aperio AT2 scanner with 20x or 40x magnifications. (Numbers will be calculated)

• Cases were diagnosed with Sectra interface.

• Images were anonymised (with https://github.com/pearcetm/svs-deidentifier) and uploaded to paige website. Report diagnoses were retrieved by a pathologist and compared with AI analysis output.

• Cases had median 14 blocks (min:10 - max:25). max number to be corrected fue to duplicates in c17

• A total of 874 cores were reached. number to be corrected fue to duplicates in c17 and IHC stains.

Results:

Per Core

• 623 (r (round(623/874, digits = 2))*100%) cores were labelled as benign by both AI and diagnoses as benign pathologist.

• 215 (r (round(215/874, digits = 2))*100 %) cores were labelled as tumor by AI and diagnosed as tumor pathologist.

• 8 cases were diagnosed as benign by AI, but malignant by pathologist:

Among these 8 cases 3 images were blurred and 4 had processing artefacts, thus excluded from further agreement analysis.

1 of the blurred images had a focused slide as well, and it was labelled as tumor (c17)

1 case (c59_s7.svs) was diagnosed as benign by AI, but malignant as pathologist with IHC confirmation.

• 28 cases were labelled as tumor by AI, but diagnosed as benign by pathologist.

Among these 28 cases:

• 4 was suspected by pathologist, and diagnosed as benign after IHC confirmation.

• 1 was suspected by pathologist, and diagnosed as ASAP after IHC confirmation.

• 1 was suspected by pathologist, and diagnosed as ASAP.

• Remaining 22 cores added IHC results:

4 was reviewed again by expert pathologist (FA). They were rediagnosed as tumor.

c5 other cores contained tumor as well. c5_s1 probably missed by pathologist. Blocks are not available.

4 was reviewed again by expert pathologist (FA). They were suspicious for tumor.

2 was reviewed again by expert pathologist (FA). They were suspicious for ASAP.

12 was reviewed again by expert pathologist (FA). They were probably overdiagnosed by AI.

Per Case

• 60 cases were included.

20 cases were labelled as benign by AI, and diagnosed as benign in report.

40 cases were labelled as tumor by AI:

34 of them were were labelled by tumor by AI, and diagnosed as tumor.

6 cases are as follows:

• c19 other cores benign by both AI and report. c19_s12 labelled tumor by AI.

• c20 all cores benign by report.

c20_s7.svs labelled tumor by AI. c20_s3.svs labelled tumor by AI. c20_s5.svs not prostate tissue excluded from study.

• c26 other cores benign by both AI and report.

c26_s15 labelled tumor by AI.

• c20 all cores benign by report.

c29_s1.svs labelled tumor by AI. c29_s5.svs labelled tumor by AI.

• c30 other cores benign by both AI and report.

c30_s1 labelled tumor by AI.

• c54 other cores benign by both AI and report.

c54_s14 report dx as ASAP, confirmed by IHC. This core is labelled as tumor by AI.

Comments by SB:

• Currently there is 96.7% agreement.
• Before using AI, we should have checked the quality of image (probably with https://github.com/histopathology/HistoQC)
• AI can fail in tissues with processing artifact.
• In those 4 cores where expert pathologist rediagnosed as tumor, other cores of same case also contained tumor. Maybe the pathologist missed them because other cores also contained tumor. This is a debatable point.
• I have accidentally uploaded some IHC images as well. I have excluded them from slide numbers.