Evaluation of Pathologist Agreement Using a Probability-Based Predictive Tool

The medical sciences of both humans and animals have long recognized the existence of inter-pathologist variation. The common consensus is that a Fleiss' kappa statistic of more than 40% indicates a fair to good level of agreement, and more than 75% indicates an exceptional level of agreement. In areas like cardiac exercise testing and estimating the risk of heart disease or lung cancer given a variety of test results and patient characteristics, predictive models have been employed widely in human medicine. The pilot tissue for this investigation, which is a component of a PhD research project, was urinary bladder tissue. Neoplasia, urolithiasis, and cystitis are the three broad classifications of bladder conditions that affect dogs and cats.

The goal of this study was to create a probability-based tool using data from 338 histopathology slides of canine and feline urinary bladders, and to determine whether the tool had an impact on the test pathologists' agreement.

The kappa sample size calculation was used to determine the sample size for this project. Four pathologists examining the slides with statistical power required at least 23 instances to be examined, according to power calculations. The bladder tissue samples used in this investigation were chosen from a case material pool (EJ) of bladder disease cases and processed as per standard procedure for H&E microscopy on glass slides. The American College of Veterinary Pathologists (ACVP) specialist certification and a minimum of five years of experience working in a diagnostic setting that required the review of tissues from dogs and cats and from various geographic locations were the criteria used to select four veterinary pathologists.

Technical difficulties prohibited pathologist number 4 (P4) from evaluating all the digital slides. The pathologist's missing diagnoses were imputed using the MICE R program. The MICE technique was used to create the polytomous logistic regression imputation model for unordered categorical data because the outcome in this case is a multinomial variable. A veterinary pathologist's primary area of focus is histopathology, and proficiency in histopathological interpretation is one of the specialty's distinctive skill sets. However, numerous studies have shown that pathologists are susceptible to a variety of processing issues and personal cognitive biases that may affect their ability to diagnose. The results show that more investigation is needed in this field, particularly to examine additional histological settings and diagnoses.

For the diagnosis of bladder neoplasia, four veterinary pathologists evaluated bladder sections from dogs and cats. However, there was poor to fair agreement for the diagnosis of cystitis, urolithiasis, and normal bladder tissue. When signalment and clinical history were provided, agreement amongst pathologists increased, with mixed findings when a predictive probability tool was introduced.

Source: Veterinary Sciences