Semantic description of medical image findings: structured learning approach

Pavel Kisilev, Eugene Walach, Sharbell Hashoul, Ella Barkan, Boaz Ophir and Sharon Alpert

Abstract

Computer Aided Diagnosis (CADx) systems are designed to assist doctors in medical image interpretation. However, a CADx is often thought of as a 'black box' whose diagnostic decision is not intelligible to a radiologist. Therefore, a system that uses semantic image interpretation, and mimics human image analysis, has clear benefits. We propose a new method for automatic textual description of medical image findings, such as lesions in medical images. The method performs joint estimation of semantic features of lesions from image measurements. We formalize this problem as learning to map a set of diverse medical image measurements to a set of semantic descriptor values. We use a structural learning framework to model individual semantic descriptors and their relationships. The parameters of the model are efficiently learned using the Structured Support Vector Machine (SSVM). The proposed approach generates radiological lexicon descriptors used to make a diagnosis of various diseases. This can help radiologists easily understand a diagnosis recommendation made by an automatic system, such as CADx. We apply the proposed method to publicly available and to proprietary breast and brain imaging datasets, and show that our method generates more accurate descriptions, as compared to other alternative approaches.

Session

Poster 2

Files

PDF iconExtended Abstract (PDF, 286K)
PDF iconPaper (PDF, 630K)

DOI

10.5244/C.29.171
https://dx.doi.org/10.5244/C.29.171

Citation

Pavel Kisilev, Eugene Walach, Sharbell Hashoul, Ella Barkan, Boaz Ophir and Sharon Alpert. Semantic description of medical image findings: structured learning approach. In Xianghua Xie, Mark W. Jones, and Gary K. L. Tam, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 171.1-171.11. BMVA Press, September 2015.

Bibtex

@inproceedings{BMVC2015_171,
	title={Semantic description of medical image findings: structured learning approach},
	author={Pavel Kisilev and Eugene Walach and Sharbell Hashoul and Ella Barkan and Boaz Ophir and Sharon Alpert},
	year={2015},
	month={September},
	pages={171.1-171.11},
	articleno={171},
	numpages={11},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.171},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.171}
}