Face Painting: querying art with photos

Elliot J. Crowley, Omkar M. Parkhi and Andrew Zisserman

Abstract

We study the problem of matching photos of a person to paintings of that person, in order to retrieve similar paintings given a query photo. This is challenging as paintings span many media (oil, ink, watercolor) and can vary tremendously in style (caricature, pop art, minimalist). We make the following contributions: (i) we show that, depending on the face representation used, performance can be improved substantially by learning -- either by a linear projection matrix common across identities, or by a per-identity classifier. We compare Fisher Vector and Convolutional Neural Network representations for this task; (ii) we introduce new datasets for learning and evaluating this problem; (iii) we also consider the reverse problem of retrieving photos from a large corpus given a painting; and finally, (iv) using the learnt descriptors, we show that, given a photo of a person, we are able to find their doppelgänger in a large dataset of oil paintings, and how this result can be varied by modifying attributes (e.g. frowning, old looking).

Session

Poster 1

Files

Extended Abstract (PDF, 1254K)
Paper (PDF, 7M)

DOI

10.5244/C.29.65
https://dx.doi.org/10.5244/C.29.65

Citation

Bibtex

@inproceedings{BMVC2015_65,
	title={Face Painting: querying art with photos},
	author={Elliot J. Crowley and Omkar M. Parkhi and Andrew Zisserman},
	year={2015},
	month={September},
	pages={65.1-65.13},
	articleno={65},
	numpages={13},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.65},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.65}
}