Weakly Supervised Metric Learning towards Signer Adaptation for Sign Language Recognition

Fang Yin, Xiujuan Chai, Yu Zhou and Xilin Chen

Abstract

Inter-signer variation is one of the challenging factors in real application of Sign Language Recognition(SLR). To tackle this problem, this paper proposes a weakly supervised metric learning framework to realize signer adaptation with some unlabeled sign data of the new signer. Concretely speaking, through the labeled data of several different signers, a generic distance metric can be learnt. Then the key step is to adapt the generic metric to the new signer according to the collected unlabeled samples. Clustering constraint and manifold constraint are considered together to realize the weakly supervised metric learning. In our implementation, a novel fragment-based feature is designed to describe each sign by fusing both trajectory and hand shape features, which is also proved more discriminative than the frame-based multi-modal feature. Extensive experiments on our collected large vocabulary datasets convincingly show that the proposed method is effective for signer adaptation and outperforms the state-of-the-art methods on signer-independent SLR.

Session

Poster 1

Files

Extended Abstract (PDF, 224K)
Paper (PDF, 1294K)

DOI

10.5244/C.29.35
https://dx.doi.org/10.5244/C.29.35

Citation

Bibtex

@inproceedings{BMVC2015_35,
	title={Weakly Supervised Metric Learning towards Signer Adaptation for Sign Language Recognition},
	author={Fang Yin and Xiujuan Chai and Yu Zhou and Xilin Chen},
	year={2015},
	month={September},
	pages={35.1-35.12},
	articleno={35},
	numpages={12},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.35},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.35}
}