Logo MachVis

Boosting Face Biometrics under COVID-19

Boosting Face Biometrics under COVID-19

In the current COVID-19 scenario, social distancing and wearing facial masks for covering mouth and nose are the popular ways of avoiding the adverse effects of the pandemic. In these special circumstances, contactless biometric verification systems have gained importance. In biometric verification systems we have Facial recognition, Iris recognition and Gait based verification systems. Among them, Facial Recognition is the best of all in terms of accuracy, effectiveness and pricing. Facial recognition has found its applications in a variety of areas for example surveillance systems, attendance systems, transportation ticket verification systems, border management systems etc. But the problem with conventional facial recognition systems is that most of these systems are not designed to deal with masked or occluded faces. And the effectiveness of these systems falls tremendously with masked faces. According to the NIST2020 report, the accuracy of the facial recognition systems have 20-50% error due to masked faces. Therefore, in order for facial recognition systems to work effectively, people have to pull off the mask to get themselves verified but at the same time there is risk of getting the infection and in addition to this touching face and mask are highly discouraged as per SOPs of COVID-19. Also in the surveillance systems, each and every individual cannot be forced to remove the facial mask. Therefore, keeping these facts in view, there is dire need to improve the existing facial recognition techniques to cater the problems created by masked faces. Currently facial recognition with masked faces is an under researched area and many researches are going on in this domain.
In this research, we are looking for possible approaches which can improve the performance of masked facial recognition. Mainly we are considering machine learning and deep learning based approaches. But these approaches need data as a prerequisite. Currently there is no publicly available benchmark dataset of masked face recognition and researchers have either created their own datasets or used augmentation to mask the faces in existing facial recognition datasets. Therefore, we are using augmentation for masking the face images in already available face recognition datasets to simulate the masked face scenarios. Once we have generated the masked version of the dataset, we use pre pre-trained CNN model like Resnet101 to generate the facial embeddings. For masked images of a person, the embeddings are very dissimilar to that of embeddings of unmasked images of a person hence making facial recognition difficult. Therefore we pass masked facial embeddings from a dense unit trained with Self Restrained Triplet Loss. This dense unit makes the masked facial image embedding similar to unmasked face image embeddings. So in this way, we are improving the performance of existing facial recognition algorithms on masked faces while preserving their performance on unmasked faces.


  • Dr Muhammad Moazam Fraz


  • Muhammad Aasharib Nawshad