In the rapidly evolving era of video content, the need for automated methods to extract and analyze information is more pressing than ever. Face-based person retrieval stands out as a particularly intriguing task, involving the use of face recognition to track and identify individuals in video data. Traditional face datasets, often composed of web images of celebrities, fall short of addressing the complexities inherent in video face recognition. This has spurred research into developing large-scale datasets of facial images extracted from videos to promote advancements in face representation and identification models. This research proposes a novel approach, integrating large-scale unsupervised face recognition in videos with a focus on developing a comprehensive dataset. This dataset is designed to foster the growth of models capable of unsupervised face recognition, addressing the unique challenges posed by video data. Introduction of a hierarchical retrieval index enables online clustering, demonstrating the dataset’s effectiveness in real-time person retrieval systems. Furthermore, this dataset is tailored for evaluating feature representation and identity classification models in face verification, identification, and clustering.

Building on this foundation, the Clustering Large Online Unrecognized Detections (CLOUD) technique offers a solution to the limitations of supervised face recognition methods, which are constrained by their reliance on predefined training data. CLOUD is an unsupervised and dynamic clustering technique that operates without prior assumptions about the number of classes or the distribution of data. This flexibility makes it particularly suited for real-time or live video analysis, where the appearance of new, previously unseen entities is common. CLOUD introduces the concept of Dynamic Clustering (DC), which leverages Dynamic Database Population (DDP) to maintain a dictionary of reference faces. This approach is especially effective in live streams, like those from TV channels, where new faces continually appear. In a practical application, CLOUD is tested on live video from Pakistani news channels, recognizing 1,000 entities over 11 hours of footage and achieving a Cluster Purity (CP) of 90%. This performance is comparable to other unsupervised techniques, showcasing CLOUD’s potential in real-time unsupervised face recognition and its broader applicability to various detection problems.With a combination of these two innovative approaches, this research presents a comprehensive framework for large-scale unsupervised face recognition in videos, with the CLOUD technique offering a dynamic, unsupervised solution to the challenges of face detection and recognition in live video streams.