# ADVANCED ANPR & FRS SOLUTION **Team:** Coding Brigades **Hackathon:** KAVACH — National Cybersecurity Hackathon **Project Focus:** ANPR · OCR · Face Detection · Face Recognition · Real-Time CCTV Analytics ## Project Brief **ADVANCED ANPR & FRS SOLUTION** is an intelligent traffic-enforcement platform designed to perform **Automatic Number Plate Recognition (ANPR)** and **Facial Recognition (FRS)** from CCTV feeds, with a specific focus on recognizing **non-standard number plates** that vary in font, size, design, symbols, and formatting. The project was developed by **Team Coding Brigades** for the KAVACH hackathon. Our team consisted of six members, and we submitted two problem definitions, with **both solutions being shortlisted**. This project was selected for the final stage, where we travelled to **Mumbai/Panvel** for the final round. The core objective was to strengthen the **One Nation, One Challan** concept by connecting vehicle identification with driver identification rather than relying only on the registered vehicle number. **My Role:** Machine Learning & Backend Developer — owning the model integration, Flask backend, database layer, end-to-end inference pipeline, performance optimization, mobile integration, and Git workflow. ## Problem Statement Traditional ANPR systems work effectively with clearly formatted number plates, but real-world Indian roads contain a large variety of **non-standard plates** with different fonts, layouts, sizes, symbols, languages, and visual conditions. This creates two fundamental challenges: **Vehicle Identification** Detect and recognize a vehicle's number plate even when the plate does not follow standard formatting. **Driver Identification** A number plate identifies the vehicle, but not necessarily the person driving it. A vehicle may be shared, borrowed, stolen, or associated with an incorrect identity. The system therefore needed to establish a complete chain: **CCTV → Vehicle → Number Plate → Plate Text → Driver Face → Identity** The challenge was to make this pipeline work reliably across **different lighting conditions, camera angles, image quality, plate formats, and real-world CCTV footage**. ## Solution Architecture The solution was designed as a multi-stage computer-vision pipeline where each model performs a specialized task.  *System use case diagram — actors, operations, and the boundary between the field application and the inference backend.* ### Image-Based Flow **Mobile / CCTV Image** ↓ **Vehicle & Number Plate Detection** ↓ **Plate Classification** ↓ **OCR Processing** ↓ **Standard / Non-Standard Identification** In parallel: **Vehicle Image** ↓ **Driver Face Detection** ↓ **Face Recognition** ↓ **Identity Matching** Both pipelines converge at the backend, where the results are processed and delivered to the administrative application. ### Core Architecture * **YOLO-based models** for vehicle, driver, and number-plate detection. * **YOLOv8 and YOLO-NAS** evaluated and used for detection tasks. * A dedicated **OCR pipeline** operating on the detected plate coordinates. * Separate **face-detection and face-recognition stages**. * **Flask APIs** acting as the central inference and application layer. * **MongoDB** for fast and flexible data storage. * **Flutter mobile application** for government/admin users. * Real-time processing support for **CCTV feeds**.  *System architecture — detection, OCR, recognition, and backend orchestration as independently replaceable stages.* The most important architectural decision was to keep **detection, OCR, recognition, and backend orchestration as separate stages**, allowing each component to be independently improved without redesigning the complete system. ## Technical Execution I owned the central integration of the system, bringing the individual ML components together into a single operational pipeline. ### Dataset & Model Development For face detection and recognition, we created our own dataset by collecting face images from controlled environments, including classroom settings. The dataset contained approximately **5,000–6,000 face images**, which were used for training and evaluating the recognition pipeline. For number-plate detection, we initially created a dataset by capturing **2,000+ vehicle images** from college parking areas under different lighting conditions, camera angles, distances, backgrounds, and image contrasts. The images were manually annotated for number-plate detection. To improve real-world accuracy, we later collaborated with the **Ahmedabad Municipal Corporation** to obtain CCTV footage from different traffic locations. We developed a preprocessing script to extract frames from the footage at regular intervals, creating an additional dataset of **14,000+ real-world number-plate images** with significantly greater environmental diversity. ### Detection & OCR Pipeline * Trained multiple YOLO-based detection models. * Used **YOLOv8 and YOLO-NAS** for detection experiments. * Built a dedicated number-plate detection model. * Developed the OCR processing stage after plate detection. * Passed the detected plate coordinates directly into the OCR pipeline. * Extracted plate text from the detected region. * Classified the detected plate as **standard or non-standard**. * Improved the model using real-world CCTV data, achieving approximately **98% detection accuracy** on the final evaluation. ### Backend & Integration * Designed and developed the central backend using **Python Flask**. * Integrated the face-detection, face-recognition, number-plate detection, and OCR models. * Designed the MongoDB data layer. * Built APIs for mobile and administrative applications. * Managed the complete **image → inference → result → response** workflow. * Optimized the execution sequence to minimize processing overhead. * Reduced the overall image inference and response pipeline to **under 300 ms** in the developed system. ### Mobile Application I also developed the **Flutter-based mobile application** for government/admin users. The workflow was intentionally simple: **Capture / Upload Image → Process → Detect Vehicle → Detect Plate → Extract Text → Detect Face → Recognize Driver → Display Results** The application acts as the operational interface for the backend computer-vision pipeline. ### Real-Time CCTV The same pipeline was integrated with **live CCTV feeds**, allowing the system to process frames continuously and detect number plates and driver faces during runtime. I also managed the **Git branching, integration, and overall development flow** across the different components of the project. ## Results & Impact The project transformed separate computer-vision models into a **single integrated traffic-analysis platform** capable of connecting vehicle information with driver information. ### Key Outcomes * Developed a dedicated solution for **standard and non-standard number-plate recognition**. * Built a real-world dataset of **14,000+ CCTV-derived plate images**. * Created an additional **2,000+ manually annotated vehicle/plate dataset** during initial development. * Developed a face dataset containing approximately **5,000–6,000 images**. * Achieved approximately **98% number-plate detection accuracy** after training with diverse real-world data. * Integrated **ANPR + OCR + Face Detection + Face Recognition** into one pipeline. * Achieved **<300 ms inference/response time** for the developed image-processing workflow. * Built a Flutter application for government/admin users. * Integrated the pipeline with live CCTV processing. * Successfully connected the complete flow from **image acquisition to actionable identification data**. The project was submitted to **KAVACH**, where our team submitted two problem definitions and **both were shortlisted**. This solution progressed to the final round in **Mumbai/Panvel**.  *KAVACH national shortlist — problem statement one.*  *KAVACH national shortlist — problem statement two. Both submissions from Team Coding Brigades were selected.* Although we did not win the final round, the experience exposed us to real-world engineering constraints, live requirements, rapid feature changes, time pressure, team coordination, and technical decision-making under demanding conditions. ## Key Learnings This project significantly strengthened my understanding of building **end-to-end computer-vision systems rather than isolated ML models**. The most important lessons were: * Real-world ML performance depends heavily on **dataset diversity and quality**. * CCTV systems require models to handle **lighting, angle, resolution, occlusion, and environmental variation**. * Separating **detection, OCR, recognition, and orchestration** makes complex systems easier to improve and maintain. * Model accuracy alone is not enough; **inference latency and system integration** are equally important. * A successful ML product requires strong coordination between **models, APIs, databases, mobile applications, and deployment workflows**. * Real-time hackathon development taught me how to handle changing requirements without losing focus on the core architecture. * Working as the central integration owner improved my ability to distribute work, manage Git branches, resolve integration issues, and maintain the complete technical flow. * The experience reinforced the value of designing systems with **flexibility for future model improvements and new requirements**. Most importantly, the project taught me how to move from **raw data → trained models → optimized inference → backend APIs → mobile application → real-world decision support**, which became one of the strongest practical computer-vision experiences in my early engineering journey.