Unlock the Future: Build a Robust Automatic Number Plate Recognition (ANPR) System in 7 Steps

Unlock the Future: Build a Robust Automatic Number Plate Recognition (ANPR) System in 7 Steps

Have you ever wondered how smart cities manage traffic, how parking lots operate seamlessly, or how law enforcement quickly identifies vehicles? The answer often lies in a powerful computer vision technology: Automatic Number Plate Recognition (ANPR). This groundbreaking system is transforming various industries by enabling automated vehicle identification and data collection from license plates.

In this comprehensive tutorial, we’ll embark on a journey to build a complete Automatic Number Plate Recognition (ANPR) system from the ground up. We’ll cover everything from detecting vehicles to recognizing number plates, correcting errors, and even classifying their origin districts. Whether you’re a budding data scientist, a computer vision enthusiast, or simply curious about this innovative technology, prepare to dive deep and unlock the potential of ANPR.

This guide is based on a fantastic workshop by Ruby Abdullah. You can watch the full video tutorial on YouTube here.

---

The Power of Automatic Number Plate Recognition: More Than Just Numbers

Automatic Number Plate Recognition (ANPR), often simply referred to as ANPR, is a technological marvel that utilizes Optical Character Recognition (OCR) to read and analyze vehicle license plates from images or video streams. Its ability to accurately extract information from number plates has made it indispensable across numerous sectors:

• Traffic Management & Law Enforcement: ANPR systems are crucial for detecting traffic violations like speeding or running red lights, enabling automated e-ticketing, and aiding in tracking stolen vehicles. By capturing plate details, ANPR provides critical data for traffic analysis, helping urban planners understand vehicle flow, identify popular routes, and even determine vehicle origins (e.g., how many vehicles from a specific region pass through a certain point).
• Security & Access Control: From monitoring secure areas and border crossings to automating access for authorized vehicles in private communities or corporate campuses, ANPR enhances security protocols and streamlines operations.
• Smart Parking Systems: Say goodbye to tickets and queues! ANPR-powered parking systems can automatically record entry and exit times, calculate fees, and guide drivers to available spots, leading to a much smoother parking experience.
• Logistics & Fleet Management: Companies can track their fleet movements, monitor delivery routes, and improve overall operational efficiency by integrating ANPR into their logistics systems.
• Intelligent Transportation Systems (ITS): Beyond just plates, ANPR can be part of a broader system that classifies vehicle types (cars, trucks, buses, motorcycles), providing valuable insights for urban development and policy-making. For instance, knowing the predominant type of vehicles on a road can inform infrastructure improvements.

The versatility and accuracy of Automatic Number Plate Recognition make it a cornerstone of modern smart infrastructure and security solutions.

Understanding the Core Workflow of an ANPR System

Before we delve into the code, let’s understand the logical flow that underpins every Automatic Number Plate Recognition system. This sequential process ensures that we accurately capture, identify, and interpret license plate information.

1. Input Image/Frame Acquisition: The journey begins with capturing an image or a video frame containing the vehicle. This could be from a surveillance camera, a traffic camera, or even a smartphone.
2. Vehicle Detection: This initial step involves identifying all vehicles present in the input image. While it might seem like an extra step, especially if your goal is just the plate, detecting the entire vehicle first is crucial. It helps narrow down the search area for the plate, making the subsequent plate detection more efficient and robust, especially for images where the plate might be small or less distinct. This process localizes vehicles using bounding boxes.
3. Plate Detection: Once a vehicle is identified, the system focuses on locating the specific region within that vehicle that corresponds to the license plate. This is a critical step, as the plate’s position can vary, and its appearance might be affected by angles, lighting, or partial obstructions.
4. Plate Recognition (OCR – Optical Character Recognition): With the plate located, this is where the “reading” happens. An OCR engine analyzes the pixels within the detected plate area to convert the visual characters (letters and numbers) into machine-readable text. This step is often the most challenging due to variations in fonts, plate designs, lighting conditions, and potential blurriness.
5. Type Correction: Raw OCR output can sometimes contain errors or misinterpretations (e.g., ‘O’ instead of ‘0’, or ‘I’ instead of ‘1’). Type correction involves applying rules, contextual knowledge, or fuzzy matching techniques to refine the recognized text, ensuring accuracy. This often leverages a pre-defined corpus of known plate patterns and prefixes.
6. District Classification (or Origin Extraction): For many applications, knowing the origin of a vehicle is valuable. Based on the corrected number plate text (specifically the prefix), the system can classify the vehicle’s registration district or region, providing valuable geographical context.

Each of these steps builds upon the previous one, contributing to the overall accuracy and effectiveness of the Automatic Number Plate Recognition system.

Essential Tools & Setup: Your ANPR Development Environment

To bring our Automatic Number Plate Recognition system to life, we’ll need to set up a robust development environment. Here’s a breakdown of the key dependencies and their installation:

1. Python: Our primary programming language. Ensure you have Python 3.8+ installed.
2. PyTorch: A leading open-source machine learning framework, essential for deep learning models like YOLO.
   • Recommendation: Use Konda (Anaconda/Miniconda). Konda is an excellent environment manager that simplifies package installation, especially for complex deep learning libraries like PyTorch and its CUDA dependencies. It’s free for commercial use for teams under 200 employees.
     • Installation: Download and install Konda from the official Conda documentation. Choose the installer suitable for your operating system (Windows, MacOS, Linux).
     • PyTorch Installation via Konda: Once Konda is set up, install PyTorch. Visit the official PyTorch website. Select your preferred PyTorch version, “Konda” as the package manager, “Python” as the language, and your compute platform (e.g., CUDA 11.8 for NVIDIA GPUs or “CPU” if you don’t have a compatible GPU). Copy the generated command and run it in your Konda environment (e.g., conda activate <your_env_name>).
   • NVIDIA Driver: If you choose a CUDA-enabled PyTorch build for GPU acceleration, ensure your NVIDIA GPU drivers are up-to-date.
3. Ultralytics (YOLOv8): This library provides state-of-the-art object detection models, including YOLOv8, which we’ll use for both vehicle and plate detection.

   pip install ultralytics
   

4. EasyOCR: An incredibly user-friendly and powerful OCR library for recognizing text in images.

   pip install easyocr
   

5. FuzzyWuzzy: This library helps us with string matching and similarity, crucial for our type correction step.

   pip install fuzzywuzzy python-Levenshtein # python-Levenshtein for faster processing
   

6. Pillow: A fundamental imaging library for Python, used for handling images, especially when integrating with GUI frameworks like Streamlit.

   pip install Pillow
   

7. Roboflow: We’ll be using a publicly available dataset from Roboflow Universe for training our custom plate detection model.

With these tools installed, you’re ready to build your robust Automatic Number Plate Recognition system!

Building Your Automatic Number Plate Recognition System: A Step-by-Step Tutorial

Let’s dive into the practical implementation. We’ll combine all the components discussed to create a functional ANPR system.

Step 1: Vehicle Detection

The first crucial step in our Automatic Number Plate Recognition pipeline is to accurately detect vehicles within an image. We’ll leverage a pre-trained YOLOv8 model for this.

• Objective: Identify and locate cars, buses, trucks, and motorcycles in an image.
• Method: Use a pre-trained yolov8s.pt model, which is trained on the COCO dataset and includes classes for various vehicle types.

import cv2
from ultralytics import YOLO
import numpy as np

# Load the YOLOv8 model for vehicle detection
vehicle_model = YOLO("yolov8s.pt")

# Define the image path (replace with your sample image)
image_path = "sample/1.JPG" # Make sure you have a 'sample' folder with images

# Load the image
image = cv2.imread(image_path)
if image is None:
    print(f"Error: Could not load image from {image_path}")
    exit()

# Perform inference/prediction on the vehicle
# classes=[2, 5, 7, 8] correspond to car, bus, truck, and motorcycle in COCO dataset
# agnostic_nms=True helps prevent overlapping detections between different vehicle classes
# conf=0.5 filters detections with confidence below 50%
vehicle_results = vehicle_model(image, classes=[2, 5, 7, 8], agnostic_nms=True, conf=0.5)

# Extract bounding box coordinates for detected vehicles
# .xyxy[0] assumes a single image input and gets the bounding boxes in (x1, y1, x2, y2) format
# .cpu().numpy().astype(np.int32) converts tensor to NumPy array and to integer coordinates
vehicle_xyxy = vehicle_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

# Create an output directory if it doesn't exist
import os
if not os.path.exists("out"):
    os.makedirs("out")

# Draw bounding boxes on a copy of the original image for visualization
image_annotated = image.copy()
for box in vehicle_xyxy:
    x1, y1, x2, y2 = box
    cv2.rectangle(image_annotated, (x1, y1), (x2, y2), (255, 0, 0), 2) # Blue color for vehicles

# Save the annotated image
cv2.imwrite("out/vehicle_detection_output.jpg", image_annotated)
print("Vehicle detection complete. Output saved to out/vehicle_detection_output.jpg")

Step 2: Plate Detection (Custom Training)

Detecting the license plate itself requires a specialized model, as generic object detectors like YOLOv8 on COCO don’t have a ‘license plate’ class. We’ll train our own custom YOLOv8 model for this.

• Objective: Train a model to accurately locate license plates within vehicle images.

• Method: Fine-tune a YOLOv8 model on a dedicated license plate dataset.

• Dataset Acquisition:

  • Download a license plate dataset. A great resource is Roboflow Universe.
  • For local training, choose “Download dataset” -> “Zip to computer”.
  • For Google Colab training, choose “Show download code” -> “Jupyter” and copy the provided snippet. Ensure your dataset is extracted into a structure like datasets/license_plate/train, datasets/license_plate/valid, datasets/license_plate/test with data.yaml at the root of datasets/license_plate/.
• Ultralytics Configuration: To ensure your trained model can find the dataset easily, configure Ultralytics’ settings.yaml.
  • Locate the file: ~/.config/Ultralytics/settings.yaml (on Linux/MacOS) or C:\Users\<YourUser>\AppData\Roaming\Ultralytics\settings.yaml (on Windows).
  • Edit the datasets_dir value to .. This makes the model flexible and accessible from your project root.
• Training Command:
  Once your dataset is ready and Ultralytics is configured, run the training command in your terminal:

  yolo detect train data=datasets/license_plate/data.yaml model=yolov8s.pt epochs=100 batch=4
  

  • data: Path to your dataset’s data.yaml file (e.g., datasets/license_plate/data.yaml).
  • model: We start with yolov8s.pt (a small YOLOv8 model) as a pre-trained base for fine-tuning.
  • epochs: Number of training iterations (e.g., 100). Training can be time-consuming; even 10-20 epochs might yield good results depending on your dataset size.
  • batch: Number of images processed per training step. Adjust based on your GPU memory.
  • Output: After training, your best-performing model weights will be saved in runs/detect/train/weights/best.pt. Copy this best.pt file to your project root or a designated models folder for easy access.

Step 3: Essential Helper Functions

To facilitate image processing and coordinate extraction, we’ll define a couple of helper functions.

import cv2
import numpy as np

def crop_image(image, box):
    """
    Crops an image based on bounding box coordinates (y1, x1, y2, x2).
    Args:
        image (np.array): The input image.
        box (list): A list of four integers [x1, y1, x2, y2].
    Returns:
        np.array: The cropped image.
    """
    x1, y1, x2, y2 = map(int, box) # Ensure coordinates are integers
    return image[y1:y2, x1:x2]

# The get_xyxy function from the speaker's refined code is more direct.
# It's better to extract results directly in the main loop or use the result object's attributes.
# For clarity, let's just use direct extraction as shown in the combining step.

• crop_image(image, box): This function takes an image and a bounding box (x1, y1, x2, y2) and returns the portion of the image enclosed by that box. This is crucial for isolating detected vehicles or plates.

Step 4: Integrating Vehicle and Plate Detection

Now, let’s combine the vehicle detection and our newly trained plate detection models.

import cv2
from ultralytics import YOLO
import numpy as np
import os # For checking/creating output directory

# Helper function (defined above)
def crop_image(image, box):
    x1, y1, x2, y2 = map(int, box)
    return image[y1:y2, x1:x2]

# --- Load Models ---
vehicle_model = YOLO("yolov8s.pt") # Pre-trained model for vehicle detection
# Ensure this path points to your trained plate detection model
plate_model = YOLO("runs/detect/train/weights/best.pt")

# --- Configuration ---
image_path = "sample/1.JPG" # Change to your test image

# Create 'out' directory if it doesn't exist
if not os.path.exists("out"):
    os.makedirs("out")

# --- Main Detection Logic ---
image = cv2.imread(image_path)
if image is None:
    print(f"Error: Could not load image from {image_path}")
    exit()

# 1. Vehicle Detection
vehicle_results = vehicle_model(image, classes=[2, 5, 7, 8], agnostic_nms=True, conf=0.5)
# Extract bounding box coordinates for detected vehicles
vehicle_xyxy_list = vehicle_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

# Prepare an image copy for drawing all detections
image_with_plates = image.copy()

# 2. Iterate through detected vehicles for plate detection
for vehicle_box_idx, vehicle_box in enumerate(vehicle_xyxy_list):
    # Crop the vehicle from the original image
    cropped_vehicle_img = crop_image(image, vehicle_box)

    # If the cropped image is empty (e.g., invalid coordinates), skip
    if cropped_vehicle_img.shape[0] == 0 or cropped_vehicle_img.shape[1] == 0:
        continue

    # 3. Plate Detection on the cropped vehicle image
    plate_results = plate_model(cropped_vehicle_img, agnostic_nms=True, conf=0.5)

    # Extract bounding box coordinates for detected plates within the cropped vehicle image
    plate_xyxy_in_vehicle = plate_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

    # Skip if no plate found in this vehicle
    if plate_xyxy_in_vehicle.size == 0:
        continue

    # Get the top-left corner of the vehicle box in the original image
    vehicle_x_offset, vehicle_y_offset, _, _ = vehicle_box

    # 4. Adjust plate coordinates to the original image frame and draw
    for plate_box in plate_xyxy_in_vehicle:
        plate_x1_rel, plate_y1_rel, plate_x2_rel, plate_y2_rel = plate_box

        # Adjust coordinates by adding the vehicle's offset
        adjusted_x1 = plate_x1_rel + vehicle_x_offset
        adjusted_y1 = plate_y1_rel + vehicle_y_offset
        adjusted_x2 = plate_x2_rel + vehicle_x_offset
        adjusted_y2 = plate_y2_rel + vehicle_y_offset

        # Draw the plate bounding box on the full image
        cv2.rectangle(image_with_plates, (adjusted_x1, adjusted_y1), (adjusted_x2, adjusted_y2), (0, 255, 0), 2) # Green color for plates

# Save the final image with both vehicle (optional, can be removed) and plate detections
cv2.imwrite("out/plate_detection_output.jpg", image_with_plates)
print("Plate detection complete. Output saved to out/plate_detection_output.jpg")

• This code now performs vehicle detection, then for each detected vehicle, it crops out the vehicle image and performs plate detection within that cropped region. Finally, it translates the plate coordinates back to the original image and draws the bounding box.

Step 5: Plate Recognition (OCR) with EasyOCR

Once the license plate is detected, the next critical step for Automatic Number Plate Recognition is to read the characters on it using OCR. We’ll use the EasyOCR library, known for its simplicity and effectiveness.

import cv2
import easyocr
import re
import json
from fuzzywuzzy import fuzz # Make sure you have fuzzywuzzy installed

# Helper function (defined above)
def crop_image(image, box):
    x1, y1, x2, y2 = map(int, box)
    return image[y1:y2, x1:x2]

# Initialize EasyOCR reader (run on GPU if available, otherwise CPU=False)
# 'en' specifies English language. You can add more languages if needed.
ocr_reader = easyocr.Reader(['en'], gpu=True)

# Preprocessing functions for OCR output
def remove_pattern_string(text, pattern=r'\d{2}\.\d{2}'):
  """Removes specific digit.digit patterns (e.g., '07.20') from text."""
  return re.sub(pattern, '', text)

def remove_punctuation(text):
  """Removes all punctuation from text."""
  return re.sub(r'[^\w\s]', '', text)

def process_pattern(input_string):
    """
    Formats the license plate text into a standard pattern (e.g., AD 1234 XX).
    Assumes patterns like [Letters][Digits][Letters].
    """
    # Pattern: 1-3 letters, 2-4 digits, 1-3 letters (with optional spaces)
    pattern = r'([A-Z]{1,3})([0-9]{2,4})([A-Z]{1,3})?' # Changed last group to optional

    # Try to find a match for the standard pattern
    match = re.search(pattern, input_string.upper().replace(" ", "")) # Remove spaces first for robust matching
    if match:
        # Reconstruct with spaces if a clear pattern is found
        parts = [p for p in match.groups() if p is not None]
        return ' '.join(parts).strip()
    return input_string.strip() # Return original if pattern not matched

# Load district data for type correction and classification
# This 'data.json' file should contain plate prefixes and their corresponding regions.
# Example data.json structure:
# [
#   {"kode": "AD", "daerah": ["Solo", "Sukoharjo", "Karanganyar"]},
#   {"kode": "B", "daerah": ["Jakarta", "Bekasi", "Tangerang", "Depok"]}
# ]
try:
    with open("data.json", 'r') as f:
        plate_corpus_data = json.load(f)
except FileNotFoundError:
    print("Error: data.json not found. Please create it with plate prefix information.")
    plate_corpus_data = []


# --- Re-run the combined vehicle and plate detection to integrate OCR ---
image = cv2.imread(image_path)
if image is None:
    print(f"Error: Could not load image from {image_path}")
    exit()

vehicle_results = vehicle_model(image, classes=[2, 5, 7, 8], agnostic_nms=True, conf=0.5)
vehicle_xyxy_list = vehicle_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)
image_final_output = image.copy() # Use a new image for final annotations

for vehicle_box_idx, vehicle_box in enumerate(vehicle_xyxy_list):
    cropped_vehicle_img = crop_image(image, vehicle_box)
    if cropped_vehicle_img.shape[0] == 0 or cropped_vehicle_img.shape[1] == 0:
        continue

    plate_results = plate_model(cropped_vehicle_img, agnostic_nms=True, conf=0.5)
    plate_xyxy_in_vehicle = plate_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

    if plate_xyxy_in_vehicle.size == 0:
        continue

    vehicle_x_offset, vehicle_y_offset, _, _ = vehicle_box

    for plate_box in plate_xyxy_in_vehicle:
        plate_x1_rel, plate_y1_rel, plate_x2_rel, plate_y2_rel = plate_box

        # --- Filtering small/invalid plates for better OCR results (optional but recommended) ---
        plate_width = plate_x2_rel - plate_x1_rel
        plate_height = plate_y2_rel - plate_y1_rel
        if plate_width < 50 or plate_height < 20: # Example threshold
            continue # Skip very small or distorted plates that might yield bad OCR

        adjusted_x1 = plate_x1_rel + vehicle_x_offset
        adjusted_y1 = plate_y1_rel + vehicle_y_offset
        adjusted_x2 = plate_x2_rel + vehicle_x_offset
        adjusted_y2 = plate_y2_rel + vehicle_y_offset

        # Crop the plate image from the cropped vehicle image
        plate_image_cropped = crop_image(cropped_vehicle_img, plate_box)

        # EasyOCR expects RGB, OpenCV reads BGR
        plate_image_rgb = cv2.cvtColor(plate_image_cropped, cv2.COLOR_BGR2RGB)

        # Perform OCR
        raw_plate_text_results = ocr_reader.readtext(plate_image_rgb)

        detected_text = ""
        # EasyOCR returns a list of (bbox, text, confidence)
        if raw_plate_text_results:
            # For simplicity, concatenate all detected text. For robust systems,
            # you might need to filter based on confidence or combine logically.
            for (bbox, text_part, prob) in raw_plate_text_results:
                detected_text += text_part

        # --- Text Preprocessing ---
        detected_text = detected_text.upper() # Convert to uppercase
        detected_text = remove_pattern_string(detected_text) # Remove unwanted patterns
        detected_text = remove_punctuation(detected_text) # Remove punctuation
        processed_text = process_pattern(detected_text) # Apply pattern formatting

        # Draw plate bounding box
        cv2.rectangle(image_final_output, (adjusted_x1, adjusted_y1), (adjusted_x2, adjusted_y2), (0, 255, 0), 2) # Green for plates

        # Put the recognized text on the image
        font = cv2.FONT_HERSHEY_SIMPLEX
        font_scale = 0.8
        font_thickness = 2
        text_color = (36, 255, 12) # BGR: Light green

        # Position text just above the bounding box
        cv2.putText(image_final_output, processed_text, (adjusted_x1, adjusted_y1 - 10), font, font_scale, text_color, font_thickness)

# Save the final image with all annotations
cv2.imwrite("out/plate_recognition_output.jpg", image_final_output)
print("Plate recognition complete. Output saved to out/plate_recognition_output.jpg")

• OCR Integration: We initialize easyocr.Reader and then call ocr_reader.readtext() on the cropped plate image.
• Preprocessing: The recognized text undergoes several cleaning steps: converting to uppercase, removing specific unwanted patterns (like dates), stripping punctuation, and reformatting it into a standard license plate pattern (e.g., adding spaces between letter and number groups). This greatly enhances the quality of the raw OCR output.

Step 6: Enhancing Accuracy with Type Correction & District Extraction

Even after preprocessing, OCR can make mistakes. For our Automatic Number Plate Recognition system to be truly robust, we implement a type correction mechanism and extract the district based on common Indonesian plate codes.

import cv2
import easyocr
import re
import json
from fuzzywuzzy import fuzz
import os # For checking/creating output directory

# Helper function (defined above)
def crop_image(image, box):
    x1, y1, x2, y2 = map(int, box)
    return image[y1:y2, x1:x2]

# OCR and Preprocessing functions (defined above)
def remove_pattern_string(text, pattern=r'\d{2}\.\d{2}'):
  return re.sub(pattern, '', text)

def remove_punctuation(text):
  return re.sub(r'[^\w\s]', '', text)

def process_pattern(input_string):
    pattern = r'([A-Z]{1,3})([0-9]{2,4})([A-Z]{1,3})?'
    match = re.search(pattern, input_string.upper().replace(" ", ""))
    if match:
        parts = [p for p in match.groups() if p is not None]
        return ' '.join(parts).strip()
    return input_string.strip()

# --- Type Correction and District Extraction Function ---
def type_correction_and_district(plate_text_raw, plate_corpus_data):
    """
    Corrects the plate text (specifically the prefix) and detects the district
    based on a corpus of known plate codes and regions.
    Args:
        plate_text_raw (str): The raw processed text from OCR.
        plate_corpus_data (list): Loaded data from data.json.
    Returns:
        tuple: (corrected_plate_text, detected_district_list)
    """
    if not plate_text_raw:
        return "", None

    # Get the first part of the plate (e.g., "AD" from "AD 1234 XX")
    text_to_correct = plate_text_raw.split()[0].upper() # Ensure uppercase for comparison

    maximum_score = 0
    best_match_code = ""
    best_match_district = []

    # Iterate through the corpus to find the best match
    for entry in plate_corpus_data:
        corpus_code = entry["kode"].upper()
        # Use fuzzy matching to compare the extracted prefix with corpus codes
        ratio = fuzz.ratio(text_to_correct, corpus_code)

        # If similarity ratio is high and better than previous matches
        # Threshold of 60 is a common starting point for fuzzy matching.
        # It means 60% similarity is required.
        if ratio > maximum_score and ratio >= 60:
            maximum_score = ratio
            best_match_code = entry["kode"]
            best_match_district = entry["daerah"]

    # If a good match was found, replace the prefix with the corrected one
    if maximum_score > 0:
        # Reconstruct the plate text with the corrected prefix
        corrected_plate_text = plate_text_raw.replace(text_to_correct, best_match_code, 1)
        return corrected_plate_text, best_match_district
    else:
        # If no good match, return original text and None for district
        return plate_text_raw, None

# Initialize EasyOCR reader (run on GPU if available, otherwise CPU=False)
ocr_reader = easyocr.Reader(['en'], gpu=True)

# Load district data for type correction and classification
try:
    with open("data.json", 'r') as f:
        plate_corpus_data = json.load(f)
except FileNotFoundError:
    print("Error: data.json not found. Please create it with plate prefix information.")
    plate_corpus_data = [] # Empty list if file not found

# --- Main Detection Logic with OCR, Type Correction, and District Extraction ---
image = cv2.imread(image_path)
if image is None:
    print(f"Error: Could not load image from {image_path}")
    exit()

vehicle_results = vehicle_model(image, classes=[2, 5, 7, 8], agnostic_nms=True, conf=0.5)
vehicle_xyxy_list = vehicle_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)
image_final_output = image.copy() # Use a new image for final annotations

for vehicle_box_idx, vehicle_box in enumerate(vehicle_xyxy_list):
    cropped_vehicle_img = crop_image(image, vehicle_box)
    if cropped_vehicle_img.shape[0] == 0 or cropped_vehicle_img.shape[1] == 0:
        continue

    plate_results = plate_model(cropped_vehicle_img, agnostic_nms=True, conf=0.5)
    plate_xyxy_in_vehicle = plate_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

    if plate_xyxy_in_vehicle.size == 0:
        continue

    vehicle_x_offset, vehicle_y_offset, _, _ = vehicle_box

    for plate_box in plate_xyxy_in_vehicle:
        plate_x1_rel, plate_y1_rel, plate_x2_rel, plate_y2_rel = plate_box

        plate_width = plate_x2_rel - plate_x1_rel
        plate_height = plate_y2_rel - plate_y1_rel
        if plate_width < 50 or plate_height < 20: # Example threshold
            continue

        adjusted_x1 = plate_x1_rel + vehicle_x_offset
        adjusted_y1 = plate_y1_rel + vehicle_y_offset
        adjusted_x2 = plate_x2_rel + vehicle_x_offset
        adjusted_y2 = plate_y2_rel + vehicle_y_offset

        plate_image_cropped = crop_image(cropped_vehicle_img, plate_box)
        plate_image_rgb = cv2.cvtColor(plate_image_cropped, cv2.COLOR_BGR2RGB)

        raw_plate_text_results = ocr_reader.readtext(plate_image_rgb)
        detected_text = ""
        if raw_plate_text_results:
            for (bbox, text_part, prob) in raw_plate_text_results:
                detected_text += text_part

        detected_text = detected_text.upper()
        detected_text = remove_pattern_string(detected_text)
        detected_text = remove_punctuation(detected_text)
        processed_text = process_pattern(detected_text)

        # --- Perform Type Correction and District Extraction ---
        final_plate_text, district_list = type_correction_and_district(processed_text, plate_corpus_data)

        # Draw plate bounding box
        cv2.rectangle(image_final_output, (adjusted_x1, adjusted_y1), (adjusted_x2, adjusted_y2), (0, 255, 0), 2)

        # Put final plate text
        cv2.putText(image_final_output, final_plate_text, (adjusted_x1, adjusted_y1 - 10), font, font_scale, text_color, font_thickness)

        # Put district information if available
        if district_list:
            district_info = ", ".join(district_list)
            # Position district text below the plate box, adjusted_y2 + 20 pixels below
            cv2.putText(image_final_output, district_info, (adjusted_x1, adjusted_y2 + 20), font, font_scale * 0.8, (255, 12, 36), font_thickness) # Red color for district

# Save the final annotated image
cv2.imwrite("out/final_anpr_output.jpg", image_final_output)
print("Full ANPR pipeline complete. Output saved to out/final_anpr_output.jpg")

• type_correction_and_district Function: This new function uses fuzzywuzzy to compare the OCR’d plate prefix with a list of known prefixes from data.json. If a strong match is found (e.g., fuzz.ratio > 60), it corrects the prefix and extracts the associated district information.
• data.json: You’ll need to create a data.json file in your project root. This file should contain a list of dictionaries, where each dictionary has a “kode” (plate prefix) and “daerah” (list of associated regions/cities).

Step 7: Building a Simple GUI with Streamlit (Bonus)

To make our Automatic Number Plate Recognition system more interactive, we can build a simple web interface using Streamlit, a fantastic framework for creating data apps with Python.

import streamlit as st
import cv2
from PIL import Image
from ultralytics import YOLO
import numpy as np
import re
import json
from fuzzywuzzy import fuzz
import os
import easyocr

# --- Helper Functions (re-define for Streamlit script) ---
def crop_image(image, box):
    x1, y1, x2, y2 = map(int, box)
    return image[y1:y2, x1:x2]

def remove_pattern_string(text, pattern=r'\d{2}\.\d{2}'):
  return re.sub(pattern, '', text)

def remove_punctuation(text):
  return re.sub(r'[^\w\s]', '', text)

def process_pattern(input_string):
    pattern = r'([A-Z]{1,3})([0-9]{2,4})([A-Z]{1,3})?'
    match = re.search(pattern, input_string.upper().replace(" ", ""))
    if match:
        parts = [p for p in match.groups() if p is not None]
        return ' '.join(parts).strip()
    return input_string.strip()

def type_correction_and_district(plate_text_raw, plate_corpus_data):
    if not plate_text_raw:
        return "", None
    text_to_correct = plate_text_raw.split()[0].upper()
    maximum_score = 0
    best_match_code = ""
    best_match_district = []
    for entry in plate_corpus_data:
        corpus_code = entry["kode"].upper()
        ratio = fuzz.ratio(text_to_correct, corpus_code)
        if ratio > maximum_score and ratio >= 60:
            maximum_score = ratio
            best_match_code = entry["kode"]
            best_match_district = entry["daerah"]
    if maximum_score > 0:
        corrected_plate_text = plate_text_raw.replace(text_to_correct, best_match_code, 1)
        return corrected_plate_text, best_match_district
    else:
        return plate_text_raw, None

# --- Streamlit UI ---
st.title("Automatic Number Plate Recognition (ANPR) System")
st.write("Upload an image to detect vehicles, recognize license plates, and classify their origin.")

# --- Load Models (Cache models to avoid reloading on every rerun) ---
@st.cache_resource
def load_anpr_models():
    vehicle_model = YOLO("yolov8s.pt")
    plate_model = YOLO("runs/detect/train/weights/best.pt") # Ensure this path is correct
    ocr_reader = easyocr.Reader(['en'], gpu=True) # Set gpu=False if no GPU

    plate_corpus_data = []
    try:
        with open("data.json", 'r') as f:
            plate_corpus_data = json.load(f)
    except FileNotFoundError:
        st.error("data.json not found. Please create it with plate prefix information.")

    return vehicle_model, plate_model, ocr_reader, plate_corpus_data

vehicle_model, plate_model, ocr_reader, plate_corpus_data = load_anpr_models()

# --- File Uploader ---
uploaded_file = st.file_uploader("Upload an image for ANPR", type=["jpg", "jpeg", "png"])

if uploaded_file is not None:
    # Read the image using PIL and convert to OpenCV format (NumPy array)
    image = Image.open(uploaded_file)
    # Convert PIL Image to OpenCV format (BGR)
    # If image is RGBA (has alpha channel), convert to RGB first, then BGR
    if image.mode == 'RGBA':
        image = image.convert('RGB')
    image_np = np.array(image)
    image_bgr = cv2.cvtColor(image_np, cv2.COLOR_RGB2BGR) # OpenCV uses BGR

    st.image(image_bgr, channels="BGR", caption="Original Image", use_column_width=True)
    st.write("Detecting...")

    # Perform ANPR pipeline
    image_annotated_final = image_bgr.copy() # Use a copy for drawing

    vehicle_results = vehicle_model(image_bgr, classes=[2, 5, 7, 8], agnostic_nms=True, conf=0.5)
    vehicle_xyxy_list = vehicle_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

    for vehicle_box_idx, vehicle_box in enumerate(vehicle_xyxy_list):
        cropped_vehicle_img = crop_image(image_bgr, vehicle_box)
        if cropped_vehicle_img.shape[0] == 0 or cropped_vehicle_img.shape[1] == 0:
            continue

        plate_results = plate_model(cropped_vehicle_img, agnostic_nms=True, conf=0.5)
        plate_xyxy_in_vehicle = plate_results[0].boxes.xyxy.cpu().numpy().astype(np.int32)

        if plate_xyxy_in_vehicle.size == 0:
            continue

        vehicle_x_offset, vehicle_y_offset, _, _ = vehicle_box

        for plate_box in plate_xyxy_in_vehicle:
            plate_x1_rel, plate_y1_rel, plate_x2_rel, plate_y2_rel = plate_box

            plate_width = plate_x2_rel - plate_x1_rel
            plate_height = plate_y2_rel - plate_y1_rel
            if plate_width < 50 or plate_height < 20:
                continue

            adjusted_x1 = plate_x1_rel + vehicle_x_offset
            adjusted_y1 = plate_y1_rel + vehicle_y_offset
            adjusted_x2 = plate_x2_rel + vehicle_x_offset
            adjusted_y2 = plate_y2_rel + vehicle_y_offset

            plate_image_cropped = crop_image(cropped_vehicle_img, plate_box)
            plate_image_rgb = cv2.cvtColor(plate_image_cropped, cv2.COLOR_BGR2RGB)

            raw_plate_text_results = ocr_reader.readtext(plate_image_rgb)
            detected_text = ""
            if raw_plate_text_results:
                for (bbox, text_part, prob) in raw_plate_text_results:
                    detected_text += text_part

            detected_text = detected_text.upper()
            detected_text = remove_pattern_string(detected_text)
            detected_text = remove_punctuation(detected_text)
            processed_text = process_pattern(detected_text)

            final_plate_text, district_list = type_correction_and_district(processed_text, plate_corpus_data)

            # Draw plate bounding box
            cv2.rectangle(image_annotated_final, (adjusted_x1, adjusted_y1), (adjusted_x2, adjusted_y2), (0, 255, 0), 2)

            # Put final plate text
            font = cv2.FONT_HERSHEY_SIMPLEX
            font_scale = 0.8
            font_thickness = 2
            text_color = (36, 255, 12)

            cv2.putText(image_annotated_final, final_plate_text, (adjusted_x1, adjusted_y1 - 10), font, font_scale, text_color, font_thickness)

            # Put district information
            if district_list:
                district_info = ", ".join(district_list)
                cv2.putText(image_annotated_final, district_info, (adjusted_x1, adjusted_y2 + 20), font, font_scale * 0.8, (255, 12, 36), font_thickness)

    st.subheader("Detection Results:")
    st.image(image_annotated_final, channels="BGR", caption="Detected Plates and Information", use_column_width=True)

# To run this Streamlit app, save it as `app.py` and run `streamlit run app.py` in your terminal.

• Streamlit Setup: Import streamlit as st, PIL.Image, and integrate the entire ANPR pipeline within a Streamlit app.
• File Uploader: Streamlit’s st.file_uploader allows users to easily upload images.
• Image Handling: Pillow is used to open the uploaded image, and NumPy/OpenCV convert it into the appropriate format for processing.
• @st.cache_resource: This decorator is crucial for Streamlit apps. It caches the loaded models and data, preventing them from being reloaded every time the app reruns (e.g., when a user uploads a new image), significantly improving performance.
• Display Results: st.image() displays the annotated image.

Beyond the Basics: Next Steps for Your Automatic Number Plate Recognition System

While the system we’ve built is functional, there’s always room for improvement and adaptation for real-world scenarios, especially in a production-grade Automatic Number Plate Recognition solution.

• Expand Your Plate Corpus (data.json): Our data.json currently covers a limited number of Indonesian plate prefixes. To improve district classification accuracy, you’ll need to expand this corpus to include all 55+ official Indonesian plate codes and their corresponding regions. The more comprehensive your data, the more accurate your system will be.
• Robustness for Extreme Cases: The current OCR solution (EasyOCR) performs well under ideal conditions. However, real-world scenarios involve blurry, angled, distorted, or poorly lit license plates. For robust Automatic Number Plate Recognition in such “extreme cases,” you’ll need to:
  • Collect More Diverse Data: Train your custom plate detection and OCR models on a wider variety of challenging images.
  • Custom OCR Training: Consider training your own custom OCR model (e.g., using deep learning architectures like CNN-RNN with CTC Loss) that is specifically optimized for license plate characters and common distortions. This is a more advanced step but yields significant improvements in recognition accuracy.
• Object Tracking for Streaming Optimization: If your ANPR system is intended for video streams (e.g., traffic monitoring), running the full pipeline (vehicle detection, plate detection, OCR, etc.) on every single frame for every single object will be extremely computationally expensive and lead to high latency.
  • The Solution: Implement object tracking. Object tracking allows the system to assign a unique ID to each detected vehicle and follow it across multiple frames.
  • How it Optimizes: Instead of processing ANPR for every frame, you can run the full ANPR pipeline (plate detection, OCR, type correction, district classification) only once when a new vehicle is first detected or when its plate becomes clear. For subsequent frames, as long as the tracker confirms it’s the same vehicle, you can simply use the previously recognized plate data stored in a temporary variable or a database. This dramatically reduces the computational load and latency, making the system viable for real-time video processing.
  • Decision Logic: You might also add logic to trigger ANPR processing only when the detected plate reaches a certain size or clarity threshold to ensure high-quality OCR results.

These advanced considerations are vital for deploying a highly reliable and performant Automatic Number Plate Recognition system in dynamic environments.

---

Conclusion

You’ve now walked through the entire process of building an Automatic Number Plate Recognition system, from basic vehicle detection to advanced text processing and district classification. This journey has highlighted the power of combining modern computer vision techniques (YOLOv8, EasyOCR) with intelligent data handling (type correction, corpus-based district mapping).

The applications of Automatic Number Plate Recognition are vast and continuously expanding, making it a critical skill for anyone interested in AI, smart infrastructure, and data-driven solutions. Experiment with different datasets, fine-tune your models, and explore the advanced topics mentioned to push the boundaries of your ANPR system. The world of computer vision is constantly evolving, and with the foundational knowledge you’ve gained, you’re well-equipped to contribute to its exciting future.

Ready to dive deeper into computer vision and machine learning? Explore more resources and courses on https://rubythalib.ai/!