Aerial Image Analysis & Computer Vision

A comprehensive deep dive into Computer Vision: from classic mathematical image registration to Bag-of-Visual-Words and custom Residual Neural Networks.

Project Overview

This repository represents my complete journey through advanced Computer Vision, structured into three progressively complex assignments. The focus is entirely on aerial/satellite imagery, moving from foundational pixel-level mathematics to modern deep learning architectures.

All implementations were written from scratch in Python, utilizing OpenCV, NumPy, Scikit-Learn, and TensorFlow.

---

Part 1: Multimodal Image Registration

The first challenge involved aligning pairs of aerial images captured by different sensors (multimodal registration). Since the images had different color mappings and intensities, simple pixel-to-pixel subtraction was impossible.

Technical Approach

• Mutual Information (MI): I implemented the MI metric from scratch calculating the joint and marginal entropy of the two images H(Ir) + H(Im) − H(Ir, Im).
• Optimization Strategy: I used scipy.optimize.minimize (specifically the Nelder-Mead and Powell algorithms) to find the optimal affine transformation parameters (Translation X, Y and Rotation $\theta$) that maximized Mutual Information.
• Preprocessing Pipeline: To improve convergence, I built a pipeline that tested different image transformations: Grayscale, HSV (V-channel), and Principal Component Analysis (PCA) projection along the first principal component, coupled with Gaussian blurring.

“By projecting both images along the first Principal Component (PCA), the optimizer reached convergence significantly faster, bypassing the extreme differences in original RGB pixel intensities.”

---

Part 2: Bag-of-Visual-Words (BoVW)

Before jumping into Deep Learning, I established a robust baseline for aerial scene classification using the AID dataset.

Pipeline Implementation

1. Feature Extraction: Extracted thousands of local feature descriptors using cv2.SIFT.
2. Vocabulary Building: Handled an upper bound of 1 million descriptors, L2-normalized them, and fed them into a K-Means clustering algorithm (K=50, 100, 500) to create “Visual Words”.
3. Image Representation: Converted every image into an L1-normalized histogram of visual words.
4. Classification: Evaluated multiple ML models (SVM Linear/RBF, Random Forest, Logistic Regression, KNN) using Stratified K-Fold cross-validation.
5. Results: The best performing pipeline achieved 0.732 F1-Macro and 0.975 AUC, serving as a highly competitive baseline.

---

Part 3: Custom Residual Convolutional Neural Network

The final assignment pushed the boundaries of classification by leveraging Deep Learning on the UCMerced_LandUse dataset. Instead of simply fine-tuning a generic ResNet50, I engineered a custom neural network from the ground up.

Custom Architecture Design

• Stem & Residual Blocks: Engineered a bespoke convolutional backbone starting with a 7x7 Convolutional stem, followed by three progressively deeper Residual Blocks. Each block incorporated Batch Normalization and skip connections with 1x1 convolutions for strict dimensional matching.
• MLP Classification Head: The backbone (producing a high-dimensional 6272-D feature vector) was coupled with a Multi-Layer Perceptron (MLP) classifier using aggressive Dropout (0.5) to mitigate overfitting on smaller datasets.
• Parameter Optimization: The architectural capacity was meticulously balanced, allocating approximately ~1 Million trainable parameters to the feature extraction backbone and ~1 Million to the dense classification head.

Transfer Learning Strategy

To validate the robustness of the architecture, I designed an ablation study comparing two distinct training paradigms:

1. From-Scratch Optimization: The architecture was initialized randomly and trained exclusively on the target UCMerced dataset, serving as the deep learning baseline.
2. Cross-Domain Transfer Learning: The network was first pre-trained on a larger, domain-adjacent dataset (AID) to learn generalized hierarchical visual features. The convolutional backbone was then frozen, and a newly initialized classification head was fine-tuned on the UCMerced dataset.

The Verdict

The deep learning paradigm vastly outperformed the traditional BoVW baseline. The custom residual architecture achieved a remarkable 0.889 F1-Macro score. Furthermore, the Cross-Domain Transfer Learning strategy demonstrated significantly faster convergence rates and greater numerical stability during gradient descent compared to the from-scratch approach.