Mask R-CNN

Mask R-CNN (Mask Region-based Convolutional Neural Network) is a Faster R-CNN object identification framework upgrade that adds the ability to do instance segmentation. It was proposed by Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick in their work “Mask R-CNN” published in 2017.

Instance segmentation is the task of not only detecting objects in an image but also segmenting each object instance at the pixel level, providing a binary mask for each detected object. Mask R-CNN develops on Faster R-CNN’s two-stage architecture with a third branch for pixel-level segmentation masks.

The following are the essential features and components of Mask R-CNN:

1. Region Proposal Network (RPN): Mask R-CNN uses an RPN to generate region proposals, just like Faster R-CNN does. The RPN generates candidate bounding boxes that are likely to contain objects of interest.
2. Region of Interest (RoI): Mask R-CNN introduces RoI Align, a more accurate technique for aligning pixel-level features within the region proposals, in place of RoI pooling used in Faster R-CNN. RoI Align ensures that the pixel-level features are accurately extracted from the original image feature map without quantization.
3. Instance Segmentation: Faster R-CNN uses two branches: classification and bounding box regression. Mask R-CNN adds a third branch that forecasts the segmentation masks for each region proposal. This branch generates a binary mask for each identified object using the RoI-aligned features as its input.

Image segmentation plays a crucial role in computer vision tasks, enabling machines to understand and analyze visual content at a pixel level. It involves dividing an image into distinct regions or objects, facilitating object recognition, tracking, and scene understanding. In this article, we explore three popular image segmentation techniques: Mask R-CNN, GrabCut, and OpenCV.

Let’s understand, What Image Segmentation with Mask R-CNN and GrabCut are?

What is R-CNN?

R-CNN stands for Region-based Convolutional Neural Network. It is a ground-breaking object detection system that combines object localization and recognition into an end-to-end deep learning framework. 

RNN can be summarised in the following ways.

• Region Proposal: Initially, a region proposal algorithm (such as selective search) generates a set of potential bounding box regions in an image that are likely to contain objects of interest. These regions serve as candidate object locations.
• Feature Extraction: Each region proposal is then individually cropped and resized to a fixed size and passed through a pre-trained CNN (such as AlexNet or VGGNet). The CNN extracts high-level features from the region, transforming it into a fixed-length feature vector.
• Classification and Localization: The feature vector obtained from the CNN is fed into separate fully connected layers. The classification layer predicts the probability of different object classes present in the region, while the regression layer refines the coordinates of the bounding box, improving localization accuracy.
• Non-Maximum Suppression (NMS): To eliminate redundant detections, non-maximum suppression is applied. It removes overlapping bounding boxes, keeping only the most confident detection for each object instance.