SR Model using Deep learning

Residual Networks

The code given below demonstrates the conversion of a low-resolution(LR) image to a high-resolution(HR) image using the Super-Resolution(SR) model

Step 1: Import the necessary libraries

Python3

# Importing all the required packages and libraries
import tensorflow as tf
import tensorflow_hub as hub
import cv2
import requests
import numpy as np
import matplotlib.pyplot as plt

Step 2: Load the input image and plot it.

Python3

# Loading the image of the GFG Logo
img = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTdVHUvpMzlUKnxGtZSXcZ1XXZLxfu9hqc8BB77sNTcGjSbiLhLlqRpntUZhk222DQV9UM&usqp=CAU"
getContent = requests.get(img).content
array_img = np.asarray(bytearray(getContent), dtype=np.uint8)
 
# Change the color space BGR to RGB
image_plot = cv2.cvtColor(cv2.imdecode(array_img, -1), cv2.COLOR_BGR2RGB)
 
# Plot the image
plt.figure(figsize=(10, 10))
plt.title(image_plot.shape)
plt.imshow(image_plot)
plt.show()

Output:

Input image

Step 3: Preprocess the image

Python3

# Model to preprocess the images
def preprocessing(img):
    imageSize = (tf.convert_to_tensor(image_plot.shape[:-1]) // 4) * 4
    cropped_image = tf.image.crop_to_bounding_box(
        img, 0, 0, imageSize[0], imageSize[1])
    preprocessed_image = tf.cast(cropped_image, tf.float32)
    return tf.expand_dims(preprocessed_image, 0)

Step 4: Here we are using a pre-trained Enhanced Super Resolution GAN (ESRGAN) Model from tfhub[https://tfhub.dev/captain-pool/esrgan-tf2/1]. Load the model.

Python3

# This is a model of Enhanced Super Resolution GAN Model
# The link given here is a model of ESRGAN model
esrgn_path = "https://tfhub.dev/captain-pool/esrgan-tf2/1"
model = hub.load(esrgn_path)

Step 5: Employ the model

Python3

# Employ the model
def srmodel(img):
    preprocessed_image = preprocessing(img)  # Preprocess the LR Image
    new_image = model(preprocessed_image)  # Runs the model
    # returns the size of the original argument that is given as input
    return tf.squeeze(new_image) / 255.0

Step 6: Plot the Super-Resolution output image.

Python3

# Plot the HR image
hr_image = srmodel(image_plot)
 
plt.title(hr_image.shape)
plt.imshow(hr_image)
plt.show()

Output:

Output Image

Complete code

The below code will take input an image and convert it into a high-resolution image.

Python3

# Importing all the required packages and libraries
import tensorflow as tf
import tensorflow_hub as hub
import cv2
import requests
import numpy as np
import matplotlib.pyplot as plt
 
# Loading the image of the GFG Logo
img = cv2.imread('GFG.jpeg')
 
image_plot = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.title(image_plot.shape)
plt.imshow(image_plot)
plt.show()
 
# Running the SR model
 
# Model to preprocess the images
 
 
def preprocessing(img):
    imageSize = (tf.convert_to_tensor(image_plot.shape[:-1]) // 4) * 4
    cropped_image = tf.image.crop_to_bounding_box(
        img, 0, 0, imageSize[0], imageSize[1])
    preprocessed_image = tf.cast(cropped_image, tf.float32)
    return tf.expand_dims(preprocessed_image, 0)
 
 
# This is a model of Enhanced Super Resolution GAN Model
# The link given here is a model of ESRGAN model
esrgn_path = "https://tfhub.dev/captain-pool/esrgan-tf2/1"
model = hub.load(esrgn_path)
 
# Model to employ the model
 
 
def srmodel(img):
    preprocessed_image = preprocessing(img)  # Preprocess the LR Image
    new_image = model(preprocessed_image)  # Runs the model
    # returns the size of the original argument that is given as input
    return tf.squeeze(new_image) / 255.0
 
 
# Plot the HR image
hr_image = srmodel(image_plot)
plt.title(hr_image.shape)
plt.imshow(hr_image)
plt.show()

Output:

Input Image

Output Image

Python OpenCV – Super resolution with deep learning

Super-resolution (SR) implies the conversion of an image from a lower resolution (LR) to images with a higher resolution (HR). It makes wide use of augmentation. It forms the basis of most computer vision and image processing models. However, with the advancements in deep learning technologies, deep learning-based super resolutions have gained the utmost importance. Almost all the deep learning models would make great use of super-resolution. Since Super Resolution mainly uses augmentations of data points, it is also called hallucination of the data points.

SR plays an important role in image improvement and restoration. The SR process is carried out as follows- First, a low-resolution image is taken as the input. Next, the image is upscaled and the resolution of the images are increased to a higher resolution and given as an output.

Need for Deep learning based Super Resolution

The traditional Super Resolution Model that does not make use of Deep learning lacks fine details. They fail to remove various defects and compression facts in the systems. All of these problems can be very efficiently addressed by using a deep learning-based SR model to get an image of a higher resolution keeping all the details intact.

Some commonly used conventional SR models are

Structured illumination microscopy (or SIM)
Stochastic optical reconstruction microscopy (STORM)
Photo-activated localization microscopy (PALM)
Stimulated emission depletion (STED)

SR Model using Deep learning

Python3

Python3

Python3

Python3

Python3

Python3

Python3

Output:

Python OpenCV – Super resolution with deep learning

Need for Deep learning based Super Resolution

Super Resolution using Deep Learning methods:

Similar Reads