Valid Padding
Valid padding is a technique used in convolutional neural networks (CNNs) to process the input data without adding any additional rows or columns of pixels around the edges of the data. This means that the size of the output feature map is smaller than the size of the input data. Valid padding is used when it is desired to reduce the size of the output feature map in order to reduce the number of parameters in the model and improve its computational efficiency.
Formula to find the output shape of valid padding:
Where, = Number of rows and column of input data.
= Filter kernel dimensions,
= Stride
Here is an example of how to create a convolutional layer with valid padding and a stride of 2 in the vertical and horizontal dimensions :
Python3
import tensorflow as tffrom tensorflow.keras.layers import Conv2Dinput_shape = (1, 28, 28, 3) # batch_size = 1, height=28, width=28, channels=3x = tf.random.normal(input_shape) # Create a random data with tf.random of input_shapey = Conv2D(filters=3, #the number of output filters in the convolution kernel_size=(3, 3), # filter kernel dimension strides=(2, 2), # Shift along height and width padding='valid', # padding data_format ='channels_last', # Input shape : (batch_size, height, width, channels) input_shape=input_shape # input shape )(x)print('Output shape :',y.shape[1:]) # y.shape[0] : batch_size |
Outputs:
Output shape : (13, 13, 3)
In the output, channels will be equal to the filters i.e c = 3
Let’s calculate the output shape with the formula :
You can also specify different stride values for the vertical and horizontal dimensions by providing a tuple with different values. For example:
Python3
# import tensorflow as tffrom tensorflow.keras.layers import Conv2Dinput_shape = (1, 28, 28, 3) # batch_size = 1, height=28, width=28, channels=3x = tf.random.normal(input_shape) # Create a random data with tf.random of input_shapey = Conv2D(filters=10, #the number of output filters in the convolution kernel_size=(2,2), # filter kernel dimension strides=(3, 3), # Shift along height and width padding='valid', # padding data_format ='channels_last', # Input shape : (batch_size, height, width, channels) input_shape=input_shape # input shape )(x)print('Output shape :',y.shape[1:]) # y.shape[0] : batch_size |
Output :
Output shape : (9, 9, 10)
In the output, channels will be equal to the filters i.e c = 10
Let’s calculate the output shape with the formula :
When valid padding is used, the output feature map is smaller than the input data because the filters do not extend beyond the edges of the input data. This can result in the loss of information at the edges of the image, which can affect the performance of the CNN. Valid padding is also known as no padding because no padding is applied to the input data before the convolution operation.
Let’s try with tf.nn.max_pool in TensorFlow to perform max pooling with valid padding and stride 2
Valid Padding
Python3
import tensorflow as tf# Create a 5x5 input tensorinput_tensor = tf.constant( [ [1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]])# Reshape the input tensor to [batch_size, height, width, channels]input_tensor = tf.expand_dims(input_tensor, axis=0)input_tensor = tf.expand_dims(input_tensor, axis=3)print('Input Tensor shape:', input_tensor.shape)# Perform max pooling with a window size of 3x3 and a stride of 2output_tensor = tf.nn.max_pool(input = input_tensor, ksize=[1, 3, 3, 1], # kernel size, equivalent to (3,3) strides=2, # stride, equivalent to [1, 2, 2, 1], padding='VALID')# Print the output tensorprint('\noutput_tensor :\n',output_tensor) |
Output :
Input Tensor shape: (1, 5, 5, 1) output_tensor : tf.Tensor( [[[[13] [15]] [[23] [25]]]], shape=(1, 2, 2, 1), dtype=int32)
Code Explanation
In this code, a 5×5 input tensor is created using the tf.constant function. This input tensor represents a 2D array with 5 rows and 5 columns, and its values are set to the integers from 1 to 25.
Next, the input tensor is reshaped to [batch_size, height, width, channels] using the tf.expand_dims function. The axis parameter specifies the dimension along which to expand the shape of the tensor. In this case, the input tensor is expanded along the first and third dimensions, resulting in an input tensor with shape [1, 5, 5, 1].
Then, the tf.nn.max_pool function is used to perform max pooling on the input tensor. The function takes in the input tensor, the size of the pooling window (ksize), the stride of the sliding window (strides), and the type of padding to use (padding). In this example, the pooling window has a size of [1, 3, 3, 1], which means that the window will be 3×3. The stride is set to 2 or [1, 2, 2, 1], which means that the window will move 2 units in both the height and width directions. The padding is set to ‘VALID’, which means that no padding will be used.
Finally, the output tensor is printed using the print function. The output tensor will have a shape of [1, 2, 2, 1], and it will contain the maximum values from each 3×3 window of the input tensor.
Here is an example of using tf.nn.max_pool in TensorFlow to perform max pooling with valid padding and stride 1
Valid Padding Stride = 1
Python3
import tensorflow as tf# Create a 5x5 input tensorinput_tensor = tf.constant([ [1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]])# Reshape the input tensor to [batch_size, height, width, channels]input_tensor = tf.expand_dims(input_tensor, axis=0)input_tensor = tf.expand_dims(input_tensor, axis=3)print('Input Tensor shape:', input_tensor.shape)# Perform max pooling with a window size of 3x3 and a stride of 1output_tensor = tf.nn.max_pool(input = input_tensor, ksize=[1, 3, 3, 1], #kernel size, equivalent to (3,3) strides=[1, 1, 1, 1], # stride, equivalent to 1 shift padding='VALID' # padding = valid )# Print the output tensorprint('\noutput_tensor :\n',output_tensor) |
Outputs:
Input Tensor shape: (1, 5, 5, 1) output_tensor : tf.Tensor( [[[[13] [14] [15]] [[18] [19] [20]] [[23] [24] [25]]]], shape=(1, 3, 3, 1), dtype=int32)
With a kernel size of (3,3) and a stride of 1, the pooling window will move 1 unit in both the height and width directions. This will result in an output tensor with a shape of [1, 3, 3, 1], and it will contain the maximum values from each 3×3 window of the input tensor.
What is the difference between ‘SAME’ and ‘VALID’ padding in tf.nn.max_pool of tensorflow?
Padding is a technique used in convolutional neural networks (CNNs) to preserve the spatial dimensions of the input data and prevent the loss of information at the edges of the image. It involves adding additional rows and columns of pixels around the edges of the input data. There are several different ways to apply padding in Python, depending on the type of padding being used.
Let’s an image of dimension is having filtered with dimension kernel with stride then the output shape will be:
Then the output shape will be
Where, C = number of filtered channel.
For example the image of shape is filtered with kernel with stride length =2 then the output shape of the image will be.
So, the image shrinks when the convolution is performed.
And the corner and edge pixels do have not equal participation. as we can see from the figure below.
Padding
To resolve this we add extra layers of zeros to our image, which is known as the same padding.