How to Create a Custom Loss Function in Keras
Creating a custom loss function in Keras is crucial for optimizing deep learning models. The article aims to learn how to create a custom loss function.
Need to create Custom Loss Functions
Loss function is considered as a fundamental component of deep learning as it is helpful in error minimization. Loss is computed by comparing predicted values and actual values for a given set of inputs. Loss functions vary depending on the task. The need to create custom loss functions is discussed below:
- The loss functions vary depending on the machine learning task, there might be some cases where the standard loss functions provided by Keras might not be suitable for a given assignment. This issue can be addressed by introducing custom loss functions.
- These custom loss functions are helpful not only when standard loss functions are inadequate but also when data is imbalanced. Class imbalance can be addressed by employing a custom loss function when the dataset is extremely imbalanced (one class is significantly more abundant than others).
- By assigning minority classes greater weight, custom loss functions can avoid bias in the model’s favour of the dominant class. These custom loss functions can be implemented with Keras.
Creating a Custom Loss Function in Keras
Step 1: Import the necessary libraries
In this step, we import TensorFlow and Keras libraries along with NumPy for numerical operations. We also import necessary modules like Sequential
for creating the model, Dense
for defining layers, and K
from keras.backend
for backend operations.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from keras import backend as K
import numpy as np
from tensorflow.python.ops import math_ops
Step 2: Define Custom Loss Function
Here I have defined a custom loss function which calculates mean of absolute difference of squares of true value and the predicted value.
def customloss(Y_actual, Y_predicted):
absolute_diff = abs((Y_predicted*Y_predicted) - (Y_actual*Y_actual)) #squared difference
final_loss =(K.mean(absolute_diff, axis=-1))/100 #mean over last dimension
return final_loss
Step 3: Create the model using Keras.
Using sequential model from the Keras API model is created using Rectified Linear Unit as activation function with two dense layers.
model = keras.Sequential([
keras.layers.Dense(10, activation='relu', input_shape=(1,)),
keras.layers.Dense(1)
])
Step 4: Compiling the model with custom loss.
Created model is compiled for custom loss function and the optimizer used in ‘adam’.
model.compile(loss=customloss, optimizer='adam')
Step 5: Fitting the Model
Input data (X_train
and Y_train
) is created in list format and then converted to NumPy arrays. The model is trained using the fit()
function for 5 epochs with a batch size of 3.
a=[[12.3],[20.0], [17.6],[15.0],[20.0],[7.5],[5.9], [20.0]]
b=[6.0, 2, 18,24,30, 3,6, 12]
X_train = np.array(a)
Y_train = np.array(b) #dummy data
model.fit(X_train, Y_train, batch_size=3, epochs=5)
Complete Code to create a Custom Loss Function in Keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from keras import backend as K
import numpy as np
from tensorflow.python.ops import math_ops
def customloss(Y_actual, Y_predicted):
absolute_diff = abs((Y_predicted*Y_predicted) - (Y_actual*Y_actual)) #squared difference
final_loss =(K.mean(absolute_diff, axis=-1))/100 #mean over last dimension
return final_loss
model = keras.Sequential([
keras.layers.Dense(10, activation='relu', input_shape=(1,)),
keras.layers.Dense(1)
])
model.compile(loss=customloss, optimizer='adam')
a=[[12.3],[20.0], [17.6],[15.0],[20.0],[7.5],[5.9], [20.0]]
b=[6.0, 2, 18,24,30, 3,6, 12]
X_train = np.array(a)
Y_train = np.array(b) #dummy data
model.fit(X_train, Y_train, batch_size=3, epochs=5)
Output:
Epoch 1/5
3/3 [==============================] - 1s 23ms/step - loss: 2.4541
Epoch 2/5
3/3 [==============================] - 0s 13ms/step - loss: 2.4497
Epoch 3/5
3/3 [==============================] - 0s 8ms/step - loss: 2.4474
Epoch 4/5
3/3 [==============================] - 0s 10ms/step - loss: 2.4445
Epoch 5/5
3/3 [==============================] - 0s 10ms/step - loss: 2.4402
The output displayed shows the loss calculated in each epoch using the custom loss function defined. Each epoch’s loss is printed, indicating how well the model is optimizing towards the desired outcome.