Custom scorer for a multi-class classification problem

Custom scorer for a multi-class Regression problem

Steps:

Import the necessary libraries
Load the iris dataset
Define multiple metrics like accuracy_score, precision_score, recall_score, f1_score with make_scorer.
Create a XGBClassifier model
Evaluate the model using cross-validation and the custom scorer
Print the mean scores for each metric

Python

from sklearn.metrics import make_scorer, accuracy_score
from sklearn.metrics import precision_score, recall_score, f1_score
from sklearn.model_selection import cross_validate
from sklearn.datasets import load_iris
from xgboost import XGBClassifier
 
 
# Load the iris dataset
iris = load_iris()
 
# Define multiple metrics
scoring = {'accuracy': make_scorer(accuracy_score),
           'precision': make_scorer(precision_score, average='macro'),
           'recall': make_scorer(recall_score, average='macro'),
           'f1-score': make_scorer(f1_score, average='macro')
          }
 
# Create a XGBClassifier
clf = XGBClassifier(n_estimators=2, 
                    max_depth=3, 
                    learning_rate=0.1)
 
# Evaluate the model using cross-validation and the custom scorer
scores = cross_validate(clf, iris.data, iris.target, cv=5, scoring=scoring)
 
# Print the mean scores for each metric
print("Accuracy mean score:", scores['test_accuracy'].mean())
print("Precision mean score:", scores['test_precision'].mean())
print("Recall mean score:", scores['test_recall'].mean())
print("f1-score:", scores['test_f1-score'].mean())

Output:

Accuracy mean score: 0.9666666666666668
Precision mean score: 0.9707070707070707
Recall mean score: 0.9666666666666668
f1-score: 0.9664818612187034

How To Create/Customize Your Own Scorer Function In Scikit-Learn?

A well-known Python machine learning toolkit called Scikit-learn provides a variety of machine learning tools and methods to assist programmers in creating sophisticated machine learning models. A strong framework for assessing the effectiveness of these models using a variety of metrics and scoring functions is also offered by Scikit-learn. To assess the effectiveness of their models, users might want to design their scoring function in specific circumstances. Scikit-learn makes this possible, and in this article, we’ll go over how to design and tweak your very own scoring function.

A scikit-learn function called a scorer accepts two arguments: the ground truth (actual values) and the model’s predicted values. A single score that evaluates the accuracy of the anticipated values is returned by the function. Accuracy, precision, recall, F1-score, and other predefined scoring functions are available in Scikit-learn. To assess the effectiveness of their models, users might want to develop their unique scoring system.

Tags:

#python #AI-ML-DS #Machine Learning #Machine Learning #python

Custom scorer for a multi-class Regression problem

Custom scorer for a multi-class classification problem

Steps:

Python

How To Create/Customize Your Own Scorer Function In Scikit-Learn?

Similar Reads