Handling Missing Data in Logistic Regression by Imputation
Imputation involves replacing missing values with estimated values. Common imputation techniques include mean imputation, median imputation, and K-nearest neighbors (KNN) imputation.
Pros of Handling Missing Data in Logistic Regression by Imputation
- Preservation of Data Integrity: Imputation retains all available data points, preventing the loss of valuable information compared to deletion methods.
- Maintenance of Sample Size: By replacing missing values with estimates, imputation ensures the dataset’s original sample size is maintained, crucial for statistical power and enhancing predictive performance.
- Bias Reduction: Imputation methods help mitigate bias in parameter estimates and standard errors by including incomplete cases, leading to more accurate and dependable model outcomes.
Cons of Handling Missing Data in Logistic Regression by Imputation
- Bias Introduction: Imputation relies on assumptions about missing data, and inaccurate assumptions may introduce bias, potentially distorting results.
- Variability Distortion: Imputation can artificially reduce observed variance if estimated values are not accurate, impacting the model’s performance.
- Complexity of Methods: Certain imputation techniques, like multiple imputation, can be computationally intensive and require careful selection and tuning, increasing the modeling process’s complexity.
Implementation
- Missing values in the dataset are imputed using the mean value of each feature.
- The SimpleImputer class from scikit-learn is used with the strategy set to ‘mean’ for imputation.
- A logistic regression model is trained on the training set with imputed missing values.
- The LogisticRegression class from scikit-learn is used for model training.
- The accuracy of the trained logistic regression model is evaluated on the testing set.
- The output displays the accuracy achieved by the logistic regression model trained using the imputation method for handling missing data.
- In this specific run, the accuracy obtained is approximately 59%.
- The achieved accuracy of approximately 59% on the testing set indicates the performance of the logistic regression model trained with imputed missing values.
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.impute import SimpleImputer
# Step 1: Generate Synthetic Dataset with Missing Values
np.random.seed(1)
n_samples = 1000
n_features = 5
X = np.random.rand(n_samples, n_features)
y = np.random.randint(0, 2, n_samples) # binary target variable
missing_mask = np.random.rand(n_samples, n_features) < 0.2 # 20% missing values
X[missing_mask] = np.nan
# Step 2: Split Data into Training and Testing Sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Imputation Method:
# Impute missing values with mean
imputer = SimpleImputer(strategy='mean')
X_train_imputed = imputer.fit_transform(X_train)
X_test_imputed = imputer.transform(X_test)
# Train logistic regression model
model_imputed = LogisticRegression()
model_imputed.fit(X_train_imputed, y_train)
# Evaluate model on test set
accuracy_imputed = model_imputed.score(X_test_imputed, y_test)
print("Accuracy with Imputation Method:", accuracy_imputed)
Output:
Accuracy with Imputation Method: 0.59
The output indicates that a logistic regression model trained using the imputation method achieved an accuracy of approximately 59%. Imputation involved replacing missing values with the mean of each feature in the training set, leading to improved performance compared to deletion methods. This accuracy suggests that imputation effectively retained valuable information and contributed to the model’s ability to predict the target variable in the testing set.
How to Handle Missing Data in Logistic Regression?
Logistic regression is a robust statistical method employed to model the likelihood of binary results. Nevertheless, real-world datasets frequently have missing values, presenting obstacles while fitting logistic regression models. Dealing with missing data effectively is essential to prevent skewed estimates and maintain the model’s accuracy. In this article, we have discussed how can we handle missing data in logistic regression.
Table of Content
- How to Handle Missing Data in Logistic Regression?
- 1. Handling Missing Data in Logistic Regression by Deletion
- 2. Handling Missing Data in Logistic Regression by Imputation
- 3. Handling Missing Data in Logistic Regression using Missingness Indicator