Explore Data Characteristics
By exploring the characteristics of your information very well, you can gain treasured insights into its structure, pick out capability problems or anomalies, and inform your subsequent evaluation and modeling choices. Documenting any findings or observations from this step is critical, as they may be relevant for destiny reference or communication with stakeholders.
Let’s start by exploring the data according to the dataset. We’ll begin with Gender Diversity Analysis by looking at:
- Gender distribution across the company.
- Departments or teams with significant gender imbalances.
Gender Distribution Across the Company
We’ll calculate the proportion of each gender across the company.
Start Date is an important column for employees. However, it is not of much use if we can not handle it properly. To handle this type of data pandas provide a special function from which we can change object type to DateTime format datetime().
# Convert 'Start Date' to datetime format
df['Start Date'] = pd.to_datetime(df['Start Date'])
# Convert 'Last Login Time' to time format
df['Last Login Time'] = pd.to_datetime(df['Last Login Time']).dt.time
df.dtypes, df.head()
Output:
(First Name object
Gender object
Start Date datetime64[ns]
Last Login Time object
Salary int64
Bonus % float64
Senior Management bool
Team object
dtype: object,
First Name Gender Start Date Last Login Time Salary Bonus % \
0 Douglas Male 1993-08-06 12:42:00 97308 6.945
2 Maria Female 1993-04-23 11:17:00 130590 11.858
3 Jerry Male 2005-03-04 13:00:00 138705 9.340
4 Larry Male 1998-01-24 16:47:00 101004 1.389
5 Dennis Male 1987-04-18 01:35:00 115163 10.125
Senior Management Team
0 True Marketing
2 False Finance
3 True Finance
4 True Client Services
5 False Legal )
The gender distribution across the company is approximately 57.6% female and 42.4% male.
Teams with Significant Gender Imbalances
Next, let’s examine the gender distribution within each team to identify any significant imbalances.
# Calculate gender distribution across the company
gender_distribution = df['Gender'].value_counts(normalize=True) * 100
gender_distribution
Output:
Gender
Female 43.715239
Male 41.268076
No Gender 15.016685
Name: proportion, dtype: float64
Steps for Mastering Exploratory Data Analysis | EDA Steps
Mastering exploratory data analysis (EDA) is crucial for understanding your data, identifying patterns, and generating insights that can inform further analysis or decision-making. Data is the lifeblood of cutting-edge groups, and the capability to extract insights from records has become a crucial talent in today’s statistics-pushed world. Exploratory Data Analysis (EDA) is a powerful method that allows analysts, scientists, and researchers to gain complete knowledge of their data earlier than projecting formal modeling or speculation testing.
It is an iterative procedure that entails summarizing, visualizing, and exploring information to find patterns, anomalies, and relationships that might not be apparent at once. In this complete article, we will understand and implement critical steps for performing Exploratory Data Analysis. Here are steps to help you master EDA:
Steps for Mastering Exploratory Data Analysis
- Step 1: Understand the Problem and the Data
- Step 2: Import and Inspect the Data
- Step 3: Handling Missing Values
- Step 4: Explore Data Characteristics
- Step 5: Perform Data Transformation
- Step 6: Visualize Data Relationships
- Step 7: Handling Outliers
- Step 8: Communicate Findings and Insights