Exploratory Data Analysis (EDA)

What Is Exploratory Data Analysis?

Exploratory Data Analysis (EDA) is the critical process of investigating a dataset before formal modeling or reporting. Introduced by statistician John Tukey in the 1970s, EDA is fundamentally about asking questions of your data — using statistical summaries, visualizations, and transformations to understand what the data looks like, what it contains, and what stories it might tell.

Unlike confirmatory analysis, which tests predefined hypotheses, EDA is open-ended. You follow the data wherever it leads, letting unexpected patterns and anomalies guide your questions. Done well, EDA prevents you from building models on flawed assumptions and ensures your analyses are grounded in reality.

The Goals of EDA

EDA serves several interconnected purposes. First, it helps you understand the structure and content of your data — how many rows and columns exist, what data types are present, and what each variable represents. Second, it surfaces data quality issues like missing values, duplicates, and inconsistencies that need to be addressed before analysis. Third, it reveals distributions, relationships, and patterns that inform feature engineering, model selection, and hypothesis generation. Finally, EDA exposes outliers and anomalies that might represent errors or genuinely interesting edge cases.

Starting with Summary Statistics

Every EDA begins with basic summary statistics. For numerical columns, this means computing the mean, median, standard deviation, minimum, maximum, and percentiles. These numbers give you an immediate sense of the scale and spread of your data. A variable with a mean of 50 and a standard deviation of 2 behaves very differently from one with the same mean but a standard deviation of 200.

For categorical columns, frequency counts and unique value counts are the equivalent. How many distinct categories are there? Are some categories vastly more common than others? Are there any unexpected values or obvious typos in the category labels?

In Python, a single call to df.describe() in pandas gives a comprehensive summary table for all numeric columns. Calling it with include='all' extends the summary to categorical columns. These few lines often reveal more about a dataset than hours of manual inspection.

Univariate Analysis: Understanding Each Variable

Univariate analysis examines one variable at a time. For continuous numeric variables, histograms reveal the shape of the distribution — whether it's symmetric and bell-shaped, skewed left or right, bimodal, or uniform. Understanding the distribution matters because many statistical tests and machine learning algorithms assume normally distributed inputs.

Box plots are another essential tool for univariate analysis. They simultaneously display the median, interquartile range, and outliers, making them ideal for comparing distributions across groups. A box plot of salaries by department immediately highlights which teams earn more and how variable compensation is within each group.

For categorical variables, bar charts showing the frequency of each category are the standard approach. Be alert to severe imbalances — if 95% of records belong to one category, your model might achieve high accuracy simply by always predicting the majority class.

Bivariate Analysis: Exploring Relationships

Bivariate analysis examines how two variables relate to each other. Scatter plots are the primary tool for exploring relationships between two continuous variables. A tight linear pattern suggests a strong relationship; a scattered cloud suggests weak or no relationship; a curved pattern suggests a non-linear relationship.

Correlation coefficients quantify the strength and direction of linear relationships. The Pearson correlation ranges from -1 (perfect negative linear relationship) to +1 (perfect positive linear relationship), with 0 indicating no linear relationship. Computing a correlation matrix for all numeric variables and visualizing it as a heatmap gives a rapid overview of which variables tend to move together.

When one variable is categorical and the other continuous, grouped box plots or violin plots effectively show how the distribution of the continuous variable differs across categories. When both variables are categorical, a cross-tabulation (confusion matrix style) or grouped bar chart works well.

Multivariate Analysis

Multivariate analysis examines three or more variables simultaneously. Pair plots (also called scatterplot matrices) show scatter plots for every combination of numeric variables in a single grid, providing a comprehensive overview of all pairwise relationships at once. They're particularly useful when you have 5–10 variables and want to quickly scan for patterns.

Color encoding can add a third dimension to scatter plots — for example, coloring points by a categorical variable to see whether groups cluster differently. Dimensionality reduction techniques like PCA (Principal Component Analysis) or UMAP reduce high-dimensional datasets to 2D for visualization, revealing cluster structure that isn't visible in individual variable plots.

Identifying and Investigating Outliers

Outliers warrant special attention in EDA. They might indicate data entry errors (a customer age of 350), measurement anomalies, or genuinely extreme but valid observations (a viral social media post with 10 million views in a dataset of posts with typical engagement around 1,000).