Exploratory analysis and data visualisation
2026-09-27
Primers on exploratory data analysis and the craft of data visualisation.
These chapters started as learning notes and primers.
Visualisation as disciplined communication: choose honest encodings, reduce clutter, use colour carefully, and let the analytical question drive the chart.
Data visualisation is where analysis meets understanding. A clear chart can surface a batch effect, a cohort imbalance, or a surprising biological pattern in seconds; a confusing chart can hide the same signal for weeks. The “art” is not decoration, it is disciplined communication.
This primer focuses on practical choices for analysts working across bioinformatics and general data work, using Australian English spelling throughout: we are talking about visualisation.
Every figure should answer a question a reader actually has. Examples:
If you cannot state the question in one sentence, pause before opening a plotting library. Visualisation without a question becomes gallery art, pleasant, not persuasive.
Match chart type to structure: distributions (histograms, ECDFs, violin/box with care), relationships (scatter), compositions (stacked bars or clear tables), and change (line charts with honest baselines).
Start axes at zero for bar charts of magnitudes. For line charts of change, truncated axes can be legitimate, but label them clearly. Show uncertainty when you have it (intervals, bootstrap bands) instead of implying false precision. Avoid dual axes that invite false alignment stories.
In omics QC, plot raw diagnostics before fancy summaries: library sizes, gene detection rates, PCA coloured by batch and biological labels. The boring plot often prevents the wrong conclusion.
Remove chartjunk, heavy gridlines, 3D effects, ornamental legends. Keep what aids reading: units, sample sizes, and the definition of each series. Direct labels often beat a distant legend when there are few categories.
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(6, 4))
ax.scatter(
df["pc1"],
df["pc2"],
c=df["batch"].astype("category").cat.codes,
alpha=0.8,
)
ax.set_xlabel("PC1")
ax.set_ylabel("PC2")
ax.set_title("PCA coloured by sequencing batch")
fig.tight_layout()Prefer an explicit colour map and legend in figures you share, the sketch above is about structure, not a finished graphic.
Use colourblind-safe palettes for anything shared widely. Reserve strong colours for the series that matter; grey out the rest. In heatmaps, diverging scales need a meaningful midpoint (often zero for centred values). Never rely on colour alone when print or projectors may flatten the story, reinforce with shape or facets.
When you have several related views (per-chromosome, per-assay, per-site), facet them. Humans compare aligned small charts better than one chart with eight overlapping encodings. Consistency of scale across facets is a feature, not a limitation.
Sometimes the best “plot” is a tidy table with sorted rows, clear units, and a handful of well-chosen summary columns. Dashboards that force every metric into a sparkline often make comparison harder, not easier.
Captions should say what to notice, not merely repeat axis labels. “PC1 separates library prep batches more than genotype” is a caption; “PCA plot” is a missed opportunity.
Great visualisation is empathy plus accuracy: you encode data so another person can see what you saw, or politely disagree with evidence in view. Master a small set of honest chart forms, label generously, and let the question drive the design. The art follows from that craft.
A practical EDA sequence for analysts and bioinformaticians, inventory, quality, distributions, relationships, high-dimensional structure, and documented decisions.
Exploratory data analysis (EDA) is the disciplined habit of looking before you model. In bioinformatics and applied analytics, EDA is how you discover batch effects, mislabelled samples, skewed distributions, and the shape of relationships that should inform, or forbid, a particular method. Skipping it is how elegant models fail on contact with reality.
EDA answers questions like:
It is not a substitute for confirmatory analysis. It is the map you draw so confirmatory work asks fair questions.
Before plotting expression or sensor streams, confirm that every measurement column matches a metadata row. Check ID formats, rename maps, and one-to-one expectations. Many “biological” PCA separations are actually swapped labels or mixed genome builds.
assert set(counts.columns) == set(meta["sample_id"])
meta = meta.set_index("sample_id").loc[counts.columns]Assertions like these are EDA infrastructure, cheap and invaluable.
Summary tables hide multimodality. Plot library sizes, gene-wise detection rates, ages, doses, or whatever drives your domain. Ask whether a log transform is needed, whether zeros are structural, and whether a few samples dominate totals.
For counts data, mean-variance trends and filter effects deserve a look before differential testing. For tabular business or field data, histograms and ECDFs often beat a single KPI tile.
Plot missingness by column and by row. Is it random, block-wise (an assay failed), or informative (not measured below a limit)? The mechanism changes whether imputation, exclusion, or explicit “missing” categories are appropriate. Do not silently drop rows in a pipeline without recording how many disappeared and why.
Scatterplots, box plots by group, and simple correlation matrices are useful, and easy to over-interpret. Correlation is not causation; confounding is common in observational biological and agricultural data. Use EDA to generate hypotheses and to spot confounders (batch, site, depth, age) that belong in later models.
For omics matrices, PCA or similar embeddings coloured by technical and biological factors are standard EDA. Look at scree plots: if early components track plate or prep date more than condition, address that before celebrating a volcano plot. Complementary views, hierarchical clustering of samples, QC metric overlays, help confirm the story.
# Sketch: standardise and compute PCA on a feature matrix X
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
X_std = StandardScaler().fit_transform(X)
pcs = PCA(n_components=10).fit_transform(X_std)EDA that lives only in scrollback is wasted. Keep a short markdown or notebook section: filters applied, samples removed, oddities pending follow-up. That log becomes methods text and protects you when a result needs auditing.
EDA can expand forever. Set a goal (“decide whether batch correction is needed”, “confirm cohort eligibility”) and a time box. When the goal is met, write the decision and move to confirmatory analysis. You can always return with a sharper question.
Good EDA is curious, sceptical, and documented. It privileges plots and checks that could change your mind. Build a repeatable personal checklist for each new dataset, inventory, quality, distributions, relationships, structure, notes, and you will catch more truth earlier, with fewer heroic debugging sessions later.