What Is Standardization in Machine Learning?
- 1 day ago
- 34 min read
Two features on wildly different scales — age in years and income in dollars — can quietly wreck a model before training even starts, because many algorithms treat "bigger numbers" as "more important" even when that's mathematically false; standardization is the fix that most working machine learning practitioners reach for first, and understanding exactly what it does — and does not — do is the difference between a model that converges cleanly and one that silently underperforms.
TL;DR
Standardization rescales a numeric feature to have a mean of 0 and a standard deviation of 1, using the z-score formula z = (x − μ) / σ.
It does not make data normally distributed, does not remove outliers, and does not guarantee better accuracy — it only changes scale.
Distance-based, gradient-based, and regularized models (KNN, SVM, logistic regression, neural networks) usually benefit; tree-based models (decision trees, random forests, gradient boosting) usually do not need it.
The scaler must be fit only on training data, then used to transform validation, test, and production data — fitting on the full dataset causes data leakage.
Outliers distort the mean and standard deviation, so heavily skewed data often calls for RobustScaler, a log transform, or a Yeo-Johnson power transform instead.
scikit-learn's StandardScaler (with fit, transform, and inverse_transform) is the standard tool, and it should live inside a Pipeline so cross-validation and hyperparameter tuning stay leakage-safe.
What Is Standardization in Machine Learning?
Standardization in machine learning is a preprocessing step that rescales a numeric feature by subtracting its mean and dividing by its standard deviation, producing a z-score with a mean of 0 and a standard deviation of 1. It puts features on comparable scales without changing their shape, ordering, or relationships to one another.
Table of Contents
Plain-English Definition of Standardization
Standardization takes one numeric column of data and rewrites every value in terms of how many standard deviations it sits from the column's mean. A value above the mean becomes a positive number; a value below the mean becomes negative; a value equal to the mean becomes exactly zero. Nothing about the underlying pattern in the data changes — only the ruler used to measure it changes.
This matters because raw features rarely arrive on comparable scales. A dataset might have "age" in years (roughly 0–100) sitting next to "annual income" in dollars (roughly 20,000–500,000). Any algorithm that computes distances, dot products, or gradients across these two columns will let income dominate simply because its numbers are bigger — not because income is more predictive. Standardization equalizes the playing field by converting every feature into the same unit: standard deviations from its own mean.
In the feature space that a model actually learns in, this rescaling can reshape decision boundaries and convergence paths dramatically, even though the raw information content of the data hasn't changed at all.
The Mathematical Formula Behind the Z-Score
The standard score, or z-score, of an observation x is defined as:
z = (x − μ) / σ
Where:
x is the original value of the feature for one observation.
μ (mu) is the mean of that feature, calculated across the training data.
σ (sigma) is the standard deviation of that feature, also calculated across the training data.
z is the standardized value: a unitless number expressing distance from the mean in standard-deviation units.
The transformation happens in two steps. First, mean centering subtracts μ from every value, shifting the distribution so its center sits at zero. Second, variance scaling divides by σ, compressing or stretching the distribution so its spread equals one. After both steps, the transformed feature has a mean of (approximately) 0 and a standard deviation of (approximately) 1 — scikit-learn's documentation describes this directly: the standard score of a sample is calculated as z = (x − u) / s, where u is the mean of the training samples and s is the standard deviation of the training samples (scikit-learn StandardScaler documentation, 2026).
Population vs. sample standard deviation
Statisticians distinguish between the population standard deviation (dividing by N) and the sample standard deviation (dividing by N − 1, called Bessel's correction). This N − 1 divisor is controlled by a "degrees of freedom" parameter, commonly written ddof. NumPy's std function defaults to ddof=0, the population convention, while pandas defaults to ddof=1, the sample convention (NumPy documentation states the divisor used in calculations is N − ddof, and by default ddof is zero, corresponding to the population standard deviation). scikit-learn's StandardScaler also uses a biased estimator equivalent to numpy.std(x, ddof=0), and the library notes that this choice is unlikely to affect model performance in practice.
A critical clarification: standardizing a feature does not make it normally distributed. If a feature is right-skewed before standardization, it remains right-skewed afterward — only its scale changes, not its shape. The phrase "standard normal distribution" refers to a specific bell-curve distribution with mean 0 and variance 1; "standardized data" simply means data that has been rescaled to have mean 0 and variance 1, regardless of its actual shape. These two concepts are often confused, but they are not the same thing.
A Small Worked Example
Consider a tiny feature: the ages of five people in a training set.
Original values: 22, 25, 29, 31, 43
Step 1 — Calculate the mean: μ = (22 + 25 + 29 + 31 + 43) / 5 = 150 / 5 = 30
Step 2 — Calculate the standard deviation (using the population convention, ddof = 0):
Value | Deviation from mean | Squared deviation |
22 | −8 | 64 |
25 | −5 | 25 |
29 | −1 | 1 |
31 | 1 | 1 |
43 | 13 | 169 |
Sum of squared deviations = 260. Variance = 260 / 5 = 52. Standard deviation σ = √52 ≈ 7.211 (rounded to three decimals).
Step 3 — Calculate each z-score using z = (x − 30) / 7.211:
Original value (x) | z-score |
22 | (22 − 30) / 7.211 ≈ −1.109 |
25 | (25 − 30) / 7.211 ≈ −0.693 |
29 | (29 − 30) / 7.211 ≈ −0.139 |
31 | (31 − 30) / 7.211 ≈ 0.139 |
43 | (43 − 30) / 7.211 ≈ 1.803 |
Interpretation: The person aged 22 has a z-score of about −1.11, meaning their age sits roughly 1.11 standard deviations below the group mean of 30. The person aged 43 has a z-score of about 1.80, meaning their age is about 1.80 standard deviations above the mean — the largest outlier in this small sample, but not extreme enough to be flagged as a statistical anomaly on its own. Note that the ordering of the five people hasn't changed; only the numbers used to describe their relative position have.
What Changes and What Stays the Same
Standardization is a linear transformation, so it changes some properties of the data and leaves others untouched.
What changes:
Units — the feature loses its original unit (years, dollars, kilometers) and becomes a unitless z-score.
Mean — shifts to (approximately) zero.
Variance/standard deviation — rescales to (approximately) one.
Absolute distances between points — these change in magnitude, since every value is divided by σ.
Model coefficients — in linear models, coefficients now represent the effect of a one-standard-deviation change, not a one-unit change.
What stays the same:
Relative ordering — the smallest value is still the smallest, the largest is still the largest.
Distribution shape — skewness, kurtosis, and multimodality are preserved exactly.
Outliers — a value that was far from the mean before standardization is still far from the mean afterward (in relative terms).
Correlations between features — Pearson correlation coefficients are unaffected by standardization, because correlation itself is scale-invariant.
A common misconception is that standardization "fixes" outliers or skew. It does not. A value 5 standard deviations above the mean is still 5 standard deviations above the mean after transformation — the number itself is unchanged in relative terms; only the units it's expressed in have changed.
Why Feature Scale Matters to Algorithms
Imagine a dataset with two features: age (measured in years, ranging roughly 18–90) and annual income (measured in dollars, ranging roughly 20,000–500,000). If you compute the Euclidean distance between two people using raw values, the income difference — which might be tens of thousands — completely swamps the age difference, which might be a handful of years. The algorithm effectively ignores age, not because age is unimportant, but because its numeric scale is smaller.
This scale sensitivity shows up in several mathematical operations common to machine learning:
Distance calculations — k-nearest neighbors and k-means clustering rely directly on Euclidean or Manhattan distance, so any feature with a larger numeric range dominates the distance metric.
Dot products and kernels — support vector machines using the RBF kernel, and any kernel method computing similarity via dot products, implicitly assume features are on comparable scales, since the objective function of a learning algorithm — such as the RBF kernel of SVMs or the L1 and L2 regularizers of linear models — assumes features are centered around zero with similar variance (scikit-learn StandardScaler documentation).
Gradient-based optimization — logistic regression, linear regression, and neural networks trained with gradient descent converge faster and more reliably when features share a similar scale, because unscaled features create elongated, poorly conditioned loss surfaces that slow convergence.
Regularization penalties — L1 (Lasso) and L2 (Ridge) penalties apply the same shrinkage strength to every coefficient; if features are on different scales, the penalty affects them unevenly, disproportionately shrinking coefficients tied to naturally larger-scaled features.
Principal component analysis — PCA seeks directions of maximum variance, and scikit-learn's own documentation shows that if one feature such as height varies less than another such as weight purely because of their measurement units, PCA may incorrectly determine that the direction of maximum variance aligns with the larger-scaled feature (scikit-learn's Importance of Feature Scaling example).
Numerical conditioning — badly scaled inputs can produce ill-conditioned matrices in linear algebra operations, leading to slower convergence and, in extreme cases, numerical instability.
Which Algorithms Usually Benefit From Standardization
Algorithm / model family | Usually helped by standardization? | Why | Important caveat |
K-nearest neighbors | Yes | Distance calculations are scale-sensitive | Choice of distance metric still matters |
K-means clustering | Yes | Cluster assignment relies on Euclidean distance | Sensitive to outliers distorting centroids |
Support vector machines | Yes | RBF and other kernels assume comparable feature scales | Linear kernels on already-comparable data are less affected |
Logistic regression | Yes | Gradient descent converges faster; L1/L2 penalties act fairly | Effect depends on solver and regularization strength |
Linear/Ridge/Lasso/Elastic Net regression | Yes | Regularization penalizes coefficients uniformly only when scales match | Unregularized ordinary least squares is scale-invariant in predictions, though coefficient interpretation still benefits |
Principal component analysis | Yes | Variance-maximizing directions are scale-dependent | Not always desired if original units are meaningful |
Neural networks | Yes | Gradient-based optimizers converge faster with normalized inputs | Architecture-specific normalization layers may reduce this need |
Gaussian processes / kernel methods | Yes | Kernel similarity functions assume comparable scales | Depends on kernel hyperparameters and length-scale tuning |
Across all of these, the improvement is real but not universal — the exact effect depends on the data, the optimizer, the regularization strength, and the specific implementation. Standardization is a rule of thumb for these families, not a guarantee of better performance.
Which Algorithms Are Usually Less Sensitive
Algorithm / model family | Usually helped by standardization? | Why | Important caveat |
Decision trees | Generally no | Splits are based on ordering, not absolute scale | Still needed if a tree feeds into a distance-based or linear component |
Random forests | Generally no | Same split-based logic as individual trees | Feature importance metrics can still be affected by scale of categorical encodings |
Extremely randomized trees | Generally no | Random split thresholds are also scale-invariant | Numerical precision issues can arise with extreme value ranges |
Gradient-boosted decision trees | Generally no | Boosting still relies on tree splits at each stage | Hybrid pipelines (e.g., trees plus a linear stacking layer) may still need scaling |
Split-based tree models evaluate thresholds like "is feature X greater than value V?" A monotonic rescaling — such as standardization — does not change which observations fall on each side of that threshold, so the resulting splits, and therefore the entire tree structure, are unaffected. This is why decision trees, random forests, and gradient-boosted trees are commonly described as scale-invariant for individual features.
That said, "usually insensitive" is not the same as "never useful." If a pipeline combines a tree-based model with a distance-based preprocessing step (like a PCA transform or a nearest-neighbor feature), or if the trees feed into a downstream linear or neural component in an ensemble, scaling can still matter somewhere in that pipeline — just not inside the tree-splitting logic itself.
Standardization vs. Normalization vs. Other Scalers
"Normalization" is one of the most overloaded terms in data science. Depending on context, it can mean min-max scaling to a fixed range, scaling individual samples to unit norm, general feature scaling, or the normalization layers used inside neural networks. This ambiguity is a frequent source of confusion, so it's worth being explicit about which technique is being discussed.
Method | Statistic used | Typical output range | Sensitive to outliers? | When it's appropriate |
Standardization (z-score) | Mean, standard deviation | Unbounded, centered at 0 | Yes — mean and σ shift with extreme values | Distance-, gradient-, and regularization-based models on roughly symmetric data |
Min-max scaling | Minimum, maximum | Fixed range, typically [0, 1] | Very — a single extreme value stretches the whole range | Bounded-input models (e.g., some neural network activations); features with a known fixed range |
Max-absolute scaling | Maximum absolute value | [-1, 1] | Yes | Sparse data where preserving zero entries matters |
Robust scaling | Median, interquartile range (IQR) | Unbounded, centered at median | No — designed specifically to resist outliers | Data with known outliers or heavy tails |
Unit-norm (vector) normalization | L1 or L2 norm of each sample (row) | Each sample vector has length 1 | Depends on norm definition | Text vectors, embeddings, and cases where direction matters more than magnitude |
Quantile transformation | Empirical quantiles (rank order) | Uniform or approximately Gaussian output | No — designed to be rank-based | Highly skewed data where a bounded, robust transform is acceptable |
Power transformation (Box-Cox / Yeo-Johnson) | Estimated power parameter (lambda) | Approximately Gaussian-shaped | Moderate | Reducing skewness and stabilizing variance before modeling |
MinMaxScaler does not reduce the effect of outliers; it linearly scales all values into the target range, so the largest value maps to the maximum and the smallest maps to the minimum, meaning a single extreme value stretches or compresses everything else (scikit-learn's MinMaxScaler documentation). MaxAbsScaler behaves similarly to MinMaxScaler but maps data based on the largest absolute value present, and because it does not correct for extreme values either, it also suffers when large outliers are present (scikit-learn's scaler comparison example).
RobustScaler is deliberately different: its centering and scaling statistics are based on percentiles — the median and the interquartile range — so a small number of very large marginal outliers do not influence the resulting transformation the way they influence StandardScaler or MinMaxScaler (scikit-learn's scaler comparison example). QuantileTransformer takes a related but distinct approach: it is also robust to outliers in the sense that adding or removing outliers in the training set yields approximately the same transformation, but unlike RobustScaler, it automatically collapses any outlier to the pre-defined range boundaries, which can create saturation artifacts for extreme values (scikit-learn documentation).
Power transformations sit in a slightly different category: rather than just rescaling, they reshape the distribution itself. PowerTransformer supports both the Box-Cox transform, which requires strictly positive input, and the Yeo-Johnson transform, which supports both positive and negative values; both estimate an optimal parameter through maximum likelihood to reduce skewness and stabilize variance, and by default the output is also standardized to zero mean and unit variance (scikit-learn's PowerTransformer documentation). These methods trace back to two well-known statistical papers: Yeo and Johnson's 2000 paper in Biometrika introducing the Yeo-Johnson family, and Box and Cox's foundational 1964 paper in the Journal of the Royal Statistical Society (as cited in scikit-learn's PowerTransformer references).
No single method here is universally best — the right choice depends on the data's shape, the presence of outliers, whether negative values exist, and the requirements of the downstream algorithm.
Outliers, Skew, and When to Choose a Different Scaler
The mean and standard deviation are both highly sensitive to extreme values. A single very large observation can pull the mean upward and inflate the standard deviation, which then compresses the z-scores of every "normal" observation into a narrower band near zero. This is exactly the pattern shown in scikit-learn's own comparison using the California housing dataset, where median income and average house occupancy — both containing very large outliers — produce standardized values in which most points crowd into a small range while a handful of outliers dominate the tails (scikit-learn, Compare the effect of different scalers on data with outliers).
When outliers or heavy skew are present, several alternatives are worth considering:
RobustScaler — replaces mean and standard deviation with median and interquartile range, so extreme values don't distort the transformation for the rest of the data.
Log transformation — compresses large values more than small ones, useful for right-skewed, strictly positive data such as income or transaction amounts.
Yeo-Johnson or Box-Cox transforms — statistically estimate the best power transform to reduce skew, with Yeo-Johnson supporting negative values and Box-Cox requiring strictly positive data.
Quantile transformation — maps data to a uniform or approximately Gaussian distribution based on rank, which is very robust to outliers but can distort relative spacing between values.
Winsorization or domain-informed capping — manually limiting extreme values based on domain knowledge, used cautiously.
A crucial caution: outliers should never be deleted or capped automatically without investigation. An extreme value might be a data-entry error, a rare but entirely valid event (a legitimate high-value transaction), or a meaningful domain signal (a fraud case, a medical emergency). Scaling addresses the numerical effect of outliers on a model; it does not answer the substantive question of whether that data point should be in the dataset at all. Those are separate decisions, and conflating them can quietly remove exactly the cases a model most needs to learn from.
Preventing Data Leakage in Train-Test Splits
This is the single most important operational rule in this entire article: fit the scaler only on the training data.
The correct sequence is:
Split the data into training, validation, and test sets first.
Fit the scaler (compute mean and standard deviation) using only the training partition.
Transform the training data using those training-derived statistics.
Transform the validation and test partitions using the same stored training statistics — never refit on them.
Reuse that exact fitted scaler object at inference time in production.
Fitting a scaler on the entire dataset before splitting leaks information from the validation or test set into the training process, because the mean and standard deviation used to scale the training data would then be partly informed by data the model is supposed to never have seen. scikit-learn's documentation is explicit about this risk: incorporating statistics from test data into preprocessors makes cross-validation scores unreliable — this is precisely what is meant by data leakage — and this applies to scalers and imputers alike (scikit-learn, Pipelines and composite estimators). The broader guidance from the same documentation states the general rule plainly: test data should never be used to make choices about the model, and while both train and test subsets should receive the same preprocessing transformation, it is important that the transformation itself is learned only from the training data (scikit-learn, Common pitfalls and recommended practices).
This is not a criticism of calling fit_transform — that method is entirely appropriate on the training partition. The mistake is calling fit or fit_transform on validation or test data, or on the combined dataset before splitting.
Leakage risk extends beyond a simple holdout split:
Cross-validation — if a scaler is fit before the folds are created, every fold's "training" portion has already seen statistics derived from its own held-out portion.
Hyperparameter tuning — grid search or randomized search that fits preprocessing outside the search loop repeats the same leakage across every candidate configuration. See our guide on hyperparameter tuning for more on tuning workflows.
Nested cross-validation — leakage can occur at either the inner or outer loop if preprocessing sits outside the fold boundary.
Time-series validation — scaling statistics computed from future observations leak information backward in time, violating the temporal ordering the model is meant to respect.
Grouped validation — when data is grouped (e.g., by patient or by customer), leakage can occur if scaling statistics mix information across groups that should remain separate.
The safest default is to wrap the scaler inside a scikit-learn Pipeline. Pipelines fix this by construction: the same samples are used to train the transformers and predictors, so the pipeline can be handed directly to cross-validation and hyperparameter-search functions without any risk of the fold boundary being crossed (scikit-learn, Pipelines and composite estimators). A pipeline object also handles the fit/transform boundary automatically for each cross-validation fold: preprocessing is learned only from the training data within each fold (scikit-learn pipeline and cross-validation guidance).
Python Implementation Walkthrough
The following examples use current, stable scikit-learn and NumPy APIs.
Example A: Manual NumPy implementation
import numpy as np
# A small numeric feature (e.g., ages)
X = np.array([22, 25, 29, 31, 43], dtype=float)
# Compute mean and population standard deviation (ddof=0)
mean = np.mean(X)
std = np.std(X, ddof=0)
# Guard against division by zero for constant features
if std == 0:
z_scores = np.zeros_like(X)
else:
z_scores = (X - mean) / std
print("Mean:", mean)
print("Std:", std)
print("Z-scores:", z_scores)This mirrors the worked example above and shows explicitly why a zero-standard-deviation check matters: dividing by zero for a constant feature would otherwise raise a runtime warning or produce inf/NaN values.
Example B: Basic scikit-learn StandardScaler
import numpy as np
from sklearn.preprocessing import StandardScaler
X = np.array([[22.0], [25.0], [29.0], [31.0], [43.0]])
scaler = StandardScaler()
scaler.fit(X) # learn mean_ and scale_ (std) from this data
X_scaled = scaler.transform(X)
print("Learned mean:", scaler.mean_)
print("Learned scale (std):", scaler.scale_)
print("Scaled values:\n", X_scaled)
# fit_transform combines both steps on the SAME data
X_scaled_again = scaler.fit_transform(X)
# inverse_transform recovers the original scale
X_recovered = scaler.inverse_transform(X_scaled)
print("Recovered original values:\n", X_recovered)StandardScaler() stores the fitted statistics as mean_ and scale_ after calling fit, and both transform and inverse_transform use those stored values (scikit-learn StandardScaler documentation).
Example C: Correct train-test workflow
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
# Assume X (features) and y (labels) already exist as NumPy arrays
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # fit ONLY on training data
X_test_scaled = scaler.transform(X_test) # reuse training statistics
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train_scaled, y_train)
accuracy = model.score(X_test_scaled, y_test)
print("Test accuracy:", accuracy)Note that X_test is transformed, never fit — this is the leakage-safe pattern described in the previous section.
Example D: Pipeline-based workflow
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score, GridSearchCV
pipeline = make_pipeline(
StandardScaler(),
SVC(kernel="rbf", random_state=42)
)
# Cross-validation: the scaler is refit inside every fold automatically
scores = cross_val_score(pipeline, X_train, y_train, cv=5)
print("Cross-validation scores:", scores)
# Hyperparameter tuning stays leakage-safe as well
param_grid = {"svc__C": [0.1, 1, 10], "svc__gamma": ["scale", "auto"]}
grid_search = GridSearchCV(pipeline, param_grid, cv=5)
grid_search.fit(X_train, y_train)
print("Best parameters:", grid_search.best_params_)Because the scaler and the model are bundled into one pipeline object, cross_val_score and GridSearchCV both refit the scaler from scratch on each fold's training portion — exactly the pattern scikit-learn recommends for avoiding leakage during cross-validation and hyperparameter search (scikit-learn pipelines and cross-validation).
Example E: ColumnTransformer for mixed data
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
numeric_features = ["age", "income"]
categorical_features = ["region"]
numeric_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler())
])
categorical_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="most_frequent")),
("encoder", OneHotEncoder(handle_unknown="ignore"))
])
preprocessor = ColumnTransformer(transformers=[
("num", numeric_transformer, numeric_features),
("cat", categorical_transformer, categorical_features)
])
full_pipeline = Pipeline(steps=[
("preprocessor", preprocessor),
("classifier", LogisticRegression(max_iter=1000))
])
# full_pipeline.fit(X_train, y_train) applies each transformer to its own columnsColumnTransformer allows different transformations to be applied to different groups of columns within a single pipeline that remains safe from data leakage (scikit-learn, Pipelines and composite estimators).
Handling Mixed Data Types
Real-world tabular datasets rarely consist of a single, clean numeric type. A typical table might contain continuous measurements, integer counts, binary flags, ordinal ratings, nominal categories, one-hot encoded columns, dates, free text, and sparse indicator matrices — all in the same table. Applying StandardScaler blindly to every column is a common and consequential mistake.
Continuous numerical features (age, income, temperature) are the clearest candidates for standardization.
Count features (number of purchases, number of visits) can often be standardized, though heavy skew may call for a log or power transform first.
Binary indicators (0/1 flags) are already on a comparable scale and rarely need standardization; scaling them can actually reduce interpretability without adding value.
Ordinal features (low/medium/high encoded as 1/2/3) can sometimes be standardized, but the decision depends on whether the model treats the encoding as genuinely numeric or as a proxy for category identity.
Nominal categorical features should generally be encoded (e.g., one-hot) rather than standardized.
One-hot encoded columns are a nuanced case: some practitioners standardize them anyway for consistency inside regularized linear models, while others leave them untouched because they are already bounded between 0 and 1. Neither choice is universally "correct" — it depends on the regularization scheme and how the model is evaluated.
Dates or derived time features typically need feature engineering (day-of-week, days-since-event) before scaling makes sense.
Text features represented as bag-of-words or TF-IDF vectors require fundamentally different treatment, discussed below.
The ColumnTransformer shown in Example E is the standard tool for applying standardization selectively — routing numeric columns to a scaler while routing categorical columns to an encoder, all within one leakage-safe pipeline.
Sparse Matrices
Many real datasets — especially those derived from one-hot encoding or bag-of-words text representations — are stored as sparse matrices, where the overwhelming majority of entries are zero. Mean-centering a sparse matrix is problematic because subtracting a non-zero mean from every entry, including the millions of zero entries, converts them all into non-zero values. This destroys the sparsity structure and can require far more memory than the original sparse representation.
scikit-learn addresses this directly: StandardScaler can be applied to sparse CSR or CSC matrices by passing with_mean=False, which avoids breaking the sparsity structure of the data (scikit-learn StandardScaler documentation). With with_mean=False, the scaler still divides by the standard deviation (variance scaling) but skips the centering step, preserving zero entries as zero. The same documentation notes that setting with_mean=True on sparse input will raise an exception, because centering sparse data requires materializing a dense matrix that is often too large to fit in memory (scikit-learn StandardScaler documentation). MaxAbsScaler is another option well-suited to sparse data, since it scales by the maximum absolute value without any centering step, preserving zero entries in sparse data by design (scikit-learn preprocessing user guide).
Missing Values and Preprocessing Order
Standardization and missing-value imputation interact, and getting the order wrong can silently corrupt a pipeline. The generally recommended order is:
Imputation — fill or otherwise handle missing values first.
Scaling — standardize the now-complete numeric columns.
Model fitting — train the estimator on the fully preprocessed data.
If scaling happens before imputation, the mean and standard deviation calculations may either fail on missing values or silently exclude them in ways that are inconsistent with how the imputer later fills them in. Placing the imputer earlier in a Pipeline (as shown in Example E) keeps the order consistent and leakage-safe, since the imputer's own statistics — like the median it fills in — must also be learned only from training data, following the same principle that applies to scalers.
The right imputation strategy — mean, median, a constant, or a model-based method — depends on the estimator, the nature of the data, and the underlying mechanism causing values to be missing. Standardization is not a substitute for a genuine missing-value strategy; it operates only on values that are already present.
Constant and Near-Constant Features
A feature with zero variance — every observation holds the exact same value — creates a divide-by-zero problem in the standardization formula, since σ = 0 makes z = (x − μ) / 0 undefined. In a manual NumPy implementation, this appears as a runtime warning and produces inf or NaN values, exactly the situation guarded against in Example A above.
Mature libraries handle this more gracefully. StandardScaler's internal handling effectively treats a standard deviation of zero as one, so a constant feature is centered (shifted to zero) but not divided by an undefined quantity — the practical effect is that the feature remains constant at zero after transformation, contributing no signal to the model.
Beyond the technical handling, a constant or near-constant feature usually deserves closer feature-selection review on its own merits. If a column never varies across the training set, it cannot help distinguish between outcomes, and its presence may simply be dead weight — or, in some cases, a signal that something went wrong upstream in data collection.
Standardizing Targets vs. Features
Standardizing input features and standardizing the prediction target are two different decisions with different implications.
For regression problems, scaling the target variable can help numerical optimization, particularly for neural networks or gradient-based solvers where a target with a very large or very small scale can slow convergence or destabilize model training. If the target is scaled, predictions come out in the scaled units, so they must be inverse-transformed back to the original scale before being reported, evaluated, or acted upon — forgetting this step is one of the most common mistakes in this entire workflow.
For classification problems, labels represent category identity, not a continuous quantity, so they should never be z-score standardized as though they were ordinary numeric input features. A label like "0" or "1" (or "cat," "dog," "bird") is not meant to be treated as a measurable distance from a mean; doing so would introduce a meaningless ordinal relationship between categories that don't actually have one.
Standardization vs. Batch, Layer, and Instance Normalization
Dataset-level feature standardization — the topic of this article — is a preprocessing step applied once to the input data before training. Deep learning introduces a related but architecturally distinct family of techniques that normalize activations inside a network, at different locations and using different statistics:
Batch normalization normalizes activations across a mini-batch during training, recomputing statistics for each batch; it was introduced specifically to address "internal covariate shift" — the way each layer's input distribution changes as earlier layers' parameters update during training (Ioffe & Szegedy, 2015). The original paper demonstrated that batch normalization allows much higher learning rates and less careful initialization, achieving equivalent accuracy with substantially fewer training steps on a state-of-the-art image classification model (Ioffe & Szegedy, 2015).
Layer normalization normalizes across the features of a single training example rather than across a batch, making it independent of batch size — a property that has made it common in sequence models and transformer architectures.
Instance normalization normalizes each individual sample (and, in image models, each channel of each sample) independently, commonly used in image style-transfer networks.
Group normalization divides channels into groups and normalizes within each group, offering a middle ground between layer and batch normalization that is less sensitive to batch size.
These techniques share the same underlying mathematical intuition as dataset-level standardization — recentering and rescaling values — but they operate at a different point in the pipeline (inside the network, recomputed at every forward pass) rather than once, upfront, on the raw input data. The two concepts are frequently confused in casual conversation, but confusing feature standardization with batch normalization is listed as a common and consequential mistake, discussed further below.
Images, Text, and Time Series
Images: Image pipelines involve several distinct scaling decisions that are easy to conflate. Rescaling pixel values (commonly dividing by 255 to map an 8-bit image into the [0, 1] range) is a form of min-max scaling. Dataset-level channel standardization computes a mean and standard deviation per color channel across an entire training set of images. Per-image standardization instead computes statistics from a single image's own pixels. Normalization layers inside a network (batch, layer, or instance normalization, as described above) are a separate, architecture-internal mechanism. These four approaches address related but distinct goals and are not interchangeable.
Text: Sparse bag-of-words or TF-IDF representations require different treatment from ordinary dense continuous features, largely because of the sparsity concerns discussed earlier — mean-centering a sparse text matrix destroys its sparsity and can be memory-prohibitive. Unit-norm (L2) normalization of each document vector is far more common in text pipelines than z-score standardization of each vocabulary term.
Time series: Scalers must respect temporal order. Computing a mean and standard deviation from the entire series — including future observations — and then using those statistics to scale earlier points is a leakage pattern specific to time series: it lets information from the future influence how the past is represented. The safer patterns are an expanding window (recalculating statistics using only data up to the current point) or a rolling window (using a fixed recent lookback), both of which avoid peeking ahead. Time series also frequently exhibit distribution drift, where the statistical properties of the data change over time, meaning that scaling statistics learned early in a series may become stale later on. In multi-series datasets (e.g., many different sensors or many different customers), a further decision is whether to fit one global scaler across all series or a separate scaler per series — the right choice depends on whether the series share a common underlying scale or are meaningfully different from one another.
Production Deployment Considerations
Standardization does not end at training time — the fitted scaler becomes part of the deployed system and must be handled with the same rigor as the model itself.
Save the fitted scaler alongside the model. The mean and standard deviation learned during training must be persisted (for example, with joblib or pickle) and loaded together with the model at inference time.
Reuse the exact training statistics at inference. Never refit a scaler on incoming production requests — doing so silently reintroduces leakage-like behavior and breaks the assumption that the model is evaluated using the same transformation it was trained on.
Match feature names and order. If a scaler was fit on a DataFrame with a specific column order, production data must be transformed using the same order; a single misaligned column can silently corrupt every downstream prediction.
Avoid independent scaling in separate services. If preprocessing and model inference run in different microservices, both must reference the identical fitted scaler object or its serialized statistics — not two independently fit copies.
Handle schema changes carefully. Adding, removing, or renaming a column requires refitting (and re-validating) the entire preprocessing pipeline, not just patching around the change.
Monitor for distribution drift. If production data's mean and spread diverge meaningfully from the training statistics over time, standardized values will no longer reflect the assumptions the model was trained under — this is a signal to consider recalibration or full retraining.
Plan for batch vs. streaming differences. Batch scoring can transform many rows at once with a stored scaler; streaming systems must apply the same transform to single rows without access to a "batch" to reference.
Version preprocessing alongside the model. Reproducibility requires tracking which scaler version, fit on which training snapshot, was paired with which model version — treating preprocessing as a first-class artifact, not an afterthought.
Interpretability and Model Coefficients
Standardizing features changes the meaning of a linear model's coefficients. Before standardization, a coefficient represents the expected change in outcome per one-unit change in the original feature (e.g., per one additional year of age). After standardization, the same coefficient represents the expected change in outcome per one-standard-deviation change in that feature.
This has a useful side effect: standardized coefficients from features with very different original units (years vs. dollars vs. kilometers) become directly comparable to each other, since they're now all expressed in the same "per standard deviation" terms. This is why standardized coefficients are often used to get a rough sense of relative feature importance in linear models.
However, this comparison comes with real caveats. Coefficient magnitude alone does not always equal genuine feature importance — a feature with a small coefficient but very low variance in the real world can still be a critical driver of outcomes, and a feature that's collinear with another can have its coefficient magnitude split unpredictably between the two. Regularization further complicates the picture: L1 and L2 penalties interact with feature scale directly, so coefficients under a Lasso or Ridge penalty reflect both the feature's real relationship with the outcome and the effect of the regularization strength. To recover a prediction in original units, or to compare it against a domain benchmark, the fitted scaler's inverse_transform method restores values to their pre-standardization scale.
A Practical Decision Guide
Use this checklist to decide whether standardization belongs in a given pipeline.
Question | If yes, lean toward... |
Is the model based on distances, kernels, gradients, or regularization? | Standardization is likely to help |
Do feature ranges differ substantially across columns? | Standardization is likely to help |
Are there major outliers in the numeric features? | Consider RobustScaler or a power transform instead of plain standardization |
Is the input a sparse matrix? | Use with_mean=False or MaxAbsScaler to preserve sparsity |
Does the downstream model need bounded input values? | Consider MinMaxScaler instead |
Is skewness a concern independent of scale? | Consider Yeo-Johnson, Box-Cox, or a log transform |
Is the dataset time-dependent? | Fit scaling statistics using only past data (expanding or rolling window) |
Is the model tree-based (decision tree, random forest, gradient boosting)? | Scaling numeric features is usually unnecessary |
Are preprocessing steps enclosed in a pipeline? | If not, wrap them before running cross-validation or hyperparameter search |
A simple flow: identify the model family → check feature scale differences and outlier presence → choose the scaler that matches both → wrap the choice inside a pipeline → validate the decision through cross-validation rather than assuming it will help.
Common Mistakes to Avoid
Fitting the scaler before the train-test split — the single most common source of leakage in preprocessing.
Fitting separate scalers independently on training and test data — this creates two different sets of statistics, meaning the "same" transformation is not actually being applied consistently.
Scaling every column without considering data type — blindly standardizing one-hot encoded or binary columns alongside continuous ones.
Believing standardization makes data normally distributed — it changes scale, not shape.
Ignoring outliers — letting a handful of extreme values dominate the mean and standard deviation without investigating whether they're valid.
Centering a large sparse matrix — destroying sparsity and risking memory exhaustion.
Forgetting to transform production data — training with a scaled pipeline but serving raw, unscaled features at inference.
Forgetting to inverse-transform a scaled regression target — reporting predictions in the wrong units.
Comparing coefficients without considering preprocessing — misreading standardized coefficients as if they were on the original scale.
Using different feature orders during training and inference — a silent, hard-to-detect bug.
Applying preprocessing outside cross-validation folds — reintroducing the exact leakage a pipeline is meant to prevent.
Assuming higher accuracy after scaling is guaranteed — scaling changes numerical behavior, not the underlying signal in the data.
Confusing feature standardization with batch normalization — these operate at different points in the modeling process and are not interchangeable concepts.
Benefits and Limitations
Potential benefits:
Comparable numerical scales across features with different original units.
Improved numerical conditioning for gradient-based and linear-algebra-heavy algorithms.
More meaningful distance calculations for KNN, k-means, and similar methods.
Fairer application of L1/L2 regularization penalties across features.
Faster, more stable convergence for gradient-based optimizers.
Easier relative comparison of linear-model coefficients.
Compatibility with scale-sensitive algorithms like SVMs and neural networks.
Limitations:
Sensitivity to outliers, since both the mean and standard deviation shift with extreme values.
Loss of the feature's original, interpretable units.
Risk of data leakage if fit incorrectly.
Additional production state (a fitted scaler) that must be versioned and maintained.
No guarantee of normality — shape is untouched.
No universal performance improvement — benefit is model- and data-dependent.
Possible unnecessary complexity for tree-based models that don't need it.
Challenges with distribution drift and nonstationary data over time.
End-to-End Practical Example
The following example demonstrates a complete, leakage-safe workflow using a small tabular dataset with both numerical and categorical columns.
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
# 1. Small illustrative dataset
data = pd.DataFrame({
"age": [23, 45, 31, np.nan, 52, 29, 41, 38],
"income": [32000, 88000, 51000, 61000, 105000, 47000, 72000, 58000],
"region": ["north", "south", "north", "east", "west", "south", "east", "north"],
"purchased": [0, 1, 0, 1, 1, 0, 1, 0]
})
X = data.drop(columns=["purchased"])
y = data["purchased"]
numeric_cols = ["age", "income"]
categorical_cols = ["region"]
# 2. Split BEFORE any preprocessing is fit
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=42, stratify=y
)
# 3. Build a leakage-safe preprocessing + model pipeline
numeric_pipeline = Pipeline(steps=[
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler())
])
categorical_pipeline = Pipeline(steps=[
("encoder", OneHotEncoder(handle_unknown="ignore"))
])
preprocessor = ColumnTransformer(transformers=[
("num", numeric_pipeline, numeric_cols),
("cat", categorical_pipeline, categorical_cols)
])
model_pipeline = Pipeline(steps=[
("preprocessor", preprocessor),
("classifier", LogisticRegression(max_iter=1000, random_state=42))
])
# 4. Evaluate without leakage using cross-validation on the training set
cv_scores = cross_val_score(model_pipeline, X_train, y_train, cv=3)
print("Cross-validation scores:", cv_scores)
# 5. Fit on the full training set, then evaluate once on the held-out test set
model_pipeline.fit(X_train, y_train)
test_accuracy = model_pipeline.score(X_test, y_test)
print("Held-out test accuracy:", test_accuracy)
# 6. What gets saved for production: the ENTIRE fitted pipeline object,
# including the imputer's learned median, the scaler's mean_/scale_,
# the encoder's learned categories, and the trained classifier.
import joblib
joblib.dump(model_pipeline, "production_pipeline.joblib")This example identifies numeric and categorical columns explicitly, splits before any preprocessing is fit, imputes missing values, standardizes the numeric columns, encodes the categorical column, evaluates with cross-validation, and finally saves the entire fitted pipeline — not just the model — as the artifact that production systems need.
Final Synthesis
Standardization is a preprocessing step that recenters a numeric feature around a mean of zero and rescales it to a standard deviation of one, using the formula z = (x − μ) / σ. It matters because many widely used algorithms — distance-based methods, gradient-based optimizers, kernel methods, and regularized linear models — implicitly assume features share a comparable numerical scale, and without that assumption, one large-scaled feature can silently dominate a model's behavior. Standardization tends to help these families, while split-based tree models are generally unaffected because their logic depends on ordering rather than absolute scale.
Standardization is not a universal fix: it does not make data normally distributed, does not remove outliers, and does not guarantee improved accuracy. Where outliers or skew are a concern, RobustScaler, quantile transforms, or power transforms are often better matches. And regardless of which scaler is chosen, fitting it only on training data — and reusing those exact statistics for validation, test, and production data — is essential; anything else risks leaking information the model should never have had access to during evaluation. Wrapping the entire preprocessing sequence inside a pipeline is the most reliable way to guarantee that discipline holds across cross-validation, hyperparameter tuning, and deployment alike.
FAQ
What is standardization in machine learning?
Standardization is a preprocessing technique that rescales a numeric feature so it has a mean of 0 and a standard deviation of 1, using the formula z = (x − μ) / σ. It changes the scale of the data without changing its shape, ordering, or the relationships between features.
Is standardization the same as normalization?
Not exactly. "Normalization" is an overloaded term that can mean min-max scaling to a fixed range, unit-norm scaling of individual samples, or general feature scaling. Standardization specifically refers to the mean-centering and standard-deviation-scaling process described here, and it produces an unbounded, zero-centered result rather than a fixed range like [0, 1].
Does standardization make data normally distributed?
No. Standardization only shifts and rescales values; it does not reshape the distribution. A skewed feature remains skewed after standardization, just measured in different units. Achieving an approximately normal shape requires a separate technique, such as a Yeo-Johnson or Box-Cox power transform.
Which machine-learning algorithms need standardization?
Algorithms based on distances (k-nearest neighbors, k-means), kernels (support vector machines), gradients (logistic regression, neural networks), or regularization penalties (Ridge, Lasso, Elastic Net) generally benefit from standardization, because these methods assume features are on comparable scales.
Do decision trees require standardization?
Generally, no. Decision trees, random forests, and gradient-boosted trees split data based on threshold comparisons, which are unaffected by a monotonic rescaling of an individual feature. Scaling may still matter if trees are combined with a distance-based or linear component elsewhere in a pipeline.
Should standardization happen before or after the train-test split?
After. The data should be split first, and the scaler should be fit only on the training partition. The same fitted scaler is then used to transform the validation and test partitions, never refit on them.
Can standardization cause data leakage?
Yes, if it's fit incorrectly. Fitting a scaler on the entire dataset — or on validation/test data — before splitting lets information from data the model shouldn't see influence the training-time statistics, producing overly optimistic evaluation results.
How does StandardScaler handle new data?
Once StandardScaler is fit on training data, it stores the training mean and standard deviation as mean_ and scale_. Calling transform on new data (validation, test, or production data) applies those same stored statistics rather than recalculating new ones.
Is standardization sensitive to outliers?
Yes. Both the mean and the standard deviation shift when extreme values are present, which can compress the standardized values of typical observations into a narrow range. RobustScaler, which uses the median and interquartile range instead, is specifically designed to resist this effect.
Should categorical or one-hot encoded features be standardized?
It depends. Nominal categorical features should generally be encoded rather than standardized. One-hot encoded columns are already bounded between 0 and 1, so standardizing them is a judgment call that depends on the regularization scheme and modeling goals rather than a fixed rule.
Should the target variable be standardized?
For regression, scaling the target can help numerical optimization, particularly for neural networks, but predictions must then be inverse-transformed back to the original scale. For classification, labels represent categories, not continuous quantities, and should not be z-score standardized.
What happens if a feature has zero variance?
A zero standard deviation makes the z-score formula undefined, since it requires dividing by zero. In a manual implementation this must be explicitly guarded against; mature libraries like scikit-learn handle constant features by effectively treating the scale as one rather than raising a division error.
Can standardization be used with sparse data?
Yes, with adjustments. Passing with_mean=False to StandardScaler skips the centering step, preserving the zero entries that make a sparse matrix efficient. Attempting to center a sparse matrix directly can raise an exception or exhaust memory.
When should RobustScaler be used instead?
RobustScaler is a better fit when the data contains meaningful outliers or heavy-tailed distributions, since it uses the median and interquartile range rather than the mean and standard deviation, making it far less influenced by extreme values.
Does standardization improve accuracy?
Not automatically. Standardization changes the numerical behavior of a model — convergence speed, distance calculations, regularization fairness — but whether that translates into higher accuracy depends entirely on the specific algorithm, dataset, and evaluation setup. It should be validated through cross-validation, not assumed.
How is feature standardization different from batch normalization?
Feature standardization is a one-time preprocessing step applied to the input dataset before training. Batch normalization is a layer inside a neural network that recalculates normalization statistics from each mini-batch during training, addressing a different problem (shifting activation distributions across layers) at a different point in the pipeline.
Key Takeaways
Standardization rescales a feature using z = (x − μ) / σ, producing a result with mean 0 and standard deviation 1 — a change in scale, not in shape.
Distance-based, gradient-based, kernel-based, and regularized linear models typically benefit; tree-based models generally do not need it.
The scaler must be fit only on training data and reused — never refit — on validation, test, and production data to avoid data leakage.
Outliers distort the mean and standard deviation; RobustScaler, quantile transforms, or power transforms are often better choices for skewed or outlier-heavy data.
The fitted scaler is a production artifact that must be saved, versioned, and reused exactly as trained — not recalculated at inference time.
Standardization does not guarantee normality, does not remove outliers, and does not guarantee improved model accuracy on its own.
Wrapping scaling inside a scikit-learn Pipeline is the most reliable way to keep cross-validation and hyperparameter tuning leakage-safe.
Actionable Next Steps
Inspect the numerical features in your dataset for differences in scale, skewness, and outliers before choosing a preprocessing method.
Identify which model family you plan to use, and check it against the algorithm tables above to see whether standardization is likely to help.
Split your data into training and test sets before fitting any preprocessing step.
Select a scaler — StandardScaler for roughly symmetric data, RobustScaler for outlier-heavy data, or a power transform for skewed distributions.
Build a scikit-learn Pipeline (or ColumnTransformer for mixed data types) that bundles imputation, scaling, and the final estimator together.
Validate your preprocessing choice using cross-validation rather than assuming it will improve performance.
Save the entire fitted pipeline — not just the model — for use in production.
Monitor production feature distributions over time to catch drift that could make your training-time scaling statistics stale.
Glossary
Batch normalization — A neural network layer that normalizes activations using statistics computed from each training mini-batch, distinct from dataset-level feature standardization.
Data leakage — When information that would not be available at prediction time is used during model training or evaluation, producing overly optimistic results.
Distribution drift — A change over time in the statistical properties of incoming data compared to the data a model or preprocessing step was originally fit on.
Fit — The step where a preprocessing object (like a scaler) or model learns its internal parameters or statistics from a given dataset.
Interquartile range (IQR) — The range between the first quartile (25th percentile) and third quartile (75th percentile) of a dataset, used by RobustScaler as a measure of spread.
Inverse transform — The operation that reverses a preprocessing transformation, converting scaled values back into their original units.
Mean — The arithmetic average of a set of values.
Min-max scaling — A scaling method that rescales data into a fixed range, typically [0, 1], based on the minimum and maximum values.
Normalization — An overloaded term that can refer to min-max scaling, unit-norm scaling, general feature scaling, or neural network normalization layers, depending on context.
Outlier — An observation that lies unusually far from the rest of the data, which can disproportionately affect statistics like the mean and standard deviation.
Pipeline — A scikit-learn object that chains preprocessing steps and a final estimator together, ensuring the same samples are used to fit each step and helping prevent data leakage.
Robust scaling — A scaling method using the median and interquartile range instead of the mean and standard deviation, making it resistant to outliers.
Sparse matrix — A matrix in which most entries are zero, commonly stored in a compressed format to save memory.
Standard deviation — A measure of how spread out a set of values is around its mean.
Standardization — A preprocessing technique that rescales a feature to have a mean of 0 and a standard deviation of 1 using the z-score formula.
Transform — The step where a fitted preprocessing object applies its learned statistics or parameters to convert raw data into processed data.
Unit variance — A property where a dataset's variance (and therefore standard deviation) equals 1, typically achieved through standardization.
Variance — The average of the squared differences between each value and the mean, equal to the square of the standard deviation.
Z-score — The standardized value of an observation, expressing how many standard deviations it lies from the mean.
Sources & References
scikit-learn developers. "StandardScaler." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html
scikit-learn developers. "RobustScaler." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html
scikit-learn developers. "MinMaxScaler." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html
scikit-learn developers. "PowerTransformer." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PowerTransformer.html
scikit-learn developers. "Compare the effect of different scalers on data with outliers." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html
scikit-learn developers. "Importance of Feature Scaling." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/auto_examples/preprocessing/plot_scaling_importance.html
scikit-learn developers. "8.3. Preprocessing data." scikit-learn 1.9.0 documentation, User Guide. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/preprocessing.html
scikit-learn developers. "12. Common pitfalls and recommended practices." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/common_pitfalls.html
scikit-learn developers. "8.1. Pipelines and composite estimators." scikit-learn 1.9.0 documentation. Accessed 2026-07-20. https://scikit-learn.org/stable/modules/compose.html
Ioffe, S. and Szegedy, C. (2015). "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift." Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:448-456. https://arxiv.org/abs/1502.03167
NumPy developers. "numpy.std." NumPy v2.5 Manual. Accessed 2026-07-20. https://numpy.org/doc/stable/reference/generated/numpy.std.html
Yeo, I.K. and Johnson, R.A. (2000). "A new family of power transformations to improve normality or symmetry." Biometrika, 87(4), 954-959. (As cited in scikit-learn's PowerTransformer documentation, reference [1].)
Box, G.E.P. and Cox, D.R. (1964). "An Analysis of Transformations." Journal of the Royal Statistical Society, Series B, 26, 211-252. (As cited in scikit-learn's PowerTransformer documentation, reference [2].)
