Target awareness
Some representations place their bins, centers, or knots using the target y. Positioning
units where the target actually changes sharpens the representation, but it also reads labels
at fit time, which introduces a leakage risk if done carelessly. PreTab makes target usage
explicit and gives you leakage-safe tools. This page explains the contract.
Which methods use the target
Every method declares how it uses y through three levels.
Level |
Meaning |
Numerical methods |
Categorical methods |
|---|---|---|---|
|
Never uses the target |
The scalers ( |
|
|
Uses the target only when |
The feature maps ( |
- |
|
Always places against the target |
|
- |
from pretab.transformers import PLETransformer
t = PLETransformer()
t.requires_y # True: PLE always needs y
t.is_supervised # True
Warning
A required method fitted without y, or with target_aware=False, raises a typed error
rather than silently producing an unsupervised result. Always pass y to a pipeline that
contains PLE.
Target-aware placement fits one decision tree or boosting model on a single target, so a
multi-output y of shape (n_samples, k) with k > 1 raises IncompatibleParamsError for
ple and for an optional method with target_aware=True; a column vector of shape
(n_samples, 1) counts as a single target. With target_aware=False the optional methods
ignore y, so they also fit inside a multi-output pipeline. Explicit spline
knot_locations also ignore y, because no target-aware selector runs.
The fitted-usage flag
After fitting, a transformer reports whether it actually consumed the target through
uses_target_. This is the ground truth for an individual fit, and it flows into the
RepresentationSpec so a serialized representation records
whether it was supervised.
t = PLETransformer().fit(x, y)
t.uses_target_ # True
Leakage safety
Fitting a supervised transformer on your full dataset and then evaluating on part of it leaks target information and inflates scores. PreTab warns when it detects this pattern.
Important
A supervised transformer emits a LeakageWarning when it is fit with a target outside a
cross-validation or Pipeline context. Inside a scikit-learn Pipeline, ColumnTransformer,
Preprocessor, or a cross-fitting wrapper, the warning is suppressed because those contexts
already keep the fit confined to the training fold.
The safe patterns are:
Put the supervised transformer inside a
Pipeline, socross_val_scoreandGridSearchCVfit it on the training fold only.Use the
Preprocessor, which fits its imputers and supervised expansions on the training data you pass tofit.Wrap it in a
CrossFittedTransformerwhen you want out-of-fold training features.
Cross-fitted features
Even inside a Pipeline, fitting a supervised transformer once on the full training set and
then using it to build the training features for the same rows introduces a subtle leak:
each row’s PLE bins were placed using its own target value. CrossFittedTransformer fixes
this for the training features specifically. It splits the training data into folds, fits a
fresh copy of the transformer on all folds except one, and uses that copy to transform the
held-out fold, so every training row is transformed by a model that never saw its own target.
transform on genuinely new data (a validation or test set) instead uses one model fit on all
the training data, since there is no leakage risk there. Both return the same kind of output,
the one the wrapped transformer produces: a dense array of the same dtype, a sparse matrix in
the same format, or a pandas/Polars DataFrame. Fold outputs are restored to the input row
order. A pandas output keeps the input index when the wrapped transformer preserves it;
otherwise it uses a fresh row index.
Cross-fitting currently requires matrix output. A wrapped Preprocessor configured with
output_structure="blocks" raises an explicit error; use output_structure="matrix" instead.
Each fold must return a 2D output with one row per held-out sample, a fixed width, and, for
DataFrames, the same column labels in the same order as the all-data fit.
A wrapped Preprocessor is not refit from scratch on each fold: every fold reuses its all-data
fit and refits only the target-aware representations and subsequent pipeline steps. Column
types, target-free blocks and their category vocabularies, imputers, scalers before the
representation, and missing-indicator branches are shared. Target-free values match transform exactly;
target-aware values can differ because they exclude the held-out targets.
Missing indicators stay as raw columns next to the representation; they are not expanded into basis functions and do not disappear when a fold’s training rows contain no missing values. Each refitted block must still retain its all-data output width. If a representation’s data-dependent sizing changes that width, cross-fitting raises an error instead of mixing incompatible feature layouts.
import numpy as np
from pretab import CrossFittedTransformer
from pretab.transformers import PLETransformer
rng = np.random.default_rng(0)
x_train = rng.uniform(-3.0, 3.0, size=(200, 1))
y_train = rng.normal(size=200)
# Naive: one PLE fit on all the training data, then used to transform that same data.
naive = PLETransformer(output_dim=10, random_state=0).fit(x_train, y_train).transform(x_train)
# Cross-fitted: each row is transformed by a model that did not see its own target.
cf = CrossFittedTransformer(PLETransformer(output_dim=10, random_state=0), n_folds=5, random_state=0)
out_of_fold = cf.fit_transform(x_train, y_train)
changed = (~np.all(naive == out_of_fold, axis=1)).sum()
changed, len(x_train)
(193, 200)
193 of the 200 training rows get different feature values once out-of-fold cross-fitting is
used, which is the leakage CrossFittedTransformer removes: the naive version handed a
downstream model features that were partly informed by the very target it is trying to
predict.
Tip
Use CrossFittedTransformer when you need the training features themselves to be
leakage-free, for example to feed a second-stage model or to report an honest training-set
metric. A supervised transformer inside a plain Pipeline already keeps cross-validation
honest for cross_val_score / GridSearchCV, since each fold refits from scratch; you only
need cross-fitting when you build the training features once and reuse them directly.
The fitted spec records cross_fitted=True and the number of folds, so the choice is
visible and serializable.
CrossFittedTransformer can also wrap a Preprocessor, ColumnTransformer, or
Pipeline. Pandas DataFrames keep their column names, dtypes, and index labels when
passed to the wrapped estimator; each fold is selected by row position. Polars
DataFrames are converted once to pandas before selecting folds, so nullable column
dtypes remain consistent. Other array-like inputs retain the NumPy conversion, with
one-dimensional input treated as a single column.
Note
Cross-fitting matters most for strongly supervised encodings such as PLE, where the target directly determines the bins. For unsupervised methods it is unnecessary.
Searching over representations
Choosing a numerical method by comparing validation scores is itself a form of model
selection, and doing it carelessly (for example scoring each candidate on the same data used
to fit it) leaks information the same way an unguarded supervised transformer does.
RepresentationSearchCV cross-validates a downstream estimator over a set of candidate
numerical_method values, refits the best one on all the data, and keeps every candidate’s
scoring honest by fitting a fresh Preprocessor per fold. With a classifier as the estimator,
every Preprocessor uses task="classification", so target-aware methods place against the
class labels; a task set in preprocessor_params takes precedence.
import numpy as np
import pandas as pd
from sklearn.linear_model import Ridge
from pretab import RepresentationSearchCV
rng = np.random.default_rng(0)
X = pd.DataFrame({"x": rng.uniform(-3, 3, size=300)})
y = np.sin(X["x"]) + rng.normal(0, 0.1, size=300)
search = RepresentationSearchCV(
estimator=Ridge(),
methods=["minmax", "ple", "bspline", "rbf"],
cv=5,
random_state=0,
)
search.fit(X, y)
search.cv_results_
search.best_method_
{'minmax': 0.636, 'ple': 0.949, 'bspline': 0.973, 'rbf': 0.841}
'bspline'
bspline scored highest across the 5 folds for this sine-shaped signal, so search. best_preprocessor_ and search.best_estimator_ are refit on all the data with bspline and
ready to call .predict(X_new).
When several rows belong to the same unit, such as repeated measurements of one patient, a
row-level split validates each candidate on units it has already seen in training. Pass a
group splitter as cv and the group labels to fit, so no group is on both sides of a
train/validation split:
from sklearn.model_selection import GroupKFold
patient_id = np.repeat(np.arange(30), 10) # 30 patients with 10 rows each
search = RepresentationSearchCV(Ridge(), methods=["minmax", "bspline"], cv=GroupKFold(5))
search.fit(X, y, groups=patient_id)
Any other keyword argument to fit, such as sample_weight, is passed to the estimator’s
fit on every fold (restricted to that fold’s training rows) and on the final refit. The
Preprocessor and the scorer do not see it, so placement and validation scores are
unweighted.
The search takes the estimator type of the estimator it wraps. Around a classifier it is a
classifier itself and delegates classes_, predict_proba, predict_log_proba and
decision_function (whichever the estimator provides) to the refit best_estimator_. It can
therefore be evaluated as a whole in an outer, stratified cross-validation, for example with
cross_val_score(search, X, y_class, scoring="roc_auc").
Note
This is deliberately narrow: it only searches the single numerical_method axis with one
global method for every numerical column, not a per-column feature_preprocessing search.
Use it to answer “which single method suits this dataset” before committing to a
Preprocessor configuration, not as a general hyperparameter search.
See the target-aware classification tutorial for an end-to-end, leakage-safe evaluation.
Where to go next
Resolution and placement for the placement strategies.
Reproducibility for how supervised state is recorded and serialized.
Leakage-safe classification for a worked example.