yikit.visualize package#
Visualization tools for machine learning models.
This module provides utilities for visualizing model results, including permutation importance, learning curves, distribution plots, and matplotlib configuration helpers.
- class yikit.visualize.SummarizePI(importances: DataFrame)#
Bases:
objectSummarize permutation importances as a bar chart.
The importances are divided by the sum of the per-feature means, so the means of the features sum to 1. For each feature,
get_data()returns the mean and the standard deviation of the divided importances over the repeats, andget_figure()draws them as horizontal bars with error bars, the most important feature on top.- Parameters:
importances (pandas.DataFrame of shape (n_features, n_repeats)) – The permutation importances: one row per feature (the index holds the feature names) and one column per repeat, such as
pd.DataFrame(result.importances, index=X.columns)for theresultofsklearn.inspection.permutation_importance().
- importances#
The importances given to the constructor.
- Type:
pandas.DataFrame of shape (n_features, n_repeats)
- fig#
The figure drawn by the last call of
get_figure(). Defined only after it is called.- Type:
matplotlib.figure.Figure
- ax#
The axes of
fig. Defined only afterget_figure()is called.- Type:
matplotlib.axes.Axes
Examples
>>> import pandas as pd >>> from yikit.visualize import SummarizePI >>> importances = pd.DataFrame( ... [[0.4, 0.8], [0.2, 0.2], [1.0, 1.4]], index=["a", "b", "c"] ... ) >>> summarize_pi = SummarizePI(importances) >>> summarize_pi.get_data() importances_mean importances_std c 0.6 0.1 a 0.3 0.1 b 0.1 0.0 >>> fig, ax = summarize_pi.get_figure(fontfamily="DejaVu Sans") >>> [label.get_text() for label in ax.get_yticklabels()] ['c', 'a', 'b']
>>> import matplotlib.pyplot as plt >>> plt.close(fig)
- get_data() DataFrame#
Return the values drawn by
get_figure().- Returns:
One row per feature, indexed by the index of
importancesand sorted by"importances_mean"in descending order (features with equal means keep their order). The column"importances_mean"holds the mean of the importances over the repeats divided by the sum of these means, and"importances_std"the standard deviation of the divided importances over the repeats (withddof=0, as theimportances_stdofsklearn.inspection.permutation_importance()).- Return type:
pandas.DataFrame of shape (n_features, 2)
- get_figure(fontfamily: str | None = None) tuple[Figure, Axes]#
Draw the values of
get_data()as horizontal bars.Each bar is the normalized mean importance of a feature and its error bar spans one standard deviation on each side. The global font settings of matplotlib (
rcParams["font.family"]andrcParams["font.size"]) are changed while the figure is drawn and restored afterwards.- Parameters:
fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see
yikit.visualize.set_font()).- Returns:
- yikit.visualize.get_dist_figure(y_dist, y_true=None, keep_y_range: bool = True, return_axis: bool = False, verbose: bool = True, titles: Sequence[object] | None = None, fontfamily: str | None = None)#
Draw the predicted distribution of each sample in its own subplot.
Each subplot shows the probability density of the distribution of one sample, a vertical line at its mean (“pred”) and, when
y_trueis given, a vertical line at the true value (“ground truth”). The subplots fill a grid ofceil(sqrt(n_samples))rows row by row, and the unused subplots at the end are hidden. Every density is drawn on the same 200 points, spanning the predicted means with a margin of 5% of their range on each side. When the means span no range (such as for one sample), the margin is three times the largest finite, positive standard deviation of the distributions instead, or 1.0 if no distribution has one (such as Student’s t withdf <= 2, whose standard deviation is infinite). The global font settings of matplotlib (rcParams["font.family"]andrcParams["font.size"]) are changed while the figure is drawn and restored afterwards.- Parameters:
y_dist (ngboost.distns.Distn) – The predicted distributions of the samples, such as
NGBRegressor.pred_dist(X). It needs the methodspdf,meanandstd, as continuous regression distributions such asNormalhave (Poissonhas nopdf, and the distributions of classifiers have nomean).y_true (array-like of shape (n_samples,) or None, default=None) – The true values of the samples. None draws no true values.
keep_y_range (bool, default=True) – Whether every subplot has the same top of the y axis: 1.05 times the highest density of all the samples. If False, each subplot scales its own y axis.
return_axis (bool, default=False) – Whether to return the grid of subplots along with the figure.
verbose (bool, default=True) – Whether to show a progress bar while drawing.
titles (sequence of length n_samples or None, default=None) – The titles of the subplots, one for each sample. None, or an empty sequence, titles them “idx: 0”, “idx: 1”, and so on.
fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see
yikit.visualize.set_font()).
- Returns:
fig (matplotlib.figure.Figure) – The figure. Returned alone when
return_axisis False.axes (numpy.ndarray of matplotlib.axes.Axes of shape (n_rows, n_cols)) – The grid of subplots, two-dimensional even when it has one row or one column. Returned as
(fig, axes)only whenreturn_axisis True.
- Raises:
TypeError – If
y_distis not a distribution of ngboost.ValueError – If
titlesis not empty and its length differs from the number of samples.
Examples
>>> import matplotlib.pyplot as plt >>> import numpy as np >>> from ngboost.distns import Normal >>> from yikit.visualize import get_dist_figure
ngboost’s
Normaltakes the rows (loc, log(scale)), here the means 10, 25 and 40 with the standard deviations 4, 6 and 5.>>> params = np.array([[10.0, 25.0, 40.0], np.log([4.0, 6.0, 5.0])]) >>> y_dist = Normal(params) >>> fig, axes = get_dist_figure( ... y_dist, ... y_true=[12.0, 20.0, 43.0], ... verbose=False, ... fontfamily="DejaVu Sans", ... return_axis=True, ... ) >>> axes.shape (2, 2) >>> [ax.get_title() for ax in axes.flat] ['idx: 0', 'idx: 1', 'idx: 2', ''] >>> plt.close(fig)
- yikit.visualize.get_learning_curve_gb(estimator, fontfamily: str | None = None, return_axis: bool = False)#
Plot the learning curve of a fitted NGBoost or LightGBM model.
For each evaluation data recorded during the fit, the scores of the first metric are drawn as a line over the boosting iterations, labeled with the name of the data. The data named “train”, “val” and “test” (NGBoost names its training and validation data “train” and “val”) are drawn in fixed colors; other names, such as LightGBM’s default “training” and “valid_0”, take the colors of matplotlib’s color cycle in order. The global font settings of matplotlib (
rcParams["font.family"]andrcParams["font.size"]) are changed while the figure is drawn and restored afterwards.- Parameters:
estimator (NGBoost or LightGBM model) – A fitted NGBoost model, such as NGBRegressor or NGBClassifier, whose
evals_resultholds the scores of the training data and of the validation data passed tofit(X_valandY_val). Or a fitted LightGBM model of the scikit-learn API, such as LGBMRegressor or LGBMClassifier, whoseevals_result_holds the scores of the evaluation data passed tofit(eval_set, oreval_Xandeval_yin LightGBM 4.7 and later).fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see
yikit.visualize.set_font()).return_axis (bool, default=False) – Whether to return the axes of the figure along with the figure.
- Returns:
fig (matplotlib.figure.Figure) – The figure. Returned alone when
return_axisis False.ax (matplotlib.axes.Axes) – The axes of the figure. Returned as
(fig, ax)only whenreturn_axisis True.
- Raises:
TypeError – If
estimatoris not an NGBoost or LightGBM model.ValueError – If
estimatorhas no evaluation record: it is not fitted, or it is a LightGBM model fitted without evaluation data or withmetric="None".
Examples
>>> import matplotlib.pyplot as plt >>> import numpy as np >>> from lightgbm import LGBMRegressor >>> from yikit.visualize import get_learning_curve_gb >>> rng = np.random.RandomState(334) >>> X = rng.uniform(-1.0, 1.0, size=(100, 3)) >>> y = X @ np.array([3.0, -2.0, 1.0]) >>> X_train, X_val, y_train, y_val = X[:80], X[80:], y[:80], y[80:]
Unless
eval_namesis given, LightGBM names the training data “training” and the other evaluation data “valid_<position>”.>>> model = LGBMRegressor(n_estimators=20, verbose=-1).fit( ... X_train, y_train, eval_set=[(X_train, y_train), (X_val, y_val)] ... ) >>> fig, ax = get_learning_curve_gb( ... model, fontfamily="DejaVu Sans", return_axis=True ... ) >>> [line.get_label() for line in ax.get_lines()] ['training', 'valid_1'] >>> ax.get_xlabel(), ax.get_ylabel() ('n_iter', 'l2') >>> plt.close(fig)
- yikit.visualize.get_learning_curve_optuna(study: optuna.study.Study, loc: LegendLocType = 'best', fontfamily: str | None = None, return_axis: bool = False)#
Plot the optimization history of a single-objective Optuna study.
The objective value of each complete trial is drawn as a point at its trial number, and the best value so far (the lowest for a minimized study, the highest for a maximized one) as a line. Trials in the other states (failed, pruned, running or waiting) are left out, even when they hold a value, such as the last intermediate value that optuna keeps for a pruned trial. The global font settings of matplotlib (
rcParams["font.family"]andrcParams["font.size"]) are changed while the figure is drawn and restored afterwards.- Parameters:
study (optuna.study.Study) – A single-objective study with a direction and at least one complete trial.
loc (str, int or tuple of float, default="best") – The location of the legend, passed to
matplotlib.axes.Axes.legend(): a location string such as"upper right", a location code, or the(x, y)position of the lower left corner of the legend in axes coordinates.fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see
yikit.visualize.set_font()).return_axis (bool, default=False) – Whether to return the axes of the figure along with the figure.
- Returns:
fig (matplotlib.figure.Figure) – The figure. Returned alone when
return_axisis False.ax (matplotlib.axes.Axes) – The axes of the figure. Returned as
(fig, ax)only whenreturn_axisis True.
- Raises:
ValueError – If the study has more than one objective, has no direction (
StudyDirection.NOT_SET), or has no complete trial.
Examples
>>> import matplotlib.pyplot as plt >>> import optuna >>> from yikit.visualize import get_learning_curve_optuna >>> def objective(trial): ... x = trial.suggest_float("x", -10, 10) ... return (x - 2) ** 2 >>> study = optuna.create_study( ... sampler=optuna.samplers.RandomSampler(seed=334) ... ) >>> study.optimize(objective, n_trials=20) >>> fig, ax = get_learning_curve_optuna( ... study, fontfamily="DejaVu Sans", return_axis=True ... ) >>> ax.get_xlabel(), ax.get_ylabel() ('Trials', 'Objective Values') >>> plt.close(fig)
- yikit.visualize.set_font(fontfamily: str | None = None, fontsize: float = 13) None#
Set the global font family and font size of matplotlib.
This changes
matplotlib.pyplot.rcParams["font.family"]andrcParams["font.size"]for the rest of the session: every text created afterwards uses them, and they are not restored. To change them only for a block of code, wrap it inmatplotlib.rc_context(); to change them only while a function runs, decorate it withwith_custom_matplotlib_settings().- Parameters:
fontfamily (str or None, default=None) – The font family. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms. The name is not checked here: when a figure is drawn with a font that is not installed, matplotlib warns and falls back to its default font.
fontsize (float, default=13) – The default font size in points. Sizes such as “medium” or “large” of the labels are relative to it.
See also
with_custom_matplotlib_settingsSet the font only while a function runs.
Examples
>>> import matplotlib.pyplot as plt >>> from yikit.visualize import set_font >>> set_font(fontfamily="Arial", fontsize=14)
The settings stay changed after the call;
matplotlib.rc_contextrestores them at the end of the block.>>> with plt.rc_context(): ... set_font(fontfamily="DejaVu Sans", fontsize=14) ... print(plt.rcParams["font.family"], plt.rcParams["font.size"]) ['DejaVu Sans'] 14.0
- yikit.visualize.with_custom_matplotlib_settings(fontfamily: str | None = None, fontsize: float = 13, restore: bool = True) Callable[[F], F]#
Return a decorator that sets the font while the function runs.
The decorated function runs with the global font family and font size of matplotlib (
rcParams["font.family"]andrcParams["font.size"]) set tofontfamilyandfontsize. Withrestore=Truethe previous values of these two settings are restored when the function returns or raises; other settings changed inside the function are not restored.- Parameters:
fontfamily (str or None, default=None) – The font family. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see
set_font()).fontsize (float, default=13) – The default font size in points.
restore (bool, default=True) – Whether to restore the previous font family and font size after the call. If False, the settings stay changed as with
set_font().
- Returns:
A decorator that wraps a function, keeping its name and docstring.
- Return type:
Callable
See also
set_fontSet the font for the rest of the session.
Examples
>>> import matplotlib.pyplot as plt >>> from yikit.visualize import with_custom_matplotlib_settings >>> @with_custom_matplotlib_settings(fontfamily="DejaVu Sans", fontsize=14) ... def font_size(): ... return plt.rcParams["font.size"] >>> with plt.rc_context({"font.size": 10}): ... print(font_size(), plt.rcParams["font.size"]) 14.0 10.0
- yikit.visualize.yyplot(y_true: ArrayLike, y_pred: ArrayLike, *, labels: Sequence[str | None] | None = None, metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) Axes#
- yikit.visualize.yyplot(y_train: ArrayLike, y_pred_on_train: ArrayLike, y_test: ArrayLike, y_pred_on_test: ArrayLike, *, labels: Sequence[str] | None = ('train', 'test'), metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) Axes
- yikit.visualize.yyplot(y_train: ArrayLike, y_pred_on_train: ArrayLike, y_val: ArrayLike, y_pred_on_val: ArrayLike, y_test: ArrayLike, y_pred_on_test: ArrayLike, *, labels: Sequence[str | None] | None = ('train', 'val', 'test'), metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) Axes
- yikit.visualize.yyplot(*y_data: ArrayLike, labels: Sequence[str | None] | None = None, metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) Axes
Plot predicted against true values for one or more data sets.
Each
(y_true, y_pred)pair is drawn as a scatter plot over the identity line \(y = x\), and the chosen regression metrics of each pair are written in the top-left corner. Typical use is to compare a model on its train, validation and test sets.- Parameters:
*y_data (array-like) – Consecutive pairs
(y_true_0, y_pred_0, y_true_1, y_pred_1, ...)of true and predicted values. At least one pair is required, and the two arrays of a pair must have the same length.labels (sequence of str or None, optional) – Name of each pair, shown in the legend and next to its metrics exactly as written: characters such as
$,_,^and%are not read as mathtext. A pair labeled None or""has no legend entry, and its metrics carry no label. If None (default), two pairs are labeled("train", "test"), three pairs("train", "val", "test"), and any other number of pairs is not labeled.metrics (sequence of {"r2", "rmse", "mae", "mse"}, default=("r2", "rmse")) – Metrics written for each pair, in this order: the coefficient of determination, the root mean squared error, the mean absolute error and the mean squared error. For multi-output targets, RMSE is the mean of the RMSE of each output, as in scikit-learn.
ax (matplotlib.axes.Axes, optional) – Axes to draw on. Only this axes changes: the layout of its figure (such as the subplot parameters) is left as it is. If None (default), a new 3.6 x 3.6 inch figure is created and laid out with
tight_layout.alpha (float, default=0.05) – Margin added on each side of the range of all the values, as a fraction of that range, to set the limits of both axes.
- Returns:
The axes that was drawn on.
- Return type:
matplotlib.axes.Axes
- Raises:
ValueError – If no pair or an odd number of arrays is given, if the length of
labelsdiffers from the number of pairs, or ifmetricscontains an unknown name. These are checked before anything is drawn, so no figure is created andaxis left untouched.
Notes
The legend shows the scatter plots labeled here together with the labeled artists already on
ax. Labels are drawn as written unlessrcParams["text.usetex"]is enabled, in which case LaTeX reads them.Examples
Plot a single data set:
>>> import numpy as np >>> from yikit.visualize import yyplot >>> y_true = np.array([1.0, 2.0, 3.0]) >>> y_pred = np.array([0.8, 2.1, 2.9]) >>> ax = yyplot(y_true, y_pred)
Plot train and test data sets with the mean absolute error as well:
>>> y_train, y_pred_train = np.arange(5), np.arange(5) + 0.1 >>> y_test, y_pred_test = np.arange(5), np.arange(5) - 0.2 >>> ax = yyplot(y_train, y_pred_train, y_test, y_pred_test, ... labels=("train", "test"), metrics=("r2", "rmse", "mae"))
Draw on one axes of an existing figure, whose layout is kept:
>>> import matplotlib.pyplot as plt >>> fig, axes = plt.subplots(1, 2) >>> ax = yyplot(y_true, y_pred, labels=["fold_1 (5% noise)"], ax=axes[0]) >>> ax is axes[0] True >>> plt.close("all")