yikit.visualize package#

Visualization tools for machine learning models.

This module provides utilities for visualizing model results, including permutation importance, learning curves, distribution plots, and matplotlib configuration helpers.

class yikit.visualize.SummarizePI(importances: DataFrame)#

Bases: object

Summarize permutation importances as a bar chart.

The importances are divided by the sum of the per-feature means, so the means of the features sum to 1. For each feature, get_data() returns the mean and the standard deviation of the divided importances over the repeats, and get_figure() draws them as horizontal bars with error bars, the most important feature on top.

Parameters:

importances (pandas.DataFrame of shape (n_features, n_repeats)) – The permutation importances: one row per feature (the index holds the feature names) and one column per repeat, such as pd.DataFrame(result.importances, index=X.columns) for the result of sklearn.inspection.permutation_importance().

importances#

The importances given to the constructor.

Type:

pandas.DataFrame of shape (n_features, n_repeats)

fig#

The figure drawn by the last call of get_figure(). Defined only after it is called.

Type:

matplotlib.figure.Figure

ax#

The axes of fig. Defined only after get_figure() is called.

Type:

matplotlib.axes.Axes

Examples

>>> import pandas as pd
>>> from yikit.visualize import SummarizePI
>>> importances = pd.DataFrame(
...     [[0.4, 0.8], [0.2, 0.2], [1.0, 1.4]], index=["a", "b", "c"]
... )
>>> summarize_pi = SummarizePI(importances)
>>> summarize_pi.get_data()
   importances_mean  importances_std
c               0.6              0.1
a               0.3              0.1
b               0.1              0.0
>>> fig, ax = summarize_pi.get_figure(fontfamily="DejaVu Sans")
>>> [label.get_text() for label in ax.get_yticklabels()]
['c', 'a', 'b']
>>> import matplotlib.pyplot as plt
>>> plt.close(fig)
get_data() → DataFrame#

Return the values drawn by get_figure().

Returns:

One row per feature, indexed by the index of importances and sorted by "importances_mean" in descending order (features with equal means keep their order). The column "importances_mean" holds the mean of the importances over the repeats divided by the sum of these means, and "importances_std" the standard deviation of the divided importances over the repeats (with ddof=0, as the importances_std of sklearn.inspection.permutation_importance()).

Return type:

pandas.DataFrame of shape (n_features, 2)

get_figure(fontfamily: str | None = None) → tuple[Figure, Axes]#

Draw the values of get_data() as horizontal bars.

Each bar is the normalized mean importance of a feature and its error bar spans one standard deviation on each side. The global font settings of matplotlib (rcParams["font.family"] and rcParams["font.size"]) are changed while the figure is drawn and restored afterwards.

Parameters:

fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see yikit.visualize.set_font()).

Returns:

  • fig (matplotlib.figure.Figure) – The figure, also stored in fig.

  • ax (matplotlib.axes.Axes) – The axes of the figure, also stored in ax.

yikit.visualize.get_dist_figure(y_dist, y_true=None, keep_y_range: bool = True, return_axis: bool = False, verbose: bool = True, titles: Sequence[object] | None = None, fontfamily: str | None = None)#

Draw the predicted distribution of each sample in its own subplot.

Each subplot shows the probability density of the distribution of one sample, a vertical line at its mean (“pred”) and, when y_true is given, a vertical line at the true value (“ground truth”). The subplots fill a grid of ceil(sqrt(n_samples)) rows row by row, and the unused subplots at the end are hidden. Every density is drawn on the same 200 points, spanning the predicted means with a margin of 5% of their range on each side. When the means span no range (such as for one sample), the margin is three times the largest finite, positive standard deviation of the distributions instead, or 1.0 if no distribution has one (such as Student’s t with df <= 2, whose standard deviation is infinite). The global font settings of matplotlib (rcParams["font.family"] and rcParams["font.size"]) are changed while the figure is drawn and restored afterwards.

Parameters:
  • y_dist (ngboost.distns.Distn) – The predicted distributions of the samples, such as NGBRegressor.pred_dist(X). It needs the methods pdf, mean and std, as continuous regression distributions such as Normal have (Poisson has no pdf, and the distributions of classifiers have no mean).

  • y_true (array-like of shape (n_samples,) or None, default=None) – The true values of the samples. None draws no true values.

  • keep_y_range (bool, default=True) – Whether every subplot has the same top of the y axis: 1.05 times the highest density of all the samples. If False, each subplot scales its own y axis.

  • return_axis (bool, default=False) – Whether to return the grid of subplots along with the figure.

  • verbose (bool, default=True) – Whether to show a progress bar while drawing.

  • titles (sequence of length n_samples or None, default=None) – The titles of the subplots, one for each sample. None, or an empty sequence, titles them “idx: 0”, “idx: 1”, and so on.

  • fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see yikit.visualize.set_font()).

Returns:

  • fig (matplotlib.figure.Figure) – The figure. Returned alone when return_axis is False.

  • axes (numpy.ndarray of matplotlib.axes.Axes of shape (n_rows, n_cols)) – The grid of subplots, two-dimensional even when it has one row or one column. Returned as (fig, axes) only when return_axis is True.

Raises:
  • TypeError – If y_dist is not a distribution of ngboost.

  • ValueError – If titles is not empty and its length differs from the number of samples.

Examples

>>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from ngboost.distns import Normal
>>> from yikit.visualize import get_dist_figure

ngboost’s Normal takes the rows (loc, log(scale)), here the means 10, 25 and 40 with the standard deviations 4, 6 and 5.

>>> params = np.array([[10.0, 25.0, 40.0], np.log([4.0, 6.0, 5.0])])
>>> y_dist = Normal(params)
>>> fig, axes = get_dist_figure(
...     y_dist,
...     y_true=[12.0, 20.0, 43.0],
...     verbose=False,
...     fontfamily="DejaVu Sans",
...     return_axis=True,
... )
>>> axes.shape
(2, 2)
>>> [ax.get_title() for ax in axes.flat]
['idx: 0', 'idx: 1', 'idx: 2', '']
>>> plt.close(fig)
yikit.visualize.get_learning_curve_gb(estimator, fontfamily: str | None = None, return_axis: bool = False)#

Plot the learning curve of a fitted NGBoost or LightGBM model.

For each evaluation data recorded during the fit, the scores of the first metric are drawn as a line over the boosting iterations, labeled with the name of the data. The data named “train”, “val” and “test” (NGBoost names its training and validation data “train” and “val”) are drawn in fixed colors; other names, such as LightGBM’s default “training” and “valid_0”, take the colors of matplotlib’s color cycle in order. The global font settings of matplotlib (rcParams["font.family"] and rcParams["font.size"]) are changed while the figure is drawn and restored afterwards.

Parameters:
  • estimator (NGBoost or LightGBM model) – A fitted NGBoost model, such as NGBRegressor or NGBClassifier, whose evals_result holds the scores of the training data and of the validation data passed to fit (X_val and Y_val). Or a fitted LightGBM model of the scikit-learn API, such as LGBMRegressor or LGBMClassifier, whose evals_result_ holds the scores of the evaluation data passed to fit (eval_set, or eval_X and eval_y in LightGBM 4.7 and later).

  • fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see yikit.visualize.set_font()).

  • return_axis (bool, default=False) – Whether to return the axes of the figure along with the figure.

Returns:

  • fig (matplotlib.figure.Figure) – The figure. Returned alone when return_axis is False.

  • ax (matplotlib.axes.Axes) – The axes of the figure. Returned as (fig, ax) only when return_axis is True.

Raises:
  • TypeError – If estimator is not an NGBoost or LightGBM model.

  • ValueError – If estimator has no evaluation record: it is not fitted, or it is a LightGBM model fitted without evaluation data or with metric="None".

Examples

>>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from lightgbm import LGBMRegressor
>>> from yikit.visualize import get_learning_curve_gb
>>> rng = np.random.RandomState(334)
>>> X = rng.uniform(-1.0, 1.0, size=(100, 3))
>>> y = X @ np.array([3.0, -2.0, 1.0])
>>> X_train, X_val, y_train, y_val = X[:80], X[80:], y[:80], y[80:]

Unless eval_names is given, LightGBM names the training data “training” and the other evaluation data “valid_<position>”.

>>> model = LGBMRegressor(n_estimators=20, verbose=-1).fit(
...     X_train, y_train, eval_set=[(X_train, y_train), (X_val, y_val)]
... )
>>> fig, ax = get_learning_curve_gb(
...     model, fontfamily="DejaVu Sans", return_axis=True
... )
>>> [line.get_label() for line in ax.get_lines()]
['training', 'valid_1']
>>> ax.get_xlabel(), ax.get_ylabel()
('n_iter', 'l2')
>>> plt.close(fig)
yikit.visualize.get_learning_curve_optuna(study: optuna.study.Study, loc: LegendLocType = 'best', fontfamily: str | None = None, return_axis: bool = False)#

Plot the optimization history of a single-objective Optuna study.

The objective value of each complete trial is drawn as a point at its trial number, and the best value so far (the lowest for a minimized study, the highest for a maximized one) as a line. Trials in the other states (failed, pruned, running or waiting) are left out, even when they hold a value, such as the last intermediate value that optuna keeps for a pruned trial. The global font settings of matplotlib (rcParams["font.family"] and rcParams["font.size"]) are changed while the figure is drawn and restored afterwards.

Parameters:
  • study (optuna.study.Study) – A single-objective study with a direction and at least one complete trial.

  • loc (str, int or tuple of float, default="best") – The location of the legend, passed to matplotlib.axes.Axes.legend(): a location string such as "upper right", a location code, or the (x, y) position of the lower left corner of the legend in axes coordinates.

  • fontfamily (str or None, default=None) – The font family of the figure. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see yikit.visualize.set_font()).

  • return_axis (bool, default=False) – Whether to return the axes of the figure along with the figure.

Returns:

  • fig (matplotlib.figure.Figure) – The figure. Returned alone when return_axis is False.

  • ax (matplotlib.axes.Axes) – The axes of the figure. Returned as (fig, ax) only when return_axis is True.

Raises:

ValueError – If the study has more than one objective, has no direction (StudyDirection.NOT_SET), or has no complete trial.

Examples

>>> import matplotlib.pyplot as plt
>>> import optuna
>>> from yikit.visualize import get_learning_curve_optuna
>>> def objective(trial):
...     x = trial.suggest_float("x", -10, 10)
...     return (x - 2) ** 2
>>> study = optuna.create_study(
...     sampler=optuna.samplers.RandomSampler(seed=334)
... )
>>> study.optimize(objective, n_trials=20)
>>> fig, ax = get_learning_curve_optuna(
...     study, fontfamily="DejaVu Sans", return_axis=True
... )
>>> ax.get_xlabel(), ax.get_ylabel()
('Trials', 'Objective Values')
>>> plt.close(fig)
yikit.visualize.set_font(fontfamily: str | None = None, fontsize: float = 13) → None#

Set the global font family and font size of matplotlib.

This changes matplotlib.pyplot.rcParams["font.family"] and rcParams["font.size"] for the rest of the session: every text created afterwards uses them, and they are not restored. To change them only for a block of code, wrap it in matplotlib.rc_context(); to change them only while a function runs, decorate it with with_custom_matplotlib_settings().

Parameters:
  • fontfamily (str or None, default=None) – The font family. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms. The name is not checked here: when a figure is drawn with a font that is not installed, matplotlib warns and falls back to its default font.

  • fontsize (float, default=13) – The default font size in points. Sizes such as “medium” or “large” of the labels are relative to it.

See also

with_custom_matplotlib_settings

Set the font only while a function runs.

Examples

>>> import matplotlib.pyplot as plt
>>> from yikit.visualize import set_font
>>> set_font(fontfamily="Arial", fontsize=14)

The settings stay changed after the call; matplotlib.rc_context restores them at the end of the block.

>>> with plt.rc_context():
...     set_font(fontfamily="DejaVu Sans", fontsize=14)
...     print(plt.rcParams["font.family"], plt.rcParams["font.size"])
['DejaVu Sans'] 14.0
yikit.visualize.with_custom_matplotlib_settings(fontfamily: str | None = None, fontsize: float = 13, restore: bool = True) → Callable[[F], F]#

Return a decorator that sets the font while the function runs.

The decorated function runs with the global font family and font size of matplotlib (rcParams["font.family"] and rcParams["font.size"]) set to fontfamily and fontsize. With restore=True the previous values of these two settings are restored when the function returns or raises; other settings changed inside the function are not restored.

Parameters:
  • fontfamily (str or None, default=None) – The font family. None means “Helvetica” on macOS and “DejaVu Sans” on other platforms (see set_font()).

  • fontsize (float, default=13) – The default font size in points.

  • restore (bool, default=True) – Whether to restore the previous font family and font size after the call. If False, the settings stay changed as with set_font().

Returns:

A decorator that wraps a function, keeping its name and docstring.

Return type:

Callable

See also

set_font

Set the font for the rest of the session.

Examples

>>> import matplotlib.pyplot as plt
>>> from yikit.visualize import with_custom_matplotlib_settings
>>> @with_custom_matplotlib_settings(fontfamily="DejaVu Sans", fontsize=14)
... def font_size():
...     return plt.rcParams["font.size"]
>>> with plt.rc_context({"font.size": 10}):
...     print(font_size(), plt.rcParams["font.size"])
14.0 10.0
yikit.visualize.yyplot(y_true: ArrayLike, y_pred: ArrayLike, *, labels: Sequence[str | None] | None = None, metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) → Axes#
yikit.visualize.yyplot(y_train: ArrayLike, y_pred_on_train: ArrayLike, y_test: ArrayLike, y_pred_on_test: ArrayLike, *, labels: Sequence[str] | None = ('train', 'test'), metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) → Axes
yikit.visualize.yyplot(y_train: ArrayLike, y_pred_on_train: ArrayLike, y_val: ArrayLike, y_pred_on_val: ArrayLike, y_test: ArrayLike, y_pred_on_test: ArrayLike, *, labels: Sequence[str | None] | None = ('train', 'val', 'test'), metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) → Axes
yikit.visualize.yyplot(*y_data: ArrayLike, labels: Sequence[str | None] | None = None, metrics: Sequence[Literal['r2', 'rmse', 'mae', 'mse']] = ('r2', 'rmse'), ax: Axes | None = None, alpha: float = 0.05) → Axes

Plot predicted against true values for one or more data sets.

Each (y_true, y_pred) pair is drawn as a scatter plot over the identity line \(y = x\), and the chosen regression metrics of each pair are written in the top-left corner. Typical use is to compare a model on its train, validation and test sets.

Parameters:
  • *y_data (array-like) – Consecutive pairs (y_true_0, y_pred_0, y_true_1, y_pred_1, ...) of true and predicted values. At least one pair is required, and the two arrays of a pair must have the same length.

  • labels (sequence of str or None, optional) – Name of each pair, shown in the legend and next to its metrics exactly as written: characters such as $, _, ^ and % are not read as mathtext. A pair labeled None or "" has no legend entry, and its metrics carry no label. If None (default), two pairs are labeled ("train", "test"), three pairs ("train", "val", "test"), and any other number of pairs is not labeled.

  • metrics (sequence of {"r2", "rmse", "mae", "mse"}, default=("r2", "rmse")) – Metrics written for each pair, in this order: the coefficient of determination, the root mean squared error, the mean absolute error and the mean squared error. For multi-output targets, RMSE is the mean of the RMSE of each output, as in scikit-learn.

  • ax (matplotlib.axes.Axes, optional) – Axes to draw on. Only this axes changes: the layout of its figure (such as the subplot parameters) is left as it is. If None (default), a new 3.6 x 3.6 inch figure is created and laid out with tight_layout.

  • alpha (float, default=0.05) – Margin added on each side of the range of all the values, as a fraction of that range, to set the limits of both axes.

Returns:

The axes that was drawn on.

Return type:

matplotlib.axes.Axes

Raises:

ValueError – If no pair or an odd number of arrays is given, if the length of labels differs from the number of pairs, or if metrics contains an unknown name. These are checked before anything is drawn, so no figure is created and ax is left untouched.

Notes

The legend shows the scatter plots labeled here together with the labeled artists already on ax. Labels are drawn as written unless rcParams["text.usetex"] is enabled, in which case LaTeX reads them.

Examples

Plot a single data set:

>>> import numpy as np
>>> from yikit.visualize import yyplot
>>> y_true = np.array([1.0, 2.0, 3.0])
>>> y_pred = np.array([0.8, 2.1, 2.9])
>>> ax = yyplot(y_true, y_pred)

Plot train and test data sets with the mean absolute error as well:

>>> y_train, y_pred_train = np.arange(5), np.arange(5) + 0.1
>>> y_test, y_pred_test = np.arange(5), np.arange(5) - 0.2
>>> ax = yyplot(y_train, y_pred_train, y_test, y_pred_test,
...             labels=("train", "test"), metrics=("r2", "rmse", "mae"))

Draw on one axes of an existing figure, whose layout is kept:

>>> import matplotlib.pyplot as plt
>>> fig, axes = plt.subplots(1, 2)
>>> ax = yyplot(y_true, y_pred, labels=["fold_1 (5% noise)"], ax=axes[0])
>>> ax is axes[0]
True
>>> plt.close("all")