yikit.metrics package#
Evaluation metrics that scikit-learn does not provide.
log_likelihood scores NGBoost models and is available only when ngboost
is installed. The root mean squared error was removed in 0.4.0; use
scikit-learn’s implementation instead.
- yikit.metrics.log_likelihood(estimator, X, y)#
Return the mean log-likelihood of
yunder an NGBoost model.The value is the average, over the samples, of the log of the predicted probability density (or mass) of each target for a regressor, or of the predicted probability of the true class for a classifier. It does not depend on the scoring rule (
Score) the model was trained with. Higher is better.- Parameters:
estimator (ngboost.NGBRegressor or ngboost.NGBClassifier) – A fitted NGBoost regressor or classifier.
X (array-like of shape (n_samples, n_features)) – Input samples.
y (array-like of shape (n_samples,)) – True target values or class labels.
- Returns:
The mean log-likelihood. For a model trained with the default
Score=LogScore, it equals-estimator.score(X, y).- Return type:
float
- Raises:
TypeError – If
estimatoris not anNGBRegressoror anNGBClassifier.ValueError – If ngboost has no LogScore implementation for the model’s distribution.
Notes
estimator.scorereturns the mean of the scoring rule the model was trained with, so-estimator.score(X, y)is the log-likelihood only forLogScore(forCRPScoreit is the negated mean CRPS). This function instead evaluates ngboost’s LogScore implementation for the model’s distribution at the predicted parameters, with equal weights for all samples. For the distributions that ngboost implements with censored scores only (such asLogNormalandExponential), every target is treated as observed (not censored).Examples
>>> from ngboost import NGBRegressor >>> from ngboost.scores import CRPScore >>> from sklearn.datasets import make_regression >>> from yikit.metrics import log_likelihood >>> X, y = make_regression(n_samples=50, random_state=334) >>> ngb = NGBRegressor(Score=CRPScore, verbose=False) >>> ngb = ngb.fit(X, y) >>> log_likelihood(ngb, X, y)