本站消息

  本站每日ip已达10000,出租广告位,位置价格可谈,需要合作请联系站长


+关注
已关注

分类  

暂无分类

标签  

暂无标签

日期归档  

2019-07(6)

2019-08(95)

2019-09(107)

2019-10(11)

2019-11(4)

ML之模型评价指标(损失函数):基于不同机器学习框架(sklearn/TF)下算法的模型评估函数(Scoring/metrics)集合(仅代码实现)

发布于2019-08-07 12:40     阅读(593)     评论(0)     点赞(3)     收藏(2)


版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/qq_41185868/article/details/91531343

ML之模型评价指标:基于不同机器学习框架(sklearn/TF)下算法的模型评估函数(Scoring/metrics)集合(仅代码实现)

 

 

 

 

目录

单个评价指标各种框架下实现

1、回归问题

MSE函数:DIY自定义

CrVa交叉熵函数

EVS解释方差分数

MAE平均绝对误差

MSE均方误差

RMSE均方根误差

MSLE均方对数误差

MeAE中位数绝对误差

R^2决定系数

Adjusted_R2校正决定系数

2、分类问题

sklearn综合

1、回归问题

1.1、metrics

2、分类问题

2.1、Scoring

 


 

 

 

 

 

相关文章
ML之模型评价指标:基于不同机器学习框架(sklearn/TF)下算法的模型评估函数(Scoring/metrics)集合(仅代码实现)
ML之LF:机器学习中常用的损失函数案例代码实现
ML之LF:损失函数——回归预测问题中评价指标(常用的误差度量方法,MSE/RMSE/MAE)简介、使用方法、代码实现、案例应用之详细攻略

单个评价指标各种框架下实现

1、回归问题

ML之LF:损失函数——回归预测问题中评价指标(常用的误差度量方法,MSE/RMSE/MAE)简介、使用方法、代码实现、案例应用之详细攻略

MSE函数:DIY自定义

(1)、MSE函数

  1. def mean_squared_error(y, t):
  2. return 0.5 * np.sum((y-t)**2)

(2)、综合案例:自定义实现求MSE、RMSE、MAE,比较MSE与目标方差。

  1. #综合案例:自定义实现求MSE、RMSE、MAE,比较MSE与目标方差
  2. target = [1.5, 2.1, 3.3, -4.7, -2.3, 0.75]
  3. prediction = [0.5, 1.5, 2.1, -2.2, 0.1, -0.5]
  4. #for循环求出列表的error
  5. error = []
  6. for i in range(len(target)):
  7. error.append(target[i] - prediction[i])
  8. print("Errors ", error)
  9. #自定义求MSE:
  10. #for循环实现计算每个元素的SE、AE:calculate the squared errors and absolute value of errors
  11. squaredError = []
  12. absError = []
  13. for val in error:
  14. squaredError.append(val*val)
  15. absError.append(abs(val))
  16. print("Squared Error", squaredError)
  17. print("Absolute Value of Error", absError)
  18. print("MSE = ", sum(squaredError)/len(squaredError)) #综合计算MSE
  19. #自定义求RMSE、MAE
  20. from math import sqrt
  21. print("RMSE = ", sqrt(sum(squaredError)/len(squaredError)))
  22. print("MAE = ", sum(absError)/len(absError))
  23. #比较MSE与目标方差
  24. targetDeviation = []
  25. targetMean = sum(target)/len(target)
  26. for val in target:
  27. targetDeviation.append((val - targetMean)*(val - targetMean))
  28. print("Target Variance = ", sum(targetDeviation)/len(targetDeviation)) #输出目标方差
  29. print("Target Standard Deviation = ", sqrt(sum(targetDeviation)/len(targetDeviation))) #输出目标标准偏差(方差平方根)

 

CrVa交叉熵函数

(1)、cross_entropy_error()函数

  1. def cross_entropy_error(y, t):
  2. if y.ndim == 1:
  3. t = t.reshape(1, t.size)
  4. y = y.reshape(1, y.size)
  5. # 监督数据是one-hot-vector的情况下,转换为正确解标签的索引
  6. if t.size == y.size:
  7. t = t.argmax(axis=1)
  8. batch_size = y.shape[0]
  9. return -np.sum(np.log(y[np.arange(batch_size), t] + 1e-7)) / batch_size

 

EVS解释方差分数

explained_variance_score()函数:

  1. #explained_variance_score()函数:解释方差分数
  2. from sklearn.metrics import explained_variance_score
  3. y_true = [3, -0.5, 2, 7]
  4. y_pred = [2.5, 0.0, 2, 8]
  5. EVS01 = explained_variance_score(y_true, y_pred)
  6. print(EVS01)
  7. y_true = [[0.5, 1], [-1, 1], [7, -6]]
  8. y_pred = [[0, 2], [-1, 2], [8, -5]]
  9. EVS02 = explained_variance_score(y_true, y_pred, multioutput='raw_values')
  10. print(EVS02)
  11. EVS03 = explained_variance_score(y_true, y_pred, multioutput=[0.3, 0.7])
  12. print(EVS03)

MAE平均绝对误差

mean_absolute_error()

  1. #mean_absolute_error():平均绝对误差
  2. from sklearn.metrics import mean_absolute_error
  3. y_true = [3, -0.5, 2, 7]
  4. y_pred = [2.5, 0.0, 2, 8]
  5. MAE01 = mean_absolute_error(y_true, y_pred)
  6. print(MAE01)
  7. y_true = [[0.5, 1], [-1, 1], [7, -6]]
  8. y_pred = [[0, 2], [-1, 2], [8, -5]]
  9. MAE02 = mean_absolute_error(y_true, y_pred)
  10. print(MAE02)
  11. MAE03 = mean_absolute_error(y_true, y_pred, multioutput='raw_values')
  12. MAE04 = mean_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
  13. print(MAE03)
  14. print(MAE04)

MSE均方误差

mean_squared_error()

  1. #mean_squared_error():均方误差
  2. from sklearn.metrics import mean_squared_error
  3. y_true = [3, -0.5, 2, 7]
  4. y_pred = [2.5, 0.0, 2, 8]
  5. MSE01 = mean_squared_error(y_true, y_pred)
  6. print(MSE01)
  7. y_true = [[0.5, 1], [-1, 1], [7, -6]]
  8. y_pred = [[0, 2], [-1, 2], [8, -5]]
  9. MSE02 = mean_squared_error(y_true, y_pred)
  10. print(MSE02)

RMSE均方根误差

  1. #Root Mean Squared Error均方误差
  2. from sklearn.metrics import mean_squared_error
  3. import numpy as np
  4. y_true = [3, -0.5, 2, 7]
  5. y_pred = [2.5, 0.0, 2, 8]
  6. MSE01 = mean_squared_error(y_true, y_pred)
  7. RMSE01 = np.sqrt(MSE01 )
  8. print(RMSE01)

 

 

 

MSLE均方对数误差

mean_squared_log_error():均方对数误差

  1. #mean_squared_log_error():均方对数误差
  2. from sklearn.metrics import mean_squared_log_error
  3. y_true = [3, 5, 2.5, 7]
  4. y_pred = [2.5, 5, 4, 8]
  5. MSLE01 = mean_squared_log_error(y_true, y_pred)
  6. print(MSLE01)
  7. y_true = [[0.5, 1], [1, 2], [7, 6]]
  8. y_pred = [[0.5, 2], [1, 2.5], [8, 8]]
  9. MSLE02 = mean_squared_log_error(y_true, y_pred)
  10. print(MSLE02)

MeAE中位数绝对误差

median_absolute_error()

  1. #MeAE中位数绝对误差
  2. from sklearn.metrics import median_absolute_error
  3. y_true = [3, -0.5, 2, 7]
  4. y_pred = [2.5, 0.0, 2, 8]
  5. MeAE01 = median_absolute_error(y_true, y_pred)
  6. print(MeAE01)

R^2决定系数

r2_score()

  1. #R^2决定系数:
  2. from sklearn.metrics import r2_score
  3. y_true = [3, -0.5, 2, 7]
  4. y_pred = [2.5, 0.0, 2, 8]
  5. r2_score01 = r2_score(y_true, y_pred)
  6. print(r2_score01)
  7. y_true = [[0.5, 1], [-1, 1], [7, -6]]
  8. y_pred = [[0, 2], [-1, 2], [8, -5]]
  9. r2_score02 = r2_score(y_true, y_pred, multioutput='variance_weighted')
  10. print(r2_score02)
  11. y_true = [[0.5, 1], [-1, 1], [7, -6]]
  12. y_pred = [[0, 2], [-1, 2], [8, -5]]
  13. r2_score03 = r2_score(y_true, y_pred, multioutput='uniform_average')
  14. print(r2_score03)
  15. r2_score04 = r2_score(y_true, y_pred, multioutput='raw_values')
  16. r2_score05 = r2_score(y_true, y_pred, multioutput=[0.3, 0.7])
  17. print(r2_score04)
  18. print(r2_score05)

Adjusted_R2校正决定系数

  1. #Adjusted_R2校正决定系数:
  2. import numpy as np
  3. from sklearn.metrics import r2_score
  4. y_true = [[0.5, 1], [0.1, 1], [7, 6], [7.5, 6.5]]
  5. y_pred = [[0, 2], [0.1, 2], [8, 5], [7.2, 6.2]]
  6. y_true_array = np.array([[0.5, 1], [0.1, 1], [7, 6], [7.5, 6.5]])
  7. n=y_true_array.shape[0] #样本数量
  8. p=y_true_array.shape[1] #特征数量
  9. print(n,p)
  10. r2_score01 = r2_score(y_true, y_pred, multioutput='variance_weighted')
  11. print(r2_score01)
  12. Adj_r2_score01 = 1-( (1-r2_score01)*(n-1) ) / (n-p-1)
  13. print(Adj_r2_score01)

 

2、分类问题

 

 

 

sklearn综合

1、回归问题

1.1、metrics

metrics模块还提供为其他目的而实现的预测误差评估函数
– 分类任务的评估函数如表所示,其他任务评估函数请见:https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics

Classification metrics

See the Classification metrics section of the user guide for further details.

metrics.accuracy_score(y_true, y_pred[, …]) Accuracy classification score.
metrics.auc(x, y[, reorder]) Compute Area Under the Curve (AUC) using the trapezoidal rule
metrics.average_precision_score(y_true, y_score) Compute average precision (AP) from prediction scores
metrics.balanced_accuracy_score(y_true, y_pred) Compute the balanced accuracy
metrics.brier_score_loss(y_true, y_prob[, …]) Compute the Brier score.
metrics.classification_report(y_true, y_pred) Build a text report showing the main classification metrics
metrics.cohen_kappa_score(y1, y2[, labels, …]) Cohen’s kappa: a statistic that measures inter-annotator agreement.
metrics.confusion_matrix(y_true, y_pred[, …]) Compute confusion matrix to evaluate the accuracy of a classification
metrics.f1_score(y_true, y_pred[, labels, …]) Compute the F1 score, also known as balanced F-score or F-measure
metrics.fbeta_score(y_true, y_pred, beta[, …]) Compute the F-beta score
metrics.hamming_loss(y_true, y_pred[, …]) Compute the average Hamming loss.
metrics.hinge_loss(y_true, pred_decision[, …]) Average hinge loss (non-regularized)
metrics.jaccard_score(y_true, y_pred[, …]) Jaccard similarity coefficient score
metrics.log_loss(y_true, y_pred[, eps, …]) Log loss, aka logistic loss or cross-entropy loss.
metrics.matthews_corrcoef(y_true, y_pred[, …]) Compute the Matthews correlation coefficient (MCC)
metrics.multilabel_confusion_matrix(y_true, …) Compute a confusion matrix for each class or sample
metrics.precision_recall_curve(y_true, …) Compute precision-recall pairs for different probability thresholds
metrics.precision_recall_fscore_support(…) Compute precision, recall, F-measure and support for each class
metrics.precision_score(y_true, y_pred[, …]) Compute the precision
metrics.recall_score(y_true, y_pred[, …]) Compute the recall
metrics.roc_auc_score(y_true, y_score[, …]) Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.
metrics.roc_curve(y_true, y_score[, …]) Compute Receiver operating characteristic (ROC)
metrics.zero_one_loss(y_true, y_pred[, …]) Zero-one classification loss.

 

2、分类问题

2.1、Scoring

   用交叉验证(cross_val_scoreGridSearchCV)评价模型性能时,用scoring参数定义评价指标。
(1)、评价指标是越高越好,因此用一些损失函数当评价指标时, 需要再加负号,如neg_log_lossneg_mean_squared_error 。详见 https://scikit-learn.org/stable/modules/model_evaluation.html#log-loss

Scoring

Function

Comment

Classification

‘accuracy’

metrics.accuracy_score

 

 

‘average_precision’

 

metrics.average_precision_score

 

‘f1’

metrics.f1_score

for binary targets

‘f1_micro’

metrics.f1_score

micro-averaged

‘f1_macro’

metrics.f1_score

macro-averaged

‘f1_weighted’

metrics.f1_score

weighted average

‘f1_samples’

metrics.f1_score

by multilabel sample

 

‘neg_log_loss’

 

metrics.log_loss

requires predict_proba sup port

‘precision’ etc.

metrics.precision_score

suffixes apply as with ‘f1’

‘recall’ etc.

metrics.recall_score

suffixes apply as with ‘f1’

‘roc_auc’

metrics.roc_auc_score

 

Clustering

‘adjusted_rand_score’

metrics.adjusted_rand_score

 

Regression

‘neg_mean_absolute_error’

metrics.mean_absolute_error

 

‘neg_mean_squared_error’

metrics.mean_squared_error

 

 

‘neg_median_absolute_error’

 

metrics.median_absolute_error

 

‘r2’

metrics.r2_score

 

 

 

 

 

 

 

 

 



所属网站分类: 技术文章 > 博客

作者:3434erer

链接: https://www.pythonheidong.com/blog/article/10916/

来源: python黑洞网

任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任

3 0
收藏该文
已收藏

评论内容:(最多支持255个字符)