Uncertainty estimation of class classifiers

• In a classifier, we not only want to know the class of a certain X, but also how certain it is.
  • Either decision_function or predict_proba (or both) are often implemented in classifiers.
    • The decision_function represents the confidence level. If the value is positive, it belongs to class A; if it is negative, it belongs to class B.
      • The range of values depends on the model, so it can be cumbersome.
      • In the case of multi-class classification, the confidence level for each class is provided, but the sum or scale is not specified.
    • predict_proba indicates the probability of X being in each class.
      • The sum of the values is 1.
• When a model is calibrated, it means that the confidence level expressed by the model is equal to the actual accuracy. #Class Classification