Note: the current releases of this toolbox are a beta release, to test working with Haskell's, Python's, and R's code repositories.

Metrics provides implementations of various supervised machine learning evaluation metrics in the following languages:

• Python easy_install ml_metrics
• R install.packages("Metrics") from the R prompt
• Haskell cabal install Metrics
• MATLAB / Octave (clone the repo & run setup from the MATLAB command line)

For more detailed installation instructions, see the README for each implementation.

EVALUATION METRICS

TO IMPLEMENT

• F1 score
• Multiclass log loss
• Lift
• Average Precision for binary classification
• precision / recall break-even point
• cross-entropy
• True Pos / False Pos / True Neg / False Neg rates
• precision / recall / sensitivity / specificity
• mutual information

HIGHER LEVEL TRANSFORMATIONS TO HANDLE

• GroupBy / Reduce
• Weight individual samples or groups

PROPERTIES METRICS CAN HAVE

(Nonexhaustive and to be added in the future)

• Min or Max (optimize through minimization or maximization)

• Binary Classification

  • Scores predicted class labels
  • Scores predicted ranking (most likely to least likely for being in one class)
  • Scores predicted probabilities

• Multiclass Classification

  • Scores predicted class labels
  • Scores predicted probabilities

• Regression

• Discrete Rater Comparison (confusion matrix)