Mathematical Functions of Common Metrics

Introduction

Mathematical functions of common metrics for model evaluation in the fields of NLP, CV, and time series analysis.

Time Series Metrics

Mean Absolute Error (MAE)

MAE measures the average magnitude of errors in a set of predictions, without considering their direction.

params:
• : Number of observations
• : Actual value at time
• : Predicted value at time

and the params of MAE function are same with the following functions in Time Series Metrics part.

Mean Squared Error (MSE)

MSE squares the errors before averaging, giving more weight to larger errors.

Root Mean Squared Error (RMSE)

RMSE is the square root of MSE and provides error in the same units as the original data.

Mean Absolute Percentage Error (MAPE)

MAPE expresses accuracy as a percentage, which is useful for understanding error relative to the size of the actual values.

NLP Metrics

Accurary

Measures the overall correctness of the model’s predictions.

params:
• TP: True Positives (correctly predicted positive cases)
• FP: False Positives (incorrectly predicted positive cases)
• TN: True Negatives (actual negative cases correctly predicted as negative)
• FN: False Negatives (actual positive cases incorrectly predicted as negative)

Precision

Precision indicates the accuracy of positive predictions.

Recall (Sensitivity)

Recall measures the model’s ability to identify all relevant instances.

F1 Score

The F1 Score is the harmonic mean of precision and recall, balancing the two metrics.

BLEU Score (Bilingual Evaluation Understudy)

BLEU measures the quality of machine-generated text against reference texts, focusing on n-gram matches.

params:
• : Brevity Penalty
• : Precision for n-grams
• : Weight for each n-gram precision (often uniform)

CV Metrics

Intersection over Union (IoU)

IoU measures the overlap between the predicted and actual bounding boxes, commonly used in object detection tasks.

params:
• : Predicted bounding box area
• : Ground truth bounding box area

Mean Average Precision (mAP)

mAP summarizes the precision-recall curve and provides a single measure of performance across different classes.

params:
• : Total number of categories involved
• : Average Precision for recall

Pixel Accuracy

Measures the percentage of correctly classified pixels in an image segmentation task.

params:
• Correct Pixels: Pixels classified correctly.
• Total Pixels: Total number of pixels in the image.

Structural Similarity Index (SSIM)

SSIM measures the similarity between two images, considering luminance, contrast, and structure.

params:
• : Two images being compared
• : Mean values of the images
• : Variances of the images
• : Covariance of the images
• : Constants to stabilize the division

References

Math functions of Common Metrics