{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9d1cbcd9",
   "metadata": {},
   "source": [
    "# Binary Classification Metrics Exercises\n",
    "\n",
    "## Exercise 1 — Basic Metrics\n",
    "\n",
    "A classifier produces the following confusion matrix for a test set of 100 samples:\n",
    "\n",
    "|                 | Predicted Positive | Predicted Negative |\n",
    "| --------------- | ------------------ | ------------------ |\n",
    "| Actual Positive | 40                 | 10                 |\n",
    "| Actual Negative | 20                 | 30                 |\n",
    "\n",
    "### Tasks\n",
    "\n",
    "1. Compute Accuracy, Precision, Recall (Sensitivity), Specificity, and F1-score.\n",
    "2. Interpret what each metric tells you about the model’s performance.\n",
    "3. Discuss how increasing the classification threshold might affect precision and recall.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8ecf444",
   "metadata": {},
   "source": [
    "## Exercise 2 — Comparing Two Models\n",
    "\n",
    "Two classifiers (A and B) yield the following confusion matrices on the same dataset of 200 samples\n",
    "\n",
    "| Model   |  TP |  FN |  FP |  TN |\n",
    "| ------- | --: | --: | --: | --: |\n",
    "| Model A |  70 |  30 |  20 |  80 |\n",
    "| Model B |  90 |  50 |  10 |  50 |\n",
    "\n",
    "### Tasks\n",
    "\n",
    "1. Compute the precision, recall, F1-score, and accuracy for both models.\n",
    "2. Which model would you prefer if false negatives are more costly _(e.g. disease detection)_?\n",
    "3. Which would you prefer if false positives are more costly _(e.g., fraud detection)_?\n",
    "4. Which has better overall balance _(based on F1)_?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74a62fe1",
   "metadata": {},
   "source": [
    "## Exercise 3 — ROC and AUC (Manual Points)\n",
    "\n",
    "A binary classifier outputs the following predicted probabilities and true labels for 8 samples:\n",
    "\n",
    "| Sample | True Label | Predicted Probability |\n",
    "| ------ | ---------: | --------------------: |\n",
    "| 1      |          1 |                  0.95 |\n",
    "| 2      |          0 |                  0.90 |\n",
    "| 3      |          1 |                  0.85 |\n",
    "| 4      |          0 |                  0.70 |\n",
    "| 5      |          1 |                  0.60 |\n",
    "| 6      |          0 |                  0.40 |\n",
    "| 7      |          1 |                  0.20 |\n",
    "| 8      |          0 |                  0.10 |\n",
    "\n",
    "### Tasks:\n",
    "\n",
    "1. Compute the TPR (Recall) and FPR at thresholds = {0.9, 0.8, 0.6, 0.4, 0.2, 0.1}.\n",
    "2. Plot the ROC curve (TPR vs FPR).\n",
    "3. Estimate the AUC (Area Under Curve) using the trapezoidal rule.\n",
    "4. Interpret what the AUC value means in terms of model performance\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fed62707",
   "metadata": {},
   "source": [
    "## Exercise 4 — Threshold Sensitivity\n",
    "\n",
    "A medical test gives the following results for 10 patients:\n",
    "\n",
    "| Patient | True | Score |\n",
    "| ------- | ---- | ----- |\n",
    "| P1      | 1    | 0.95  |\n",
    "| P2      | 1    | 0.92  |\n",
    "| P3      | 0    | 0.89  |\n",
    "| P4      | 1    | 0.75  |\n",
    "| P5      | 0    | 0.72  |\n",
    "| P6      | 1    | 0.65  |\n",
    "| P7      | 0    | 0.55  |\n",
    "| P8      | 0    | 0.40  |\n",
    "| P9      | 1    | 0.35  |\n",
    "| P10     | 0    | 0.10  |\n",
    "\n",
    "Tasks:\n",
    "\n",
    "1. For thresholds 0.5 and 0.8, compute confusion matrices.\n",
    "2. For each threshold, compute precision, recall, and F1-score.\n",
    "3. Plot how precision and recall change as the threshold increases (Precision–Recall trade-off).\n",
    "4. Explain why AUC-PR may be more informative than AUC-ROC for imbalanced datasets\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1657a40d",
   "metadata": {},
   "source": [
    "## Exercise 5 — Imbalanced Dataset Scenario\n",
    "\n",
    "Out of 10,000 emails:\n",
    "\n",
    "- 200 are spam (positive class)\n",
    "- 9,800 are not spam (negative class)\n",
    "\n",
    "A model flags 300 emails as spam, correctly identifying 150 of the 200 spams.\n",
    "\n",
    "### Tasks:\n",
    "\n",
    "1. Compute precision, recall, and F1-score.\n",
    "2. Compute accuracy.\n",
    "3. Explain why accuracy can be misleading in this scenario.\n",
    "4. Discuss which metric would be more informative (and why)\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
