Question

Question 3 30 pts Given the following predictive results: Sample ID (id) Predicted Probability (prob) Sample Label (label) 1 0.9 1 0.8 0 N 3 0.3 1 4 0.6 o 5 0.4 o 6 0.7 1 7 0.1 0 8 0.2 1 (10 Pts.). Suppose 0.6 is a cutoff, then the sample is predicted to be 1 if its probability >=0.6, and o otherwise. Please compute tpr and fpr. . (20 Pts.). Please compute its AUC manually. You need to show every pair of (tpr, fpr) as ROC and then compute the AUC.

Answer 1

Sample TD Predicted Probability Sample Label Сprob) -- Уpred y ŷ (0.6) 1 0.9 2 0-8 3 0-3 0 0.6 0 4 5 (0.9>=0.6) (0.8>=0,6) (0.3870.6) (0-6)=0-6) (0.4 0-6) (0-47-0.6) COT*0.6) (0.270-6) 0 6 0.4 0.7 0.1 7 o 8 0-2 0 Suppose 0.6 is a cutoff, then the sample e's predicted to be 1 if its probability >=0.6, and o otherwise. True Positive True Positive Rate Tree positive + False Negative ТР TPR = TP+FN

True Positive :- Both values are 1 (y and ŷ False Negative Y is I and ŷ is 0. Consider Y and TPR = TP=2 (Sample IDs 1 and 6) FN = 2 ( Sample IDs 3 and 8) 4 TPR = 0.5 2+2 2 False Positive Rate False Positive False Positive + True Negative FP FPR FP+TN

False positive: Y is o and 9 is 1 True Negative : Both Y and ŷ are o. FPR = 2+2 FP=2 (sample IDs & and 4) TN = 2 ( Sample IDs 5 and 7) FPR = 0.5 Computing AUC :- Let us consider throughputs as (0,0-2,0-4,0-6, 0-8] Comparison is Ipred >= 9 y Spred ŷ (o) ŷ co.2) § 10-4) § (0-6) 9 (0.8) ŷ c) 0.9 O 0 0 0-3 0 0 0 0 0.6 0 6 1 0 0.4 0.7 o Ooj 0 0 0 0 0 0 0 0 Conisdering y and ? :- Throughput (Th) = 0 (1,1) TPR = 4 =1 4to FPR = 4 4to Th= 0.2 = (1,007) 3 FPR = TPR= 4 440 = 3 4 = 0.7 3+1 Th=0.4 (0.5,0:7) 3 TPR= = 0.5 FPR -0.7 2+2 3+1

Th=0.6 (0.3, 0.5) 2 TPR = it = 0.3 FPR= =0.5 142 2+2 Th=008 (0-2,0.2) 0.2 FPR = TPR = 1+3 1+3 Th= 1 (0,0) 0 TPR = 2=0 FPR =0 o+4 014 (0.7, 1) X, Y (fpr, tpr) All pairs of pr. fpr) are (1,1) (1,1) (1,007) Y-axis X-axis (0.5, 0.7) y (0-7,0.5) (0-3,0.5) (0.5,0.3) (0:2,0-2) (TPR) (0,0) = (0,0) - (0-2,0-2) y-axis CO-7,1) (11) 0.99 0-8. 0.7 Co-40-5) 0.50 (0.5,03) 0.41 0.3 .C0.2, 0:2) 0° 2, AUC (00) of 0.2 0.3 0.4 0.5 0.6 of o8 og i X-anis (FPR)
Summary: We have seen how to calculate TPR and FPR. Considered some threshold values to calculate TPR and FPR and noted down every pair of (TPR, FPR). Represented FPR on X-axis and TPR on Y-axis