Fair Post-Processing

Definition

A derived classifier outputs $\tilde{Y}$ with $p_{ya}=P(\tilde{Y}=1\mid \hat{Y}=y,A=a)$, using only $\hat{Y}$ and $A$ (Hardt, Price, Srebro 2016).

Formula

$$\begin{array}{rl}{\mathrm{TPR}}_a(\tilde{Y})& =p_{1a}{\mathrm{TPR}}_a+p_{0a}(1-{\mathrm{TPR}}_a)\\ {\mathrm{FPR}}_a(\tilde{Y})& =p_{1a}{\mathrm{FPR}}_a+p_{0a}(1-{\mathrm{FPR}}_a)\end{array}$$

Example: lowering a TPR of 0.75 to 0.6 keeps each positive prediction with probability $q=0.8$.

Group $a$ can reach every point of the quadrilateral $(0,0)$, $(1,1)$, $({\mathrm{FPR}}_a,{\mathrm{TPR}}_a)$, $(1-{\mathrm{FPR}}_a,1-{\mathrm{TPR}}_a)$ on the ROC plane; equalized odds needs a common point. With scores, group-specific thresholds at the intersection of the ROC curves work. The price: lower accuracy, randomized decisions, explicit use of $A$. The alternatives are pre-processing (fix the data; removing $A$ fails because of proxies) and in-processing (training with a fairness constraint).

Appears in

• Lecture 9.1, pre-, in- and post-processing
• Lecture 9.1, derived classifier and ROC curves
• Lecture 9.1, exam-style ROC task
• Lecture 9.1, tradeoffs

Related concepts

• Fairness Criteria