Bias Detection Score Calculator
ANA›Life Services Authority›National Calculator Authority›Bias Detection Score Calculator
.calc-container { max-width: 640px; margin: 2rem 0; padding: 1.5rem; background: #fff; border: 1px solid #ddd; border-radius: 8px; box-shadow: 0 1px 3px rgba(0,0,0,0.06); font-family: system-ui, -apple-system, sans-serif; } .calc-container h3 { font-family: Georgia, serif; font-size: 1.15rem; color: #1a1a1a; margin-bottom: 1rem; padding-bottom: 0.5rem; border-bottom: 2px solid var(--ac, #3d5a80); } .calc-row { display: flex; align-items: center; gap: 0.75rem; margin-bottom: 0.75rem; flex-wrap: wrap; } .calc-row label { min-width: 160px; font-size: 0.9rem; color: #333; font-weight: 500; } .calc-row input[type="number"], .calc-row select { flex: 1; min-width: 120px; max-width: 200px; padding: 0.5rem 0.6rem; border: 1px solid #ccc; border-radius: 4px; font-size: 0.9rem; font-family: system-ui, sans-serif; color: #1a1a1a; background: #fafaf8; } .calc-row input:focus, .calc-row select:focus { outline: none; border-color: var(--ac, #3d5a80); box-shadow: 0 0 0 2px rgba(26,74,138,0.12); } .calc-row .unit { font-size: 0.82rem; color: #888; min-width: 30px; } .calc-btn { display: inline-block; margin-top: 0.5rem; padding: 0.55rem 1.5rem; background: var(--ac, #3d5a80); color: #fff; border: none; border-radius: 4px; font-size: 0.9rem; font-weight: 600; cursor: pointer; font-family: system-ui, sans-serif; } .calc-btn:hover { opacity: 0.9; } .calc-result { margin-top: 1.25rem; padding: 1rem 1.25rem; background: #f0f6fc; border-left: 3px solid var(--ac, #3d5a80); border-radius: 0 6px 6px 0; display: none; } .calc-result.visible { display: block; } .calc-result-label { font-size: 0.78rem; text-transform: uppercase; letter-spacing: 0.06em; color: #666; margin-bottom: 0.25rem; } .calc-result-value { font-size: 1.6rem; font-weight: 700; color: var(--ac, #3d5a80); } .calc-result-detail { font-size: 0.85rem; color: #555; margin-top: 0.5rem; line-height: 1.5; } .calc-note { margin-top: 1rem; font-size: 0.8rem; color: #888; font-style: italic; } .calc-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 0.75rem; margin-top: 0.75rem; } .calc-grid-item { padding: 0.6rem 0.8rem; background: #f8f9fa; border-radius: 4px; border: 1px solid #eee; } .calc-grid-item .label { font-size: 0.75rem; color: #888; text-transform: uppercase; letter-spacing: 0.04em; } .calc-grid-item .value { font-size: 1.1rem; font-weight: 600; color: #1a1a1a; } @media (max-width: 720px) { .calc-row { flex-direction: column; align-items: flex-start; gap: 0.3rem; } .calc-row label { min-width: auto; } .calc-row input[type="number"], .calc-row select { max-width: 100%; width: 100%; } .calc-grid { grid-template-columns: 1fr; } } .calc-chart { margin: 1rem 0; text-align: center; } .calc-chart svg { max-width: 100%; height: auto; } .calc-chart-legend { display: flex; flex-wrap: wrap; justify-content: center; gap: 0.6rem 1.2rem; margin-top: 0.6rem; font-size: 0.8rem; color: #555; } .calc-chart-legend span { display: inline-flex; align-items: center; gap: 0.3rem; } .calc-chart-legend i { display: inline-block; width: 10px; height: 10px; border-radius: 2px; font-style: normal; } .calc-related { max-width: 640px; margin: 2rem 0 1rem; padding: 1.25rem 1.5rem; background: #f8f9fa; border: 1px solid #e8e8e8; border-radius: 8px; } .calc-related h3 { font-family: Georgia, serif; font-size: 1rem; color: #1a1a1a; margin: 0 0 0.75rem; padding-bottom: 0.4rem; border-bottom: 2px solid var(--ac, #3d5a80); } .calc-related-list { list-style: none; padding: 0; margin: 0 0 0.75rem; display: grid; grid-template-columns: 1fr 1fr; gap: 0.4rem 1.5rem; } .calc-related-list li a { font-size: 0.88rem; color: var(--ac, #3d5a80); text-decoration: none; } .calc-related-list li a:hover { text-decoration: underline; } .calc-browse-all { margin: 0.5rem 0 0; font-size: 0.9rem; font-weight: 600; } .calc-browse-all a { color: var(--ac, #3d5a80); text-decoration: none; } .calc-browse-all a:hover { text-decoration: underline; } @media (max-width: 720px) { .calc-related-list { grid-template-columns: 1fr; } }
Bias Detection Score Calculator
Calculates a composite Bias Detection Score (BDS) using demographic parity difference, equalized odds difference, and calibration error — three foundational fairness metrics used in algorithmic auditing and AI ethics research.
True Positive Rate – Group A (TPR_A): (0 to 1)
True Positive Rate – Group B (TPR_B): (0 to 1)
False Positive Rate – Group A (FPR_A): (0 to 1)
False Positive Rate – Group B (FPR_B): (0 to 1)
Positive Prediction Rate – Group A (PR_A): (0 to 1)
Positive Prediction Rate – Group B (PR_B): (0 to 1)
Calibration Score – Group A (CAL_A): Mean predicted probability (0 to 1)
Calibration Score – Group B (CAL_B): Mean predicted probability (0 to 1)
Weight – Demographic Parity (w₁): Default 0.333
Weight – Equalized Odds (w₂): Default 0.333
Weight – Calibration (w₃): Default 0.334
Calculate Bias Detection Score
Results will appear here.
function biaCalc() { var resultDiv = document.getElementById('bia-result');
// --- Parse inputs --- var tprA = parseFloat(document.getElementById('bia-tpr-a').value); var tprB = parseFloat(document.getElementById('bia-tpr-b').value); var fprA = parseFloat(document.getElementById('bia-fpr-a').value); var fprB = parseFloat(document.getElementById('bia-fpr-b').value); var prA = parseFloat(document.getElementById('bia-pr-a').value); var prB = parseFloat(document.getElementById('bia-pr-b').value); var calA = parseFloat(document.getElementById('bia-cal-a').value); var calB = parseFloat(document.getElementById('bia-cal-b').value); var w1 = parseFloat(document.getElementById('bia-w1').value); var w2 = parseFloat(document.getElementById('bia-w2').value); var w3 = parseFloat(document.getElementById('bia-w3').value);
// --- Validation --- var fields = [tprA, tprB, fprA, fprB, prA, prB, calA, calB, w1, w2, w3]; for (var i = 0; i ⚠ Please fill in all fields with valid numbers.'; return; } }
var rateFields = [tprA, tprB, fprA, fprB, prA, prB, calA, calB]; for (var j = 0; j 1) { resultDiv.innerHTML = '⚠ All rate values must be between 0 and 1.'; return; } }
if (w1 ⚠ Weights must be non-negative.'; return; }
var wSum = w1 + w2 + w3; if (Math.abs(wSum - 1.0) > 0.01) { resultDiv.innerHTML = '⚠ Weights must sum to 1.0 (current sum: ' + wSum.toFixed(3) + ').'; return; }
if (tprA ⚠ Warning: TPR_A is less than FPR_A for Group A — this implies a model performing below random chance for Group A. Please verify inputs.'; return; } if (tprB ⚠ Warning: TPR_B is less than FPR_B for Group B — this implies a model performing below random chance for Group B. Please verify inputs.'; return; }
// --- Core Metric Calculations ---
// 1. Demographic Parity Difference (DPD) // DPD = |PR_A - PR_B| var dpd = Math.abs(prA - prB);
// 2. Equalized Odds Difference (EOD) // EOD = 0.5 * (|TPR_A - TPR_B| + |FPR_A - FPR_B|) var eod = 0.5 * (Math.abs(tprA - tprB) + Math.abs(fprA - fprB));
// 3. Calibration Difference (CD) // CD = |CAL_A - CAL_B| var cd = Math.abs(calA - calB);
// 4. Composite Bias Detection Score (BDS) // BDS = w1 * DPD + w2 * EOD + w3 * CD // BDS ∈ [0, 1]; 0 = no bias, 1 = maximum bias var bds = w1 * dpd + w2 * eod + w3 * cd;
// 5. Bias Severity Classification var severity, sevColor, sevIcon; if (bds Low'; if (val Moderate'; return 'High'; }
- // 7. Disparate Impact Ratio (DIR) — supplementary
- // DIR = PR_B / PR_A (if PR_A > 0); threshold 0) {
- var dir = prB / prA;
- dirText = dir.toFixed(4);
- dirNote = dir (Below 0.80 — adverse impact indicated per 4/5ths rule)'
- ' (Above 0.80 — within acceptable range)'; }
// --- Output --- resultDiv.innerHTML = '### Bias Detection Score Results ' +
'' + '' + 'Metric' + 'Value' + 'Severity' + '' + '' + 'Demographic Parity Difference (DPD)' + '' + dpd.toFixed(4) + '' + '' + metricLabel(dpd) + '' + 'Equalized Odds Difference (EOD)' + '' + eod.toFixed(4) + '' + '' + metricLabel(eod) + '' + 'Calibration Difference (CD)' + '' + cd.toFixed(4) + '' + '' + metricLabel(cd) + '' + 'Disparate Impact Ratio (DIR)' + '' + dirText + dirNote + '' + '' + '' +
'' + '' + sevIcon + ' Composite BDS = ' + bds.toFixed(4) + ' / 1.0000' + '' + '' + 'Severity: ' + severity + '' + '' + '' +
'' + 'Weighted breakdown: ' + 'BDS = (' + w1.toFixed(3) + ' × ' + dpd.toFixed(4) + ') + ' + '(' + w2.toFixed(3) + ' × ' + eod.toFixed(4) + ') + ' + '(' + w3.toFixed(3) + ' × ' + cd.toFixed(4) + ') = ' + bds.toFixed(4) + '' + '
'; }
#### Formulas Used
1. Demographic Parity Difference (DPD)
DPD = |PR_A − PR_B|
Measures whether both groups receive positive predictions at equal rates, regardless of ground truth. A DPD of 0 indicates perfect demographic parity.
2. Equalized Odds Difference (EOD)
EOD = 0.5 × (|TPR_A − TPR_B| + |FPR_A − FPR_B|)
Averages the absolute differences in True Positive Rates and False Positive Rates across groups. Proposed by Hardt et al. (2016) as a joint constraint on error rates.
3. Calibration Difference (CD)
CD = |CAL_A − CAL_B|
Compares mean predicted probabilities between groups. A well-calibrated model should produce similar mean scores for both groups when base rates are equal.
4. Composite Bias Detection Score (BDS)
BDS = w₁ × DPD + w₂ × EOD + w₃ × CD
A weighted composite score ∈ [0, 1]. Default equal weights (w₁ = w₂ = w₃ ≈ 0.333) treat all three fairness criteria equally. Weights must sum to 1.
5. Disparate Impact Ratio (DIR) — Supplementary
DIR = PR_B / PR_A
Per the EEOC 4/5ths (80%) rule: a DIR below 0.80 indicates potential adverse impact against Group B. DIR is not included in BDS but is reported as a supplementary indicator.
Severity Thresholds (BDS)
BDS Range Classification
0.00 – 0.049Minimal / No Detectable Bias 0.05 – 0.099Low Bias 0.10 – 0.199Moderate Bias 0.20 – 0.349High Bias ≥ 0.35Severe Bias
#### Assumptions & References
- Group A is treated as the reference group (e.g., majority or privileged group); Group B is the comparison group (e.g., minority or protected group). The choice of reference group affects DIR but not BDS.
- BDS assumes equal base rates across groups. If base rates differ substantially, calibration difference may reflect real-world prevalence rather than model bias — interpret CD with caution in such cases.
- This calculator evaluates group fairness (statistical parity), not individual fairness. These two notions can be in tension (Chouldechova, 2017).
- Hardt, M., Price, E., & Srebro, N. (2016). Equality of Opportunity in Supervised Learning. NeurIPS 2016.
- Chouldechova, A. (2017). Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2), 153–163.
- EEOC Uniform Guidelines (1978). 4/5ths (80%) rule for adverse impact — Disparate Impact Ratio threshold of 0.80.
- Barocas, S., Hardt, M., & Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities. MIT Press.
More Calculators
- HVAC Equipment Lifespan and Replacement Cost Calculator
- HVAC Equipment Sizing Calculator
- Texas Climate Zone Heat Load Calculator
- Heat Pump vs. Fossil Fuel Cost Comparison Calculator
- Home Insulation R-Value Savings Calculator
- Furnace Sizing Calculator for Utah Altitude Adjustment
Read Next
Study Time Planner Authority Network America › Life Services Authority › National Calculator Authority .calc-container { max-width: 640px;...