Information Entropy & Redundancy Calculator

ANA›Life Services Authority›National Calculator Authority›Information Entropy & Redundancy Calculator

.calc-container { max-width: 640px; margin: 2rem 0; padding: 1.5rem; background: #fff; border: 1px solid #ddd; border-radius: 8px; box-shadow: 0 1px 3px rgba(0,0,0,0.06); font-family: system-ui, -apple-system, sans-serif; } .calc-container h3 { font-family: Georgia, serif; font-size: 1.15rem; color: #1a1a1a; margin-bottom: 1rem; padding-bottom: 0.5rem; border-bottom: 2px solid var(--ac, #3d5a80); } .calc-row { display: flex; align-items: center; gap: 0.75rem; margin-bottom: 0.75rem; flex-wrap: wrap; } .calc-row label { min-width: 160px; font-size: 0.9rem; color: #333; font-weight: 500; } .calc-row input[type="number"], .calc-row select { flex: 1; min-width: 120px; max-width: 200px; padding: 0.5rem 0.6rem; border: 1px solid #ccc; border-radius: 4px; font-size: 0.9rem; font-family: system-ui, sans-serif; color: #1a1a1a; background: #fafaf8; } .calc-row input:focus, .calc-row select:focus { outline: none; border-color: var(--ac, #3d5a80); box-shadow: 0 0 0 2px rgba(26,74,138,0.12); } .calc-row .unit { font-size: 0.82rem; color: #888; min-width: 30px; } .calc-btn { display: inline-block; margin-top: 0.5rem; padding: 0.55rem 1.5rem; background: var(--ac, #3d5a80); color: #fff; border: none; border-radius: 4px; font-size: 0.9rem; font-weight: 600; cursor: pointer; font-family: system-ui, sans-serif; } .calc-btn:hover { opacity: 0.9; } .calc-result { margin-top: 1.25rem; padding: 1rem 1.25rem; background: #f0f6fc; border-left: 3px solid var(--ac, #3d5a80); border-radius: 0 6px 6px 0; display: none; } .calc-result.visible { display: block; } .calc-result-label { font-size: 0.78rem; text-transform: uppercase; letter-spacing: 0.06em; color: #666; margin-bottom: 0.25rem; } .calc-result-value { font-size: 1.6rem; font-weight: 700; color: var(--ac, #3d5a80); } .calc-result-detail { font-size: 0.85rem; color: #555; margin-top: 0.5rem; line-height: 1.5; } .calc-note { margin-top: 1rem; font-size: 0.8rem; color: #888; font-style: italic; } .calc-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 0.75rem; margin-top: 0.75rem; } .calc-grid-item { padding: 0.6rem 0.8rem; background: #f8f9fa; border-radius: 4px; border: 1px solid #eee; } .calc-grid-item .label { font-size: 0.75rem; color: #888; text-transform: uppercase; letter-spacing: 0.04em; } .calc-grid-item .value { font-size: 1.1rem; font-weight: 600; color: #1a1a1a; } @media (max-width: 720px) { .calc-row { flex-direction: column; align-items: flex-start; gap: 0.3rem; } .calc-row label { min-width: auto; } .calc-row input[type="number"], .calc-row select { max-width: 100%; width: 100%; } .calc-grid { grid-template-columns: 1fr; } } .calc-chart { margin: 1rem 0; text-align: center; } .calc-chart svg { max-width: 100%; height: auto; } .calc-chart-legend { display: flex; flex-wrap: wrap; justify-content: center; gap: 0.6rem 1.2rem; margin-top: 0.6rem; font-size: 0.8rem; color: #555; } .calc-chart-legend span { display: inline-flex; align-items: center; gap: 0.3rem; } .calc-chart-legend i { display: inline-block; width: 10px; height: 10px; border-radius: 2px; font-style: normal; } .calc-related { max-width: 640px; margin: 2rem 0 1rem; padding: 1.25rem 1.5rem; background: #f8f9fa; border: 1px solid #e8e8e8; border-radius: 8px; } .calc-related h3 { font-family: Georgia, serif; font-size: 1rem; color: #1a1a1a; margin: 0 0 0.75rem; padding-bottom: 0.4rem; border-bottom: 2px solid var(--ac, #3d5a80); } .calc-related-list { list-style: none; padding: 0; margin: 0 0 0.75rem; display: grid; grid-template-columns: 1fr 1fr; gap: 0.4rem 1.5rem; } .calc-related-list li a { font-size: 0.88rem; color: var(--ac, #3d5a80); text-decoration: none; } .calc-related-list li a:hover { text-decoration: underline; } .calc-browse-all { margin: 0.5rem 0 0; font-size: 0.9rem; font-weight: 600; } .calc-browse-all a { color: var(--ac, #3d5a80); text-decoration: none; } .calc-browse-all a:hover { text-decoration: underline; } @media (max-width: 720px) { .calc-related-list { grid-template-columns: 1fr; } }

Information Entropy & Redundancy Calculator

Calculate Shannon entropy, maximum entropy, redundancy, and efficiency for a probability distribution or text input.

### Input Method

Input Type

Probability Distribution (comma-separated) Text / Symbol Sequence Frequency Counts (comma-separated)

Probabilities (must sum to 1)

Text / Symbol Sequence

Ignore spaces

Case-insensitive

Frequency Counts (positive integers)

Logarithm Base

2 (bits) e (nats) 10 (hartleys / bans) Custom

Custom Base

Calculate

### Results

Metric Value Unit

#### Symbol Breakdown

Symbol Probability Self-Information Contribution to H

#### Probability Distribution

function infToggleInput() { var t = document.getElementById('inf-input-type').value; document.getElementById('inf-prob-section').style.display = t === 'prob' ? '' : 'none'; document.getElementById('inf-text-section').style.display = t === 'text' ? '' : 'none'; document.getElementById('inf-freq-section').style.display = t === 'freq' ? '' : 'none'; }

document.getElementById('inf-log-base').addEventListener('change', function() { document.getElementById('inf-custom-base-row').style.display = this.value === 'custom' ? '' : 'none'; });

function infShowError(msg) { var el = document.getElementById('inf-error'); el.textContent = msg; el.style.display = msg ? 'block' : 'none'; document.getElementById('inf-result').style.display = 'none'; }

function infGetBase() { var sel = document.getElementById('inf-log-base').value; if (sel === 'e') return Math.E; if (sel === 'custom') { var v = parseFloat(document.getElementById('inf-custom-base').value); if (isNaN(v) || v s.trim()).filter(s => s !== ''); if (parts.length v v === 0)) { // zero-prob symbols contribute 0 to entropy (0 log 0 = 0), but warn } var sum = vals.reduce((a,b) => a+b, 0); if (Math.abs(sum - 1) > 0.0001) return {error: 'Probabilities must sum to 1. Current sum: ' + sum.toFixed(6)}; probs = vals; labels = vals.map((v, i) => 'p' + (i+1));

} else if (type === 'freq') { var raw = document.getElementById('inf-freqs').value.trim(); if (!raw) return {error: 'Please enter frequency counts.'}; var parts = raw.split(',').map(s => s.trim()).filter(s => s !== ''); if (parts.length v v === 0)) return {error: 'At least one frequency must be positive.'}; var total = vals.reduce((a,b) => a+b, 0); probs = vals.map(v => v / total); labels = vals.map((v, i) => 'f' + (i+1));

} else { // text var text = document.getElementById('inf-text').value; if (!text) return {error: 'Please enter some text.'}; if (document.getElementById('inf-ignore-spaces').checked) text = text.replace(/ /g, ''); if (document.getElementById('inf-case-insensitive').checked) text = text.toLowerCase(); if (text.length === 0) return {error: 'No symbols remain after filtering.'}; var freq = {}; for (var c of text) freq[c] = (freq[c] || 0) + 1; var total = text.length; var entries = Object.entries(freq).sort((a,b) => b[1]-a[1]); probs = entries.map(e => e[1] / total); labels = entries.map(e => e[0] === ' ' ? '\n' : e[0] === ' ' ? '\t' : e[0]); }

return {probs, labels}; }

function infCalc() { infShowError(''); var base = infGetBase(); if (base === null) { infShowError('Custom base must be a number greater than 1.'); return; } var unit = infGetUnit();

var res = infGetProbabilities(); if (res.error) { infShowError(res.error); return; } var probs = res.probs, labels = res.labels;

// Filter zero-prob symbols for entropy calculation (0 * log(0) = 0 by convention) var n = probs.length; var nEffective = probs.filter(p => p > 0).length;

// Shannon Entropy H = -sum(p_i * log_b(p_i)) var H = 0; var selfInfo = probs.map(p => p > 0 ? -infLog(p, base) : 0); var contributions = probs.map((p, i) => p > 0 ? p * selfInfo[i] : 0); H = contributions.reduce((a,b) => a+b, 0);

// Maximum entropy H_max = log_b(N) where N = number of symbols with p > 0 var H_max = infLog(nEffective, base);

// Redundancy R = H_max - H var R = H_max - H;

// Relative redundancy (redundancy ratio) r = R / H_max = 1 - H/H_max var r = H_max > 0 ? R / H_max : 0;

// Efficiency (relative entropy) eta = H / H_max var eta = H_max > 0 ? H / H_max : 1;

// Minimum average code length (Shannon's source coding theorem lower bound) // L_min >= H (in bits if base 2) var L_min = H; // lower bound

// Hartley entropy (log of alphabet size) var H_hartley = infLog(nEffective, base);

// Build results table var tbody = document.getElementById('inf-result-body'); tbody.innerHTML = ''; var rows = [ ['Number of Symbols (N)', n, ''], ['Effective Symbols (p > 0)', nEffective, ''], ['Shannon Entropy (H)', H.toFixed(6), unit], ['Maximum Entropy (Hₘₐˣ)', H_max.toFixed(6), unit], ['Absolute Redundancy (R = Hₘₐˣ − H)', R.toFixed(6), unit], ['Relative Redundancy (r = R / Hₘₐˣ)', (r * 100).toFixed(4), '%'], ['Efficiency (η = H / Hₘₐˣ)', (eta * 100).toFixed(4), '%'], ['Min. Avg. Code Length Lower Bound', L_min.toFixed(6), unit + '/symbol'], ]; rows.forEach(function(row) { var tr = document.createElement('tr'); tr.innerHTML = '' + row[0] + '' + '' + row[1] + '' + '' + row[2] + ''; tbody.appendChild(tr); });

// Symbol breakdown table var sbody = document.getElementById('inf-symbol-body'); sbody.innerHTML = ''; probs.forEach(function(p, i) { var si = selfInfo[i]; var contrib = contributions[i]; var tr = document.createElement('tr'); tr.innerHTML = '' + labels[i] + '' + '' + p.toFixed(6) + '' + '' + si.toFixed(6) + '' + '' + contrib.toFixed(6) + ''; sbody.appendChild(tr); });

// Bar chart var chart = document.getElementById('inf-bar-chart'); chart.innerHTML = ''; var maxP = Math.max(...probs); var chartH = 110; probs.forEach(function(p, i) { var barH = maxP > 0 ? Math.round((p / maxP) * chartH) : 0; var col = document.createElement('div'); col.style.cssText = 'display:flex;flex-direction:column;align-items:center;min-width:28px;'; var bar = document.createElement('div'); bar.style.cssText = 'width:22px;background:#3498db;border-radius:3px 3px 0 0;height:' + barH + 'px;' + 'title="' + labels[i] + ': ' + p.toFixed(4) + '";cursor:default;'; bar.title = labels[i] + ': ' + p.toFixed(4); var lbl = document.createElement('div'); lbl.style.cssText = 'font-size:10px;margin-top:3px;max-width:28px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;text-align:center;'; lbl.textContent = labels[i]; lbl.title = labels[i]; col.appendChild(bar); col.appendChild(lbl); chart.appendChild(col); });

document.getElementById('inf-result').style.display = 'block'; }

#### Formulas

Shannon Entropy: H(X) = −∑i=1N pi · logb(pi)

where pi is the probability of symbol i, b is the logarithm base, and by convention 0 · log(0) = 0.

Maximum Entropy (Uniform Distribution): Hmax = logb(N)

where N is the number of symbols with non-zero probability. Achieved when all symbols are equally likely.

Absolute Redundancy: R = Hmax − H(X)

Relative Redundancy: r = R / Hmax = 1 − H(X) / Hmax

Efficiency: η = H(X) / Hmax = 1 − r

Self-Information of symbol i: I(xi) = −logb(pi)

Logarithm Base Units: Base 2 → bits  |  Base e → nats  |  Base 10 → hartleys (bans)

#### Assumptions & References

More Calculators

Read Next

Study Time Planner Authority Network America › Life Services Authority › National Calculator Authority .calc-container { max-width: 640px;...

References

Related Authorities

Life Services Authority › Professional Services Authority › Trade Services Authority › United States Authority ›