|Definition:|| amount by which the decision content H0 exceeds the entropy H |
R = H0 − H
EXAMPLE Let be a set of three events and let , and be the probabilities of their occurrences. The redundancy of this set is: .
Note 1 to entry: Usually, messages can be represented with fewer characters by using suitable codes; the redundancy can be considered as a measure of the decrease of the average length of the messages accomplished by appropriate coding.