Page 1 of 1
2 ML Concepts: Mutual Information Information Theory Definitions: • HY)=-Eyevalura(y) P(Y = y) log, P(Y = y) • HY | X =
Posted: Sat May 14, 2022 7:06 pm
by answerhappygod
- 2 Ml Concepts Mutual Information Information Theory Definitions Hy Eyevalura Y P Y Y Log P Y Y Hy X 1 (24.4 KiB) Viewed 68 times
2 ML Concepts: Mutual Information Information Theory Definitions: • HY)=-Eyevalura(y) P(Y = y) log, P(Y = y) • HY | X = r) = - Eyevalues() P(Y = [X = r ) log, P(Y = y|X = 1) • H( YX) = Eteratura[ X P(X = )H(Y | X = x) . (X;Y) = HY) - H( YX) Exercises 1. Calculate the entropy of tossing a fair coin, 2. Calculate the entropy of tossing a coin that lands only on tails. Note: 0 log,(0) = 0. 3. Calculate the entropy of a fair dice roll. 4. When is the mutual information I(X:Y) = 0?