In information theory Shannon entropy is defined by H(x) = -Sum(P(x)*log(P(x)) where P is probability mass...
In information theory Shannon entropy is defined by H(x) = -Sum(P(x)*log(P(x)) where P is probability mass function of random variable x, and log to base 2. Given loaded die with P(6)=0.5 and P(1)=P(2)=P(3)=P(4)=P(5), compute entropy of observed rolls 654266. Note: To answer the question you do not need to know any more information about Shannon entropy.
Homework Answers
Add Answer to:
In information theory Shannon entropy is defined by H(x) =
-Sum(P(x)*log(P(x)) where P is probability mass...
Problem 2 - Information Theory Consider the following channel X, DI . a) Calculate entropy of...
Problem 2 - Information Theory Consider the following channel X, DI . a) Calculate entropy of X, i.e., H(X) when [p(31),p(32) = (0.6,0.4). Give p(x1) and p(x2) that maximize H(X) and the corresponding maximum value of H(X). . b) Give the matrix of transition probabilities p(YX)). . c) Calculate output probabilities (p(Y)). . d) Give output probabilities so that H(Y) is maximum. For this case, then give equations with which you can calculate p(x) and p(x2). The SNR is 20...
3. Let X be the random variable characterized by a probability distribu tion p (P, ...p)...
3. Let X be the random variable characterized by a probability distribu tion p (P, ...p) and it can assume one of the values r1,....In with probabilities pi, , pni 0 < pi 1, Σǐpí = 1 The Shannon entropy of the random variable X is defined as Show that 0 s H(X) S log2 so that the maximal value in this inequality is saturated by unifornm probability distribution. What is the level of information for this state of the...
This is for an Information Theory class. H(X) is entropy rate. Problem 8: Suppose that X is a random variable with a pr...
This is for an Information Theory class. H(X) is entropy
rate.
Problem 8: Suppose that X is a random variable with a probability that X = k) given by: probability distribution (i.e., Px (k) = Prob(X = k) = (1 - ) )X* for k0 where 0 < 1 and k is a non-negative integer (and hence X can take any negative integer value). To answer this question, note that the AEP theorem we proved for a finite-alphabet random variable...
I. The random variables X,, where P(success) = P(X = 1) = p = 1-P(X = 0) for1,2,..., represent a ...
I. The random variables X,, where P(success) = P(X = 1) = p = 1-P(X = 0) for1,2,..., represent a series of independent Bernoulli trials. Let the random variable Y be the trial number on which the first success is achieved (a) Explain why the probability mass function of Y is f(y) = pqy-1, y = 12. where q 1- p. State the distribution of Y. 2 part of your answer you should verify this is a marimum likelihood estima-...
2. Let X be a Bernoulli random variable with probability of X -1 being a. a)...
2. Let X be a Bernoulli random variable with probability of X -1 being a. a) Write down the probability mass function p(X) of X in terms of a. Mark the range of a (b) Find the mean value mx(a) EX] of X, as a function of a (c) Find the variance σ剤a) IX-mx)2) of X, as a function of a. (d) Consider another random variable Y as a function of X: Y = g(X) =-log p(X) where the binary...
Consider a random variable X with the following probability mass function P(X=0)=0.25, P(X=5)=0.5, P(X=12)=0.25. What is...
Consider a random variable X with the following probability mass function P(X=0)=0.25, P(X=5)=0.5, P(X=12)=0.25. What is the expected value (or mean) of X?
The moment generating function ф(t) of random variable X is defined for all values of t by et*p(x), if X is discrete e...
The moment generating function ф(t) of random variable X is defined for all values of t by et*p(x), if X is discrete e f (x)dx, if X is continus (a) Find the moment generating function of a Binomial random variable X with parameters n (the total number of trials) and p (the probability of success). (b) If X and Y are independent Binomial random variables with parameters (n1 p) and (n2, p), respectively, then what is the distribution of X...
The probability mass function for the discrete random variable X is p(X=0)=0.13; p(X=1)=0.31; p(X=2)=m. What is...
The probability mass function for the discrete random variable X is p(X=0)=0.13; p(X=1)=0.31; p(X=2)=m. What is the expected value of X? Hint: First compute m, then find the expectation of X. Round your answer to the nearest hundredth.
2. Let X be a Bemoulli random variable. The probability mass function is f(p) p(1 p when x 0 or x 1, where p is the parameter to be estimated. Please declare the MLE, and workout the steps to so...
2. Let X be a Bemoulli random variable. The probability mass function is f(p) p(1 p when x 0 or x 1, where p is the parameter to be estimated. Please declare the MLE, and workout the steps to solve it
2. Let X be a Bemoulli random variable. The probability mass function is f(p) p(1 p when x 0 or x 1, where p is the parameter to be estimated. Please declare the MLE, and workout the steps to...
6. (Entropy) The Bernoulli random variable X takes on the values 0, 1 with equal probability,...
6. (Entropy) The Bernoulli random variable X takes on the values 0, 1 with equal probability, i.e. PX pX Compute El(x) if where logs are to base 2
Problem 2 - Information Theory Consider the following channel X, DI . a) Calculate entropy of X, i.e., H(X) when [p(31),p(32) = (0.6,0.4). Give p(x1) and p(x2) that maximize H(X) and the corresponding maximum value of H(X). . b) Give the matrix of transition probabilities p(YX)). . c) Calculate output probabilities (p(Y)). . d) Give output probabilities so that H(Y) is maximum. For this case, then give equations with which you can calculate p(x) and p(x2). The SNR is 20...
3. Let X be the random variable characterized by a probability distribu tion p (P, ...p) and it can assume one of the values r1,....In with probabilities pi, , pni 0 < pi 1, Σǐpí = 1 The Shannon entropy of the random variable X is defined as Show that 0 s H(X) S log2 so that the maximal value in this inequality is saturated by unifornm probability distribution. What is the level of information for this state of the...
This is for an Information Theory class. H(X) is entropy
rate.
Problem 8: Suppose that X is a random variable with a probability that X = k) given by: probability distribution (i.e., Px (k) = Prob(X = k) = (1 - ) )X* for k0 where 0 < 1 and k is a non-negative integer (and hence X can take any negative integer value). To answer this question, note that the AEP theorem we proved for a finite-alphabet random variable...
I. The random variables X,, where P(success) = P(X = 1) = p = 1-P(X = 0) for1,2,..., represent a series of independent Bernoulli trials. Let the random variable Y be the trial number on which the first success is achieved (a) Explain why the probability mass function of Y is f(y) = pqy-1, y = 12. where q 1- p. State the distribution of Y. 2 part of your answer you should verify this is a marimum likelihood estima-...
2. Let X be a Bernoulli random variable with probability of X -1 being a. a) Write down the probability mass function p(X) of X in terms of a. Mark the range of a (b) Find the mean value mx(a) EX] of X, as a function of a (c) Find the variance σ剤a) IX-mx)2) of X, as a function of a. (d) Consider another random variable Y as a function of X: Y = g(X) =-log p(X) where the binary...
The moment generating function ф(t) of random variable X is defined for all values of t by et*p(x), if X is discrete e f (x)dx, if X is continus (a) Find the moment generating function of a Binomial random variable X with parameters n (the total number of trials) and p (the probability of success). (b) If X and Y are independent Binomial random variables with parameters (n1 p) and (n2, p), respectively, then what is the distribution of X...
2. Let X be a Bemoulli random variable. The probability mass function is f(p) p(1 p when x 0 or x 1, where p is the parameter to be estimated. Please declare the MLE, and workout the steps to solve it
2. Let X be a Bemoulli random variable. The probability mass function is f(p) p(1 p when x 0 or x 1, where p is the parameter to be estimated. Please declare the MLE, and workout the steps to...
6. (Entropy) The Bernoulli random variable X takes on the values 0, 1 with equal probability, i.e. PX pX Compute El(x) if where logs are to base 2
Most questions answered within 3 hours.
-
Joanne Quick made an investment of $18,787.14. From this
investment, she will receive $2,000 annually for...
asked 4 minutes ago -
Enter your answer in the provided box. The solubility of CO2 in
water at 25°C and...
asked 11 minutes ago -
Amazon as with other ecommerce retailers struggle with cross
border solutions and continue to be country...
asked 10 minutes ago -
A second baseman tosses the ball to the first baseman, who
catches it at the same...
asked 27 minutes ago -
An object rests flat on a tablecloth, which is on top of a
table. Suppose you...
asked 26 minutes ago -
Which of the following will most likely occur as the result of
an unanticipated increase in...
asked 23 minutes ago -
Adria Lopez, owner of Success Systems, decides to prepare a
statement of cash flows for her...
asked 46 minutes ago -
The reach and involvement of the federal government, in terms of
its role in disaster relief,...
asked 48 minutes ago -
Problem: The equilibrium constant, Keq, is 5 x 10^-11 m HCO3
<-----------> H^+ + CO3^-2 Calculate...
asked 55 minutes ago -
Explain how blackbody radiation led to the idea of "quanta."
Will rate answer!
asked 1 hour ago -
Two noble gases (Ne and He) are injected into the chambers of an
insulated container. Each...
asked 56 minutes ago -
Suppose you purchase a zero coupon bond with a face value of
$1,000, maturing in 18...
asked 1 hour ago