Question

7. Which probability distribution P={P1,P2,P3} has the biggest entropy H((p1,P2,P3})?

To clarify the question, we are trying to find the general case for which we get the most entropy. What probability distribution, given by P={p1, p2, p3}, would give us the most entropy? For example, P={1/3, 1/3, 1/3} could be an answer (but I don't know the answer)

0 0
Add a comment Improve this question Transcribed image text
Answer #1

The entropy is maximum when P={1/3, 1/3, 1/3}

The entropy S associated with a random variable in the discrete case is

S=-\sum _{i}P_{i}\log {P_{i}}

The logarithm is with base 2

The estimated entropy for P={p1, p2, p3} is -1/3log1/3 -1/3log1/3 -1/3log1/3 = 1.585 bits.

In all other cases the value of entropy will be less than 1.585 . The entropy will be maximum when p1=p2=p3

Add a comment
Know the answer?
Add Answer to:
To clarify the question, we are trying to find the general case for which we get the most entropy...
Your Answer:

Post as a guest

Your Name:

What's your source?

Earn Coins

Coins can be redeemed for fabulous gifts.

Not the answer you're looking for? Ask your own homework help question. Our experts will answer your question WITHIN MINUTES for Free.
Similar Homework Help Questions
  • Need the process that how we get P2 P1 P4 P3 and waiting time please 1....

    Need the process that how we get P2 P1 P4 P3 and waiting time please 1. Draw a Gantt chart below similar to the ones from lecture that illustrates the execution of the processes using the shortest-job-first CPU scheduling algorithm. Process Arrival Burst | Time Time P. 7 ms 2 ms | P2 Oms 8 ms 11 ms 5 ms P4 4 ms 9 ms P2 P2 P4 P3 oms 8 10 19 24 | Using the chart you drew,...

  • In a book the characters are trying to determine whether the Poisson probabilistic model can be...

    In a book the characters are trying to determine whether the Poisson probabilistic model can be used to describe the locations that rockets landed in a city. They divided the city into 0.25-kmregions. They then counted the number of rockets that landed in each region, with the results shown in the table below. Complete parts (a) through (e). Number of rocket hits 0 1 2 3 4 5 6 7 Observed number of regions 221 215 100 32 8 0...

  • Let p=(p1,p2,…,pm) be a probability distribution over m possible outcomes. The entropy of p is a measure of how much ran...

    Let p=(p1,p2,…,pm) be a probability distribution over m possible outcomes. The entropy of p is a measure of how much randomness there is in the outcome. It is defined as where ln denotes natural logarithm. We wish to ascertain whether F(p) is a convex function of p. As usual, we begin by computing the Hessian. A) Consider the specific point p=(1/m,1/m,…,1/m). What is the (1,1) entry of the Hessian at this point? Your answer should be a function of m....

  • A polynomial p(x) is an expression in variable x which is in the form axn + bxn-1 + …. + jx + k, where a, b, …, j, k are...

    A polynomial p(x) is an expression in variable x which is in the form axn + bxn-1 + …. + jx + k, where a, b, …, j, k are real numbers, and n is a non-negative integer. n is called the degree of polynomial. Every term in a polynomial consists of a coefficient and an exponent. For example, for the first term axn, a is the coefficient and n is the exponent. This assignment is about representing and computing...

  • Question 8 Answer the next five questions based on the case given below: Tower Engineering Corporation...

    Question 8 Answer the next five questions based on the case given below: Tower Engineering Corporation is considering undertaking several proposed projects for the next fiscal year. The projects, the number of engineers and the number of support personnel required for each project, and the cost for each project are summarized in the following table! Projects 1 2 3 4 S Engineers Required 50 35 40 45 55 30 Cost ($1,000,000) 2.0 3.4 2.6 2.8 3.0 2.5 Formulate an integer...

  • Suppose we have a late-night bus and towards the end of the route, there are 3...

    Suppose we have a late-night bus and towards the end of the route, there are 3 passengers {P1, P2, P3} and 5 stops {S1, S2, S3, S4, S5} remain. Suppose further that each passenger is inebriated and is thus is equally likely to get off at any one of the stops. (i) We wish to list the set of outcomes in the sample space each of whose outcomes is an ordered triple of all three Sij for I-1,2,3, where Sij...

  • We are trying to solve for the temperature distribution of a sphere in boiling water. The...

    We are trying to solve for the temperature distribution of a sphere in boiling water. The sphere is of radius a, and it's at temperature Tc. The boiling water is at Th =Tw, and: We understand the initial condition is: at t=0, r: T(r,0) = Tc And the boundary condition is: at r=a, t: -k(dT/dr) = h[T(a,t) - Tc]. We also know that the following steps are required after this: 1. General heat equation. 2. Non-dimensionalise our heat eqn, BC,...

  • Answers to the following questions should be calculated in Matlab and given to 4 decimal places....

    Answers to the following questions should be calculated in Matlab and given to 4 decimal places. Be careful not to introduce errors by rounding in any intermediate calculations as this may lead to an incorrect final answer. Suppose the discrete random variable X has the following probability distribution 3 P1 1 22 p(x = 0.1 0.1 4 P2 fc3 = 7 P3 = 0.05 24 = 0.2 24 - 10 05 12 P5 0.55 Calculate the following. (a) E[X] A...

  • additional information · Question: When I given by Shannon is equivalent to S given by Gibbs?...

    additional information · Question: When I given by Shannon is equivalent to S given by Gibbs? 2. General Properties of the Statistical Entropy 2.1 The Boltzmann Definition os given by Boltzmann relies on the postulate of equal probability of the W accessible microstates of an isolated system. S = kgInW(E) 2.2 The Gibbs Definition o A more general definition of the statistical entropy was proposed by Gibbs : S= -kopi Impi o The Boltzmann of the entropy is a special...

  • 3. Suppose we have a late night bus and towards the end of the route, there...

    3. Suppose we have a late night bus and towards the end of the route, there are 3 passengers {P, Pz2 , P3} and 5 stops SI,S2,S3,S4,Ss, remain. Suppose further that each passenger is inebriated, and is thus is equally likely to get off at any one of the stops (i) We wish to list the set of outcomes in the sample space each of whose outcomes is an ordered triple of all three Sij for l=1,2,3, where Sij means...

ADVERTISEMENT
Free Homework Help App
Download From Google Play
Scan Your Homework
to Get Instant Free Answers
Need Online Homework Help?
Ask a Question
Get Answers For Free
Most questions answered within 3 hours.
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT