%H=entropy(x) is the entropy of column vector x. 
% Assumes x (rounded to nearest integers) 
% came from a zero-order Markov source, 
% and estimates the source entropy. 
% 
% If x is a matrix, 
% column_entropy = entropy(x); 
% finds the entropy of each column independently, 
% row_entropy = entropy(x'); 
% finds the entropy of each row independently, and 
% H = entropy(x(:)) 
% returns the single entropy value over the entire matrix. 
% For many photographs, 
% mean(column_entropy) < mean(row_entropy) < H. 
% Example: 
%   H = entropy( [ -2 -3 -4 -2; 30 16.6 17.4 300]') 
% returns H = [1.5000 1.5000 ], 
% indicating each set of 4 numbers 
% appears to come from a source with 
% 1.5 bits of entropy per output value. 
 
%Changelog:
% 1999-07-07:DAV: put online at
%   http://OIL.okstate.edu/~caryd/program/entropy.m
% 1999-07-06:DAV: modified to do each column independently. 
% 1999-06-26:DAV: David Cary <d.cary@ieee.org> added documentation. 
% ???:JCK: originally by John C. Kieffer 
%   http://www.ee.umn.edu/users/kieffer/programs.html 
% documented in 
%   ftp://oz.ee.umn.edu/users/kieffer/seminar/notes3.pdf 

function H = entropy(x) 
minimum = min(x(:)); 
if( minimum < 0 ) 
   histogram = histo(x-minimum); 
else
   histogram = histo(x);
end;
[rows, columns] = size(x);
for i=1:columns
   % find all non-zero probabilities
   P=histogram( find(histogram(:,i)), i ) ./ rows;
   H(i)=sum(-P.*log2(P));
end;