Monthly Archives: June 2006

Logic, Belief and Bayesian Inference


       It is quite rare that a thirty year old man changes the way he thinks. Only an enormous event can change thoughts that one has grown up with. There has been an event in my life that has produced this change. I was forced to relook at the way I think because of this event. This event is a book: “Probability Theory- The Logic of Science” by E.T Jaynes. The gift of new thought is the greatest gift that a mind can give another.
This book gave me that gift. And now I begin my poor attempt to show you the contents of this gift.

       I had written a VERITAS in which I had referred to this book before. But in today’s VERITAS I will describe the aim of this book and how it changed my thought process at a fundamental level.

       This book is about Bayesian probability. As I had written in the earlier article Bayesian probability theory is about knowledge and how probabilities come up when we have incomplete knowledge. The aim of this book is to show that probability theory is a theory which extends logic to areas where we have incomplete knowledge.

       When do we have incomplete knowledge? Always(nearly)! So the application of probability theory becomes important for nearly every
analysis because there are so few situations in which we have complete knowledge! So this book tells us how we can apply Bayes theorem( and probability theory) to situations around us.

       After reading this book it becomes obvious that it is nearly impossible to attach a probability of 0 or 1 to most things around us. So we cannot believe or disbelieve in something completely. We are always somewhere in the middle. Complete belief or disbelief can only come from complete knowledge.


       This book encourages us to assign degrees of belief to our thoughts. Jaynes calls this prior evidence. So a prior evidence is our belief in something before some experiment is done. It is measured in decibels. After a set of experiments our beliefs may change based on the experimental results. This is called posterior evidence. The link between prior evidence and posterior evidence is formed by Bayes theorem.

       Let’s take an example: I say that my belief in astrology is -50 decibels. We call this the prior evidence( lets just call it prior belief). If I go to an astrologer and he tells me 5 things about me by just looking at my horoscope he will add 50 decibels of evidence and take my porterior belief to 0 db. 0 db means that I do not know whether astology is true or not. So the 5 things that the astrologer told me has converted me from a disbeliever( -50 dB) to a person who has no opinions. Now if I keep meeting astrologers who keep predicting my future correctly then they will keep adding to the evidence and it might be that after a year I become a strong believer( say at 100 dB).

       So the degree of belief/dibelief is the amount of evidence needed to take you to 0 dB where you do not know if that thing is true or not.

       In probability language 0 Db means a probability of 1/2. A probability of 1 means infinite dB. A probability of 0 means  -infinite dB of belief.

       Now lets say I have complete belief in something. So I assign a probability of 1 to it. That means I assign a prior belief of infinite to it. What does that mean? It means that no matter how much evidence I get against that belief I will not change my belief. And that is being stubborn!

       Similarly if I completely disbelieve in something. That means that I assign -infinite prior belief to it. So no matter  how many experiments show the validity of the statement I will continue to completely disbelieve in it. And that is also being stubborn!

       So completely believing or disbelieving in something is to not let your beliefs to be shaped by experiments or by evidence.
And this is not the way a learner’s mind works.

       So at what level of belief does a learner’s mind work the most? AT a level when it can be most swayed by experiment: 0 dB of belief. So if one has very little knowledge about something one should start at 0 dB of belief.

       I had always been an athiest- a complete disbeliever in God. This book showed me that since I have no knowledge of the Universe or its laws or its beginning so I have no reason to assign a 0 probability( of -infinite prior belief) to a Creator( or God). I should say “I do not know” and assign
a 0dB prior belief to the possibility of God. This way I can hope to learn from evidence and experiments the most. I have become an agnostic!

       And I think being an agnostic is the most reasonable position to take for anyone who does not have a great deal of knowledge about the Universe.

       I learnt from this book that one should not be afraid of saying “I do not know”. It is better to acknowledge ignorance and put your beliefs to a 0 dB level and then to be swayed by evidence than to assign complete belief or disbelief to things and then being insensitive to new evidence.

       This book goes on to apply Bayes theorem to a huge number of everyday situations and also scientific situations. It is a beautifully written book. A book written from the heart. A book that should be there in every college curriculum just because it makes you think in a rational way about things you do not have full knowledge of. And it should be in every thinking person’s personal library.

       This book approaches even the philosophical subjects using mathematics and probability. For example in one section Jaynes asks why people do not converge on an agreement even when they see the same evidence. Let me explain. Let’s say A and B believe in telepathy to a different degree. A has a lot of belief and B has very little. Jaynes shows us that even if A and B are shown the same experiments involving telepathy they may never come to a common point of belief. They may continue to believe/disbelieve in telepathy to different degrees. And this is
all done using Bayes theorem. So Jaynes tells us why different people analyse data differently. And Jaynes does that using mathematics
and probability theory.

       A beautiful book! My strongest recommendation!



 Go, wondrous creature! mount where Science guides:
 Go, measure earth, weigh air, and state the tides:
 Instruct the planets in what orbs to run,
 Correct old time and regulate the Sun;

Creative Commons License
Veritas by Kanwarpreet Grewal is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.