Part 3: How Bayes' Theorem Is Used in AI
Part 1: Conditional Probability
▶ Part 1: Conditional Probability
▶ Part 3: Bayes' Theorem in AI
Bayes' theorem is not an AI concept by itself.
It comes from probability theory and statistics. AI and machine learning can use Bayesian methods when they need to reason about uncertain information.
The basic idea remains:
Start with what you believe, observe new evidence, and update your belief.
1. Bayes, Statistics, and AI
It helps to separate the concepts:
- Probability theory provides the mathematical foundation for reasoning about uncertainty.
- Statistics uses probability and data to make inferences, estimates, and predictions.
- Bayesian statistics uses Bayes' theorem to update beliefs or estimates when new evidence is observed.
- AI and machine learning can use these statistical and probabilistic techniques to make predictions or decisions.
So Bayes' theorem is better thought of as a tool that can be used in AI, rather than something that belongs exclusively to AI.
2. Spam Detection
Imagine an email arrives containing words such as:
"Congratulations! You have won a prize."
A Bayesian spam classifier can consider:
- How likely an email is to be spam (prior)
- How likely these words are to appear if the email is spam (likelihood)
- How common the observed words are overall (evidence)
It can then estimate:
If the probability is high enough, the email can be classified as spam.
A well-known example is Naive Bayes, a simple machine-learning classification algorithm based on Bayes' theorem.
3. Medical Diagnosis
Suppose we want to estimate whether a disease is present.
Before seeing any test results, we have some starting probability:
This is the prior.
Now a test result arrives. The result provides evidence that can change our estimate:
This is the posterior.
The same idea appeared in the imperfect-test example in Part 2: the probability was updated after observing a positive test result.
4. Decision-Making Under Uncertainty
AI systems often have to make decisions without knowing the complete state of the world.
For example, an AI might have several possible explanations for an observation:
Possible explanation A
Possible explanation B
Possible explanation C
↓
New evidence
↓
Update probabilities
↓
Choose the most likely explanation
Bayesian reasoning provides one mathematical way to perform this kind of update.
5. The Important Distinction
Bayes' theorem itself does not "make something AI."
Instead:
For example, Naive Bayes is a machine-learning algorithm that uses Bayes' theorem for classification.
So when we say that "Bayes is used in AI," what we really mean is:
AI systems can use Bayesian methods to reason about uncertainty and update probabilities when new evidence becomes available.
The important pattern is:
That pattern is fundamentally a probability and statistics idea, which can then become useful in AI.
6. Other Applications of Naive Bayes
Spam detection is only one example.
The same basic idea can be used whenever we want to classify something based on observed features.
Some other applications include:
- Sentiment analysis: classify text as Positive or Negative.
- Document and topic classification: classify documents as Sports, Politics, Business, Technology, etc.
- Language identification: identify the language of a piece of text.
- Intent classification: determine what a user is trying to do, such as cancel a booking, request a refund, or check a status.
- Toxic or abusive text detection: classify comments as Toxic or Not Toxic.
- News and social-media classification: classify posts or articles into predefined categories.
- Simple medical classification: use symptoms or test features to classify possible conditions.
- Recommendation and filtering: classify content according to whether it is likely to be relevant or interesting to a user.
The common pattern is the same: