Bayesian Classifier (naive Boryes)

Bayesian Classifier

The Bayesian Classifier, commonly known as the Naive Bayes classifier, is a supervised machine learning algorithm based on Bayes’ Theorem. It is widely used for classification tasks such as spam detection, sentiment analysis, and document categorization.

At its core, the algorithm calculates the probability that a given data point belongs to a particular class based on prior knowledge and observed features. It applies Bayes’ Theorem to compute the posterior probability of a class given the input features.

The term “naive” comes from the simplifying assumption that all features are conditionally independent of each other given the class label. Although this assumption is rarely true in real-world data, the classifier still performs surprisingly well in many practical applications.

Example of Naive Bayes Classification

Suppose we are given a training dataset containing information about different species based on features such as swimming ability, flying ability, and crawling behavior. Using the naive bayes algorithm, we need to classify a new instance with features:

• Swim = Slow

• Fly = Rarely

• Crawl = No

The possible class labels are:

• Animal

• Bird

• Fish

We will use prior probability and conditional probability to determine the most likely class for the given test instance.

Given the training data set, use naive Boryes algorithms to classify a particular species if its features are (slow, rarely, no).

$F_1=\text{"swim"}{F}_2=\text{"Fly"}{F}_3=\text{"Crowl"}$

The class Labels are
$C_1=\text{"Animal"}{C}_2=\text{"Bird"}{C}_3=\text{"Fish"}$

Construct the frequency table which summaries the data [Not the part of algo]

Step 1: Compute the probability

Step 2: Constructing Table of Conditional Propability

The conditional probability are calculated as

$$\begin{gathered} P\left(F_1=\text{slow}/C_1\right)=\frac{\text{Number of records where }{F}_1=\text{slow and class label }{C}_1}{\text{Number of records with class label }{C}_1} \\ =\frac{2}{5} \end{gathered}$$

Step 3: we now calculate the following numbers

$$\begin{gathered} Swim=Slow \\ Fly=Rarely \\ Crawl=No \end{gathered}$$

$q_1=P(\text{swim/animal}).P(\text{fly/animal}).P(\text{crawl/animal}).p(\text{animal})$

$q_1=2/5\times 1/5\times 3/5\times 5/12$

$q_2=P(Swim/c_2).P(Fly/c_2).P(Crawl/c_2).p(c_2)$

$q_2=0/4\times 0/4\times 3/4\times 4/12=0$

$q_3=P(Swim/c_3).P(Fly/c_3).P(Crawl/c_3).p(c_3)$

$q_3=2/3\times 0/3\times 3/3\times 3/12=0$

Step 4: Find Maximum

$\text{max}\left(q_1,q_2,q_3\right)=0.0200$

Step 5: The maximum is $q_1$ as it corresponds to class $C_1=\text{Animal}$

so we assign the class Label "Animal" to the test instance $(\text{Slow, Rarely, No})$

Conclusion

The Bayesian Classifier, also known as the naive bayes algorithm, is a simple yet powerful supervised learning technique used for classification tasks in machine learning. By applying Bayes’ Theorem and assuming feature independence, it can efficiently classify data into different categories. Despite its “naive” assumption, the algorithm performs well in many real-world applications such as spam filtering, sentiment analysis, and document classification. In this example, the test instance was successfully classified as “Animal” based on the calculated probabilities.

To compare with other classification approaches, see the ID3 Algorithm and Decision Tree notes in the AI & ML collection.

This note is part of the AI & ML collection on NoteHub.