Naive Bayes classification algorithm

Dataset –

ID Name Gender Height Class
1 A F 1.6 short
2 B M 2 tall
3 C F 1.9 medium
4 D F 1.85 medium
5 E M 2.8 tall
6 F M 1.7 short
7 G M 1.8 medium
8 H F 1.6 short
9 I F 1.65 short

Find probability of class attribute –

\(P(short) = \frac{4}{9}\)
\(P(medium) = \frac{3}{9}\)
\(P(tall) = \frac{2}{9}\)

Working Table –

Attribute Value
(Short)
Value
(Medium)
Value
(Tall)
Probability
(Short)
Probability
(Medium)
Probability
(Tall)
Gender Male
Female
Height [0 -1.6]
[1.61 – 1.7]
[1.71 – 1.8]
[1.81 – 1.9]
[1.91 – 2.0]
[2.0 – ∞]

Filling the table

Attribute Value
(Short)
Value
(Medium)
Value
(Tall)
Probability
(Short)
Probability
(Medium)
Probability
(Tall)
Gender Male 1 1 2 0.25 0.333 1
Female 3 2 0 0.75 0.667 0

Solution –

Attribute Value
(Short)
Value
(Medium)
Value
(Tall)
Probability
(Short)
Probability
(Medium)
Probability
(Tall)
Gender Male 1 1 2 1/4 1/3 2/2
Female 3 2 0 3/4 2/3 0
Height [0 – 1.6] 2 0 0 2/4 0 0
[1.61 – 1.7] 2 0 0 2/4 0 0
[1.71 – 1.8] 0 1 0 0 1/3 0
[1.81 – 1.9] 0 2 0 0 2/3 0
[1.91 – 2] 0 0 1 0 0 1/2
[2.1 – ] 0 0 1 0 0 1/2

Using Bayesian classification and given data, classify the following set –

\(t = \{Hola, M, 2.2\}\)

Step 1: Find probability for each class

\(P(t|short) = P(m|short) \times P(2.1 -∞|short) = \frac{1}{4} \times 0 = 0\)
\(P(t|medium) = P(m|short) \times P(2.1 -∞|medium) = \frac{1}{4} \times 0 = 0\)
\(P(t|tall) = P(m|short) \times P(2.1 -∞|tall) = \frac{2}{2} \times \frac{1}{2} = 0.5 =
\frac{1}{2}\)

Step 2: Finding likelyhood of occurence –

\(L(short) = P(t|short) \times p(short) = 0 \times \frac{4}{9} = 0\)
\(L(medium) = P(t|medium) \times p(medium) = 0 \times \frac{3}{9} = 0\)
\(L(tall) = P(t|tall) \times p(tall) = \frac{1}{2} \times \frac{2}{9} = \frac{2}{18} =
0.11\)

\(Estimate = p(t) = 0 + 0 + 0.11 = 0.11\)

Bayes Theorem –

\(P(x/y) = (P(y/x) \times P(x))/P(y)\)

Step 3: Find final probability

\(P(short|t) = \frac{(P(t|short) \times P(short)}{P(t)} = \frac{0 \times \frac{4}{9}}{0.11} = 0\)
\(P(medium|t) = \frac{P(t|medium) \times P(medium)}{P(t)} = \frac{0 \times \frac{3}{9}}{0.11} = 0\)
\(P(tall|t) = \frac{P(t|tall) \times P(tall)}{P(t)} = \frac{\frac{1}{2} \times \frac{2}{9}}{0.11} = 1\)