Logistic Regression

Created

Mar 5, 2025

Last Modified

2 weeks ago

Logistics regression

Classification is a kind of problem in machine learning in which a machine learning model assigns the respective class to unseen data based on feature value. If the deciding classes are only two then the problem is known as a binary classification problem. If the classes are more than 2, then it is a multiclass logistic regression problem.

sigmoidal function

σ (z) = (\frac{1}{1 + e ^{- z}})

where $e \approx 2.71828$

undetermined values: $0^{0},$ $\frac{0}{0},$ $\infty^{0},$ $\infty,$ $- \infty,$ $0 \times \infty,$ $\infty$

This is a specific function that converts the values of Z from $- \infty$ to $\infty$ in logistic regression. This function returns a value between 0 and 1 (probabilistic).

Logistic regression sigmoid function graph showing probability curve ranging from 0 to 1 with decision boundary at 0.5

$i f z = - \infty$

σ (z) = (\frac{1}{1 + e ^{- (- \infty)}})

σ (z) = (\frac{1}{1 + e ^{\infty}})

σ (z) = (\frac{1}{\infty})

σ (z) = 0

z = a + b x

$i f z = \infty$

σ (z) = (\frac{1}{1 + e ^{- \infty}})

σ (z) = (\frac{1}{1 + \frac{1}{e ^{\infty}}})

σ (z) = (\frac{1}{1 + \frac{1}{\infty}})

σ (z) = (\frac{1}{1 + 0})

σ (z) = 1

$σ (z) = 0 w h er e z = - \infty$

$σ (z) = 1 w h er e z = \infty$

$z = a + b x$

In linear regression $z = a + b x$ , $z$ is the output variable which gives a numerical value by using the sigmoidal function $σ (z) = (\frac{1}{1 + e ^{- z}})$ . We can turn the value between 0 and 1.

σ (z) = (\frac{1}{1 + e ^{- z}})

σ (z) = (\frac{1}{1 + e ^{- (a + b x)}})

where $0 \leq σ (z) \leq 1$

Odd can be defined as:

\frac{p x}{1 - p x}

Where $p x$ is the probability of the occurrence of $x$ .

This means:

\frac{p x}{1 - p x} = \frac{p r o babi l i t y o f occ u r r e n ce o f x}{p r o babi l i t y o f n o t occ u r r e n ce o f x}

σ (z) = (\frac{1}{1 + e ^{- z}})

e^{- z} = (\frac{1}{σ ( z )}) - 1

e^{- z} = \frac{1 - σ ( z )}{σ ( z )}

e^{z} = \frac{σ ( z )}{1 - σ ( z )}

where $0 \leq σ (z) \leq 1$

Taking $l o g_{e}$ of both sides

z = l o g (\frac{σ ( z )}{1 - σ ( z )})

In logistic regression, we obtain a value between 0 and 1, which is a probability value, and then we decide on a threshold value.

if $σ (z) \geq V_{t}$

V (value)
T (Threshold)

Then a particular class is assigned to z, otherwise the other class.

$V_{t}$ is 0. If the result is greater than class A, otherwise class B

Binomial logistic regression.

If the number of assigned classes is two, then the logistic regression is binomial logistic regression; if the number of assigned classes is more than two, then it is multinomial logistic regression.

Logistic Regression

Logistic Degrees models are used in Binary classification and multi-classification. They are widely used in email categorization, disease detection, fraud detection, and text emotion analysis

Measurement of the performance of the machine learning model metric

confusion matrix

accuracy
precision
recall
F1 score (harmonic mean)

Cow +ve
Deer -ve

Actual Class	Cow	Cow	Deer	Deer	Cow	Deer	Cow	Deer	Cow	Cow
Predicted Class	Deer	Cow	Deer	Deer	Deer	Cow	Cow	Cow	Cow	Deer
	FN	TP	TN	TN	FN	FP	TP	FP	TP	FN

A cc u r a cy = \frac{T P + T N}{T P + T N + F P + F N}

A cc u r a cy = \frac{3 + 2}{10} = \frac{5}{10} = 0.5

P r ec i s i o n = \frac{T P}{T P + F P}

R ec a l l = \frac{T P}{T P + F N}

F_{1} S cor e = \frac{2}{\frac{1}{P r ec i s i o n} + \frac{1}{R ec a l l}}

F_{1} S cor e = \frac{2 ( P r ec i s i o n \times R ec a l l )}{( P r ec i s i o n + R ec a l l )}

Logistic Regression

Logistics regression

sigmoidal function

Taking loge​ of both sides

Binomial logistic regression.

Logistic Regression

confusion matrix

Taking $l o g_{e}$ of both sides