Logistic Regression

Logistic Regression

Classification is a kind of problem in machine learning in which a machine learning model assigns the respective class to unseen data based on feature value. If the deciding classes are only two then the problem is known as a binary classification problem. If the classes are more than 2, then it is a multiclass logistic regression problem.

sigmoidal function

where $e\approx 2.71828$

undetermined values: $0^0,$ $\frac{0}{0},$ ${\infty }^0,$ $\infty ,$ $-\infty ,$ $0\times \infty ,$ $\infty$

This is a specific function that converts the values of Z from $-\infty$ to $\infty$ in logistic regression. This function returns a value between 0 and 1 (probabilistic).

$ifz=-\infty$

$ifz=\infty$

$\sigma (z)=0wherez=-\infty$

$\sigma (z)=1wherez=\infty$

Logistic Regression is often confused with Linear Regression because both start with a linear equation of the form $z=\alpha +\beta x$. However, the key difference lies in the output.

To understand how linear models behave with continuous outputs and why logistic regression modifies this behavior for classification, you can first explore:

• Linear Regression

• Polynomial Regression

These help build intuition on how regression models predict continuous values, whereas logistic regression transforms those outputs into probabilities for classification tasks.

Linear Output Transformation

In logistic regression, the linear equation is defined as

$z=a+bx$

This raw value $z$ is then passed through the sigmoid function, which converts it into a probability between 0 and 1.

where $0\le \sigma (z)\le 1$

Intuition Behind Sigmoid Function

The sigmoid function compresses any real-valued number into a probability range:

• Large negative values → close to 0

• Large positive values → close to 1

This makes it ideal for classification problems where outputs must represent probabilities instead of raw numbers.

Odds and Probability Interpretation

In probability theory, odds represent the ratio between the probability of an event occurring and not occurring.

Where $p$ is the probability of occurrence of an event.

This can also be expressed as the ratio of success to failure probability:

Log-Odds Transformation in Logistic Regression

This section explains how the sigmoid function can be rearranged to express the relationship between probability and the linear output $z$.

where $0\le \sigma (z)\le 1$

Taking $lo{g}_e$ of both sides

In logistic regression, we obtain a value between 0 and 1, which is a probability value, and then we decide on a threshold value.

if $\sigma (z)\ge V_t$

V (value)

T (Threshold)

Then a particular class is assigned to z, otherwise the other class.

$V_t$ is 0. If the result is greater than class A, otherwise class B

Application of Logistic Regression

Binomial logistic regression.

If the number of assigned classes is two, then the logistic regression is binomial logistic regression; if the number of assigned classes is more than two, then it is multinomial logistic regression.

Logistic Regression

Logistic Regression models are used in Binary classification and multi-classification. They are widely used in email categorization, disease detection, fraud detection, and text emotion analysis

Model Evaluation Metrics

The performance of a logistic regression model is measured using evaluation metrics such as accuracy, precision, recall, F1-score, and confusion matrix.

This note is part of the AI & ML collection on NoteHub.