Linear Regression Numerical Using Least Squares Method

Created

May 4, 2026

Last Modified

1 day ago

Linear Regression Numerical Using Least Squares Method

Linear regression is one of the most fundamental techniques in statistics and data science. It is used to model the relationship between two variables and to predict future values based on past data. In this article, we will solve a complete numerical problem using the Least Squares Method in a detailed, step-by-step manner.

Introduction to Linear Regression

Linear regression establishes a relationship between:

Independent variable (x)
Dependent variable (y)

It assumes that this relationship can be represented by a straight line:

y = a + b x

Where:

a is the intercept (value of y when x = 0)
b is the slope (rate of change of y with respect to x)

Least Squares Method

The Least Squares Method is used to determine the best-fit line. It minimizes the sum of squared differences between the observed values and the predicted values.

Problem Statement

Fit a straight line using linear regression and find the value of y when x = 11.

Given Data:

x	y
2	3
3	4
4	7
5	4
8	10
9	15

Step 1: Formula

We use the following formulas:

b = \frac{n \sum x y - ( \sum x ) ( \sum y )}{n \sum x ^{2} - ( \sum x ) ^{2}}

Where n is the number of observations.

Step 2: Create Calculation Table

x	y	x²	xy
2	3	4	6
3	4	9	12
4	7	16	28
5	4	25	20
8	10	64	80
9	15	81	135

Step 3: Calculate Summations

\sum x = 31

\sum y = 43

\sum x 2 = 199

\sum x y = 281

n = 6

Step 4: Calculate Slope (b)

b = \frac{6 ( 281 ) - ( 31 ) ( 43 )}{6 \times 199 - 3 1 ^{2}}

b = \frac{1686 - 1333}{1194 - 961}

b = \frac{355}{233} \approx 1.515

This means:
For every 1 unit increase in x, y increases by approximately 1.515

Step 5: Calculate Intercept (a)

a = \frac{43 - ( 1.515 \times 31 )}{6}

a = \frac{43 - 46.965}{6}

a \approx - 0.66

Step 6: Form the Regression Equation

y = - 0.66 + 1.515 x

Step 7: Predict Value for x = 11

y = - 0.66 + (1.515 \times 11)

y = - 0.66 + 16.665

y \approx 16.01

Final Result

The predicted value of y when x = 11 is approximately:

y \approx 16.01

Conclusion

Linear regression using the Least Squares Method provides a systematic way to model relationships between variables and make predictions. By carefully calculating sums, applying formulas, and forming the regression equation, we can accurately estimate unknown values. This method forms the foundation of many advanced techniques in data science and machine learning.