Coding Logistic Regression in Python

In this blog you will learn how to code logistic regression from scratch in python.

Before going into the code let’s understand the math behind logistic regression and training the model using gradient descent .

Logistic Regression :

The name of this algorithm is logistic regression because of the logistic function that we use in this algorithm. In this we linearly combine the inputs(X) and the weights/coefficients to give the output (y).

The output of this model scaled between 0 and 1 which acts as probability of the data point belonging to a particular class. We use sigmoid function to achieve this objective.

Hey! Wait .

Taken from Tenor

Just follow the following steps and you will learn how it works.

1. First of all all the inputs are multiplied with their respective weights and are summed up and we call this weighted sum.

Calculating Dot product of weights and input

2. Then we pass this weighted sum to sigmoid function which gives a value between 0 and 1 which is the probability of a data point belonging to a class.

Using Sigmoid to scale the value between 0 and 1.

3. Then we calculate the loss using the following loss function .

The weights/coefficients is a n dimensional vector that we have to learn using gradient descent. Now we will see how to learn those weights.

Gradient Descent:

Gradient descent is the optimization technique in which we use the gradient of the loss function to update our weights. Now we will see how to update the weights using this.

Show me the code:

I have implemented the logistic regression class. This code does not have regularization implemented .

Implementation of logistic regression.

Code to plot graphs for visualization.

Let’s plot some graphs to visualize how the model learns.

Epoch 1

Epoch 2

Epoch 5

Epoch 10

This is all for now. If you have any questions then, write in the comments.

If you are a beginner and wanted to know what is calibration then go through this blog.

Calibration in Machine Learning