Solving Binary Classification with Logistic Regression

University
Stanford University
Course
CS229 | Machine Learning
Pages

2

Academic year

2023
Author

anon
Views

12

Recognizing Gradient Descent and the Logistic Regression CostFunction For binary classiﬁcation issues, logistic regression is a well-liked machine learningtechnique. By using the input features, logistic regression attempts to forecast the likelihoodof a binary result (either 1 or 0). The values of the parameters w and b that minimize the costfunction J of w and b must be identiﬁed in order to ﬁt the parameters of a logistic regressionmodel. Using gradient descent, this is accomplished. In this post, we'll concentrate on comprehending the logistic regression cost function andthe application of gradient descent to selecting appropriate w and b parameters. The Cost Function of Logistic Regression The effectiveness of the model is assessed using the logistic regression cost function. Inorder for the model to produce reliable predictions, the goal is to minimize the cost function. The cost function J for logistic regression is given by: 𝐽 =− ( 1 𝑚 ) · ∑(𝑦 · 𝑙𝑜𝑔(ℎ(𝑥)) + (1 − 𝑦) · 𝑙𝑜𝑔(1 − ℎ(𝑥))) where h(x) is the projected probability that the label is 1, m is the number of trainingexamples, y is the actual binary label, x is the input features, and The Cost Functions Derivatives with Respect to w and b It is necessary to compute the derivative of J with respect to each parameter in order tominimize the cost function J with respect to w and b. The derivative of J with respect to is given by: 𝑤 𝑗 ∂𝐽 ∂𝑤 𝑗 = 1 𝑚 · ∑(ℎ(𝑥) − 𝑦) · 𝑥 𝑗 where is the jth feature of the training example i. 𝑥 𝑗 The derivative of J with respect to the parameter b is given by: ∂𝐽 ∂𝑏 = 1 𝑚 · ∑(ℎ(𝑥) − 𝑦) Logistic regression with Gradient Descent By continually updating each parameter as the value minus, the learning rate, times thederivative term, gradient descent is utilized to minimize the cost function J. The logisticregression gradient descent algorithm is expressed as follows: 𝑤 𝑗 : = 𝑤 𝑗 − 𝑎 · ( ∂𝐽 ∂𝑤 𝑗 )

𝑏: = 𝑏 − 𝑎 · ( ∂𝐽 ∂𝑏 ) where is the learning rate, which controls the size of the steps needed to get to the costfunction's minimum. Gradient Descent Algorithm convergence Similar to linear regression, the logistic regression gradient descent algorithm can bewatched to make sure it converges. To achieve this, determine whether the cost function J isgetting smaller after each iteration. Implementing Logistic Regression Vectorized To speed up the procedure, vectorization can also be used to implement the gradientdescent updates for logistic regression. In order to do this, all the updates are computed onthe right side at once, and all the left-hand values are concurrently overwritten.