Lecture Note
University
Stanford UniversityCourse
CS229 | Machine LearningPages
2
Academic year
2023
anon
Views
12
Recognizing Gradient Descent and the Logistic Regression CostFunction For binary classiο¬cation issues, logistic regression is a well-liked machine learningtechnique. By using the input features, logistic regression attempts to forecast the likelihoodof a binary result (either 1 or 0). The values of the parameters w and b that minimize the costfunction J of w and b must be identiο¬ed in order to ο¬t the parameters of a logistic regressionmodel. Using gradient descent, this is accomplished. In this post, we'll concentrate on comprehending the logistic regression cost function andthe application of gradient descent to selecting appropriate w and b parameters. The Cost Function of Logistic Regression The effectiveness of the model is assessed using the logistic regression cost function. Inorder for the model to produce reliable predictions, the goal is to minimize the cost function. The cost function J for logistic regression is given by: π½ =β ( 1 π ) Β· β(π¦ Β· πππ(β(π₯)) + (1 β π¦) Β· πππ(1 β β(π₯))) where h(x) is the projected probability that the label is 1, m is the number of trainingexamples, y is the actual binary label, x is the input features, and The Cost Functions Derivatives with Respect to w and b It is necessary to compute the derivative of J with respect to each parameter in order tominimize the cost function J with respect to w and b. The derivative of J with respect to is given by: π€ π βπ½ βπ€ π = 1 π Β· β(β(π₯) β π¦) Β· π₯ π where is the jth feature of the training example i. π₯ π The derivative of J with respect to the parameter b is given by: βπ½ βπ = 1 π Β· β(β(π₯) β π¦) Logistic regression with Gradient Descent By continually updating each parameter as the value minus, the learning rate, times thederivative term, gradient descent is utilized to minimize the cost function J. The logisticregression gradient descent algorithm is expressed as follows: π€ π : = π€ π β π Β· ( βπ½ βπ€ π )
π: = π β π Β· ( βπ½ βπ ) where is the learning rate, which controls the size of the steps needed to get to the costfunction's minimum. Gradient Descent Algorithm convergence Similar to linear regression, the logistic regression gradient descent algorithm can bewatched to make sure it converges. To achieve this, determine whether the cost function J isgetting smaller after each iteration. Implementing Logistic Regression Vectorized To speed up the procedure, vectorization can also be used to implement the gradientdescent updates for logistic regression. In order to do this, all the updates are computed onthe right side at once, and all the left-hand values are concurrently overwritten.
Solving Binary Classification with Logistic Regression
Please or to post comments