Ridge regression works by adding a penalty term to the ordinary least squares (OLS) cost function. This penalty term is proportional to the square of the magnitude of the coefficients. The modified cost function can be represented as: $$ L(\beta) = \sum_{i=1}^{n} (y_i - \beta_0 - \sum_{j=1}^{p} \beta_j x_{ij})^2 + \lambda \sum_{j=1}^{p} \beta_j^2 $$ Here, \( \lambda \) is the regularization parameter that controls the amount of shrinkage. When \( \lambda = 0 \), ridge regression reduces to OLS. As \( \lambda \) increases, the coefficients \( \beta \) shrink towards zero.
