22.214.171.124. Plot variance and regularization in linear models¶
import numpy as np # Smaller figures import matplotlib.pyplot as plt plt.rcParams["figure.figsize"] = (3, 2)
We consider the situation where we have only 2 data point
Without noise, as linear regression fits the data perfectly
[<matplotlib.lines.Line2D object at 0x7f951cd668d0>]
In real life situation, we have noise (e.g. measurement noise) in our data:
As we can see, our linear model captures and amplifies the noise in the data. It displays a lot of variance.
We can use another linear estimator that uses regularization, the
Ridge estimator. This estimator
regularizes the coefficients by shrinking them to zero, under the
assumption that very high correlations are often spurious. The alpha
parameter controls the amount of shrinkage used.
Total running time of the script: (0 minutes 0.127 seconds)