Question

What is the trade-off between bias and variance in a statistical model?

Answer 1

As a result of incorrect or too simplistic assumptions in the learning algorithm you are applying, bias gets introduced into the system. This can result in the model underfitting your data, making it difficult for it to have high predicted accuracy and for you to generalize your knowledge from the training set to the test set as a result of the underfitting.

Variance is a mistake caused by an excessive amount of complexity in the learning process you're employing. In turn, this results in the algorithm being extremely sensitive to high degrees of variation in your training data, which might lead to your model being overfitted as a result. You'll be bringing too much noise from your training data with you, and your model won't be very effective when it comes to your test data.

When using the bias-variance decomposition, the learning error from any algorithm is effectively decomposed by adding the bias, the variance, and a small amount of irreducible error owing to noise in the underlying dataset to the learning error. Overall, as the model becomes more sophisticated and includes additional variables, the bias decreases while the variance increases; hence, in order to achieve the best decreased level of error, you must trade off bias and variance in some way. Neither strong bias nor high variance are desirable characteristics in your model.