Which ensemble method uses multiple decision trees and reduces variance by averaging?

Which ensemble method uses multiple decision trees and reduces variance by averaging?

• A. Random forest
• B. Gradient boosting
• C. AdaBoost
• D. Voting classifier

Answer: (A)

An ensemble that builds many decision trees and reduces variance by averaging their outputs is designed to stabilize predictions by lessening the impact of any one tree’s fluctuations. This is exactly what a random forest does: it creates numerous decorrelated decision trees (typically via bootstrap sampling and random feature selection) and then averages their predictions (or uses majority voting for classification). The averaging across many trees dampens the variance of the model, making it more robust to overfitting than a single decision tree. Other approaches don’t fit this pattern as neatly. Gradient boosting builds trees sequentially to correct the errors of previous trees, focusing on improving accuracy (reducing bias) rather than simply averaging to reduce variance. AdaBoost similarly adjusts emphasis on difficult instances across rounds, again prioritizing error correction over variance reduction by averaging. A voting classifier combines different models by voting their outputs, which may include trees but isn’t defined by the principle of averaging predictions from many trees to reduce variance.

An ensemble that builds many decision trees and reduces variance by averaging their outputs is designed to stabilize predictions by lessening the impact of any one tree’s fluctuations. This is exactly what a random forest does: it creates numerous decorrelated decision trees (typically via bootstrap sampling and random feature selection) and then averages their predictions (or uses majority voting for classification). The averaging across many trees dampens the variance of the model, making it more robust to overfitting than a single decision tree.

Other approaches don’t fit this pattern as neatly. Gradient boosting builds trees sequentially to correct the errors of previous trees, focusing on improving accuracy (reducing bias) rather than simply averaging to reduce variance. AdaBoost similarly adjusts emphasis on difficult instances across rounds, again prioritizing error correction over variance reduction by averaging. A voting classifier combines different models by voting their outputs, which may include trees but isn’t defined by the principle of averaging predictions from many trees to reduce variance.