Question

In the random forest construction, how many decision tress we need to use to get a good result? How can we use random forest algorithm for regression problem?

Answer 1

Actually, the more trees we use the better the results will be. However, the improvement decreases to some extent as the number of trees increases, i.e. at a certain point the benefit in prediction performance from learning more trees will be lower than the cost in computation time for learning the additional trees. Since random forests are ensemble methods. Therefore, in our case, we observe that in a single experiment on a single test set a forest of 10 trees performs better than a forest of 500 trees due to statistical variance.

Typical values for the number of trees is 10, 30, or 100, or much appropriately said that a random forest should have a number of trees between 64-128 trees. With that, one should have a good balance between ROC, AUC, and processing time.

Now, random forests can also be used for regression tasks. A random forest’s nonlinear nature can give it an advantage over linear algorithms, thus, making it a great option. But it is important to know random forests can never extend the data. Random forests can only make an assumption that it is an average of previously observed data. In a regression problem, the range of predictions by a random forest could make is bounded by the highest and lowest labels in the training data.