Implementing the random forest model on scikit learn cancer dataset

Random Forest, a collection of decision trees, is one of the ensemble methods ,which combine machine learning models into a more powerful machine learning model. It is better than single decision tree because while retaining the predictive powers it can reduce over-fitting by averaging the results. Let us  implement the random forest model on scikit learn cancer dataset.

Import the necessary packages:

from sklearn.ensemble import RandomForestClassifier

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
import matplotlib.pyplot as plt
import numpy as np

Now, we need to provide the dataset which can be done as follows:

cancer = load_breast_cancer()

X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target, random_state=0)

After providing the dataset, we need to fit the model which can be done as follows:

forest = RandomForestClassifier(n_estimators=50, random_state=0)

forest.fit(X_train,y_train)

Now, get the accuracy on training as well as testing subset: if we will increase the number of estimators then, the accuracy of testing subset would also be increased.

print('Accuracy on the training subset:(:.3f)',format(forest.score(X_train,y_train)))
print('Accuracy on the training subset:(:.3f)',format(forest.score(X_test,y_test)))

Output

Accuracy on the training subset:(:.3f) 1.0
Accuracy on the training subset:(:.3f) 0.965034965034965

Now, like the decision tree, random forest has the feature_importance module which will provide a better view of feature weight than decision tree. It can be plot and visualize as follows:

n_features = cancer.data.shape[1]
plt.barh(range(n_features),forest.feature_importances_, align='center')
plt.yticks(np.arange(n_features),cancer.feature_names)
plt.xlabel('Feature Importance')
plt.ylabel('Feature')
plt.show()

The resulting plot is as shown below:

In the next post we'll discuss about Performance of a classifier.