1. Explain the Decision Tree algorithm in detail.
While the Decision Tree algorithm is a supervised machine learning technique that can be used for regression and classification, it is most commonly utilized to solve problems with classification.
A decision tree is a diagram in the form of a tree that is used to identify a course of action. The dataset is divided into smaller subsets and is represented by the nodes of the tree. There are three nodes in the tree structure: the root, internal, or decision, and leaf nodes.
The top node is called the root node. It stands for the best attribute that was chosen for categorization. The decision nodes' internal nodes test a dataset attribute, while the terminal or leaf nodes provide the decision or classification label.
The Classification and Regression Tree algorithm, or CART algorithm, is used for creating trees.
The features of the provided dataset are used to make decisions or run the test.
A decision tree only presents an issue, and it further divides the tree into subtrees according to the response (Yes/No).
2. Explain Decision tree Key terms: Root Node, Decision Node/Branch Node, Leaf or Terminal Node.
Root Node
In a decision tree, the root node is always at the top. It can be further separated into many sets and represents the full population or data sample.
Decision Node
Decision nodes have at least two branches and are sub-nodes that can divide into other sub-nodes.
Leaf Node
In a decision tree, the outcomes are carried by the leaf node. It is not possible to separate these nodes—also referred to as terminal nodes—any far enough.
3. What are the Steps for Making a decision tree?
1.Choose the initial dataset with the feature and target attributes defined.
2.Calculate the Information gain and Entropy for each attribute.
3.Pick the attribute with the highest information gain and make it the decision root node.
4.Calculate the information gain for the remaining attributes.
5.Create recurring child nodes by starting splitting at the decision node (i.e. for various values of the decision node, create separate child nodes).
6.Repeat this process until all the attributes are covered.
7.Prune the Tree to prevent overfitting.
4. What is a Puresubset
A pure subset is a subset that contains only samples of one class.
5. What are Techniques to avoid Overfitting in Decision Tree?
Pruning
Hyperparameter Tuning
Random Forest algorithm
Ensembling technique:a.Bagging b. Boosting
6. How to tune hyperparameters in decision trees Classifier?
GridSearchCV
This involves a thorough search of a list containing every potential hyperparameter value. Throughout the testing of every feasible combination, the top-performing model can be chosen.
RandomisedSearchCV
Random samples are chosen at random from the hyperparameter space using this method. The hyperparameter space can be defined using statistical distributions or lists, similar to what is done in Grid Search.
7. What is pruning in the Decision Tree?
The process of eliminating the decision tree's unnecessary branches is called pruning. The decision tree may have some branches that indicate noisy or anomalous data.
The process of reducing an undesired branch on a tree is called tree pruning. As a result, the tree will be less convoluted and more conducive to accurate predictive analysis. As it prunes the unnecessary branches off the trees, it reduces overfitting.
8. What are the advantages and disadvantages of the Decision Tree?
Advantages
1. The decision tree model can be used for both classification and regression problems, and it is easy to interpret, understand, and visualize.
2. The output of a decision tree can also be easily understood.
3. Compared with other algorithms, data preparation during pre-processing in a decision tree requires less effort and does not require normalization of data.
4. The implementation can also be done without scaling the data.
5. A decision tree is one of the quickest ways to identify relationships between variables and the most significant variable.
6. New features can also be created for better target variable prediction.
7. Decision trees are not largely influenced by outliers or missing values, and it can handle both numerical and categorical variables.
8. Since it is a non-parametric method, it has no assumptions about space distributions and classifier structure.
Disadvantages
1. Overfitting is one of the practical difficulties for decision tree models. It happens when the learning algorithm continues developing hypotheses that reduce the training set error but at the cost of increasing test set error. But this issue can be resolved by pruning and setting constraints on the model parameters.
2. Decision trees cannot be used well with continuous numerical variables.
3. A small change in the data tends to cause a big difference in the tree structure, which causes instability.
4. Calculations involved can also become complex compared to other algorithms, and it takes a longer time to train the model.
5. It is also relatively expensive as the amount of time taken and the complexity levels are greater.
9. What is Decision Tree Regressor?
Decision tree regression observes features of an object and trains a model in the structure of a tree to predict data in the future to produce meaningful continuous output. Continuous output means that the output/result is not discrete, i.e., it is not represented just by a discrete, known set of numbers or values.
Discrete output example: A weather prediction model that predicts whether or not there’ll be rain in a particular day. Continuous output example: A profit prediction model that states the probable profit that can be generated from the sale of a product. Here, continuous values are predicted with the help of a decision tree regression model.
Mean squared error (MSE) is usually used to decide whether to split a node into two or more sub-nodes in a decision tree regression. In the case of a binary tree, the algorithm picks a value and splits the data into two subsets, calculates MSE for each subset, and chooses the smallest MSE value as a result.
10. What are the Applications of Decision Trees
1. To determine whether a loan application is likely to default, a decision tree is applied.
2. It can be applied to determine an individual's likelihood of contracting a certain illness.
3. It can assist retailers on the internet in forecasting the likelihood that a customer would buy a specific product.
4. Customer churn rates can also be determined using decision trees.