Why decision tree

Try using a decision tree maker. Want to make a decision tree of your own? Try Lucidchart. It's quick, easy, and completely free. A decision tree is a map of the possible outcomes of a series of related choices. It allows an individual or organization to weigh possible actions against one another based on their costs, probabilities, and benefits.

They can can be used either to drive informal discussion or to map out an algorithm that predicts the best choice mathematically. A decision tree typically starts with a single node, which branches into possible outcomes. Each of those outcomes leads to additional nodes, which branch off into other possibilities. This gives it a treelike shape. There are three different types of nodes: chance nodes, decision nodes, and end nodes. A chance node, represented by a circle, shows the probabilities of certain results.

A decision node, represented by a square, shows a decision to be made, and an end node shows the final outcome of a decision path. Decision trees can also be drawn with flowchart symbols , which some people find easier to read and understand. To draw a decision tree, first pick a medium. You can draw it by hand on paper or a whiteboard, or you can use special decision tree software. In either case, here are the steps to follow:.

Start with the main decision. Draw a small box to represent this point, then draw a line from the box to the right for each possible solution or action. Label them accordingly. From each decision node, draw possible solutions. From each chance node, draw lines representing possible outcomes. If you intend to analyze your options numerically, include the probability of each outcome and the cost of each action.

Continue to expand until every line reaches an endpoint , meaning that there are no more choices to be made or chance outcomes to consider. Then, assign a value to each possible outcome. It could be an abstract score or a financial value. Add triangles to signify endpoints. Diagramming is quick and easy with Lucidchart. Start a free trial today to start creating and collaborating. By calculating the expected utility or value of each choice in the tree, you can minimize risk and maximize the likelihood of reaching a desirable outcome.

To calculate the expected utility of a choice, just subtract the cost of that decision from the expected benefits. For instance, some may prefer low-risk options while others are willing to take risks for a larger benefit.

To do so, simply start with the initial event, then follow the path from that event to the target event, multiplying the probability of each of those events together. In this way, a decision tree can be used like a traditional tree diagram, which maps out the probabilities of certain events, such as flipping a coin twice.

However, decision trees can become excessively complex. Or high initial demand might indicate the possibility of a sustained high-volume market. If demand is high and the company does not expand within the first two years, competitive products will surely be introduced. If the company builds a big plant, it must live with it whatever the size of market demand. If it builds a small plant, management has the option of expanding the plant in two years in the event that demand is high during the introductory period; while in the event that demand is low during the introductory period, the company will maintain operations in the small plant and make a tidy profit on the low volume.

Management is uncertain what to do. The new product, if the market turns out to be large, offers the present management a chance to push the company into a new period of profitable growth. The development department, particularly the development project engineer, is pushing to build the large-scale plant to exploit the first major product development the department has produced in some years. The chairman, a principal stockholder, is wary of the possibility of large unneeded plant capacity.

He favors a smaller plant commitment, but recognizes that later expansion to meet high-volume demand would require more investment and be less efficient to operate. The chairman also recognizes that unless the company moves promptly to fill the demand which develops, competitors will be tempted to move in with equivalent products.

The Stygian Chemical problem, oversimplified as it is, illustrates the uncertainties and issues that business management must resolve in making investment decisions.

These decisions are growing more important at the same time that they are increasing in complexity. Countless executives want to make them better—but how? The decision tree can clarify for management, as can no other analytical tool that I know of, the choices, risks, objectives, monetary gains, and information needs involved in an investment problem.

We shall be hearing a great deal about decision trees in the years ahead. Although a novelty to most businessmen today, they will surely be in common management parlance before many more years have passed.

Later in this article we shall return to the problem facing Stygian Chemical and see how management can proceed to solve it by using decision trees.

First, however, a simpler example will illustrate some characteristics of the decision-tree approach. Let us suppose it is a rather overcast Saturday morning, and you have 75 people coming for cocktails in the afternoon. You have a pleasant garden and your house is not too large; so if the weather permits, you would like to set up the refreshments in the garden and have the party there.

It would be more pleasant, and your guests would be more comfortable. On the other hand, if you set up the party for the garden and after all the guests are assembled it begins to rain, the refreshments will be ruined, your guests will get damp, and you will heartily wish you had decided to have the party in the house.

We could complicate this problem by considering the possibility of a partial commitment to one course or another and opportunities to adjust estimates of the weather as the day goes on, but the simple problem is all we need.

Much more complex decision questions can be portrayed in payoff table form. However, particularly for complex investment decisions, a different representation of the information pertinent to the problem—the decision tree—is useful to show the routes by which the various possible outcomes are achieved. The problem is posed in terms of a tree of decisions. Exhibit I illustrates a decision tree for the cocktail party problem. This tree is a different way of displaying the same information shown in the payoff table.

However, as later examples will show, in complex decisions the decision tree is frequently a much more lucid means of presenting the relevant information than is a payoff table. The tree is made up of a series of nodes and branches. At the first node on the left, the host has the choice of having the party inside or outside.

Each branch represents an alternative course of action or decision. At the end of each branch or alternative course is another node representing a chance event—whether or not it will rain. Each subsequent alternative course to the right represents an alternative outcome of this chance event.

Associated with each complete alternative course through the tree is a payoff, shown at the end of the rightmost or terminal branch of the course. When I am drawing decision trees, I like to indicate the action or decision forks with square nodes and the chance-event forks with round ones. Other symbols may be used instead, such as single-line and double-line branches, special letters, or colors.

It does not matter so much which method of distinguishing you use so long as you do employ one or another. A decision tree of any size will always combine a action choices with b different possible events or results of action which are partially affected by chance or other uncontrollable circumstances.

The previous example, though involving only a single stage of decision, illustrates the elementary principles on which larger, more complex decision trees are built. Let us take a slightly more complicated situation:. You are trying to decide whether to approve a development budget for an improved product. You are urged to do so on the grounds that the development, if successful, will give you a competitive edge, but if you do not develop the product, your competitor may—and may seriously damage your market share.

You sketch out a decision tree that looks something like the one in Exhibit II. Your initial decision is shown at the left.

Following a decision to proceed with the project, if development is successful, is a second stage of decision at Point A. Assuming no important change in the situation between now and the time of Point A, you decide now what alternatives will be important to you at that time. At the right of the tree are the outcomes of different sequences of decisions and events.

These outcomes, too, are based on your present information. Of course, you do not try to identify all the events that can happen or all the decisions you will have to make on a subject under analysis. In the decision tree you lay out only those decisions and events or results that are important to you and have consequences you wish to compare.

For more illustrations, see the Appendix. We shall not concern ourselves here with costs, yields, probabilities, or expected values. The choice of alternatives in building a plant depends upon market forecasts. The alternative chosen will, in turn, affect the market outcome. For example, the military products division of a diversified firm, after some period of low profits due to intense competition, has won a contract to produce a new type of military engine suitable for Army transport vehicles.

The division has a contract to build productive capacity and to produce at a specified contract level over a period of three years. Figure A illustrates the situation. The dotted line shows the contract rate. The solid line shows the proposed buildup of production for the military.

Some other possibilities are portrayed by dashed lines. The company is not sure whether the contract will be continued at a relatively high rate after the third year, as shown by Line A, or whether the military will turn to another newer development, as indicated by Line B.

The company has no guarantee of compensation after the third year. There is also the possibility, indicated by Line C, of a large additional commercial market for the product, this possibility being somewhat dependent on the cost at which the product can be made and sold. If this commercial market could be tapped, it would represent a major new business for the company and a substantial improvement in the profitability of the division and its importance to the company. It might undertake the major part of the fabrication itself but use general-purpose machine tools in a plant of general-purpose construction.

But, the fully grown tree is likely to overfit the data, leading to poor accuracy on unseen data. In pruning , you trim off the branches of the tree, i. This is done by segregating the actual training set into two sets: training data set, D and validation data set, V. Prepare the decision tree using the segregated training data set, D. Then continue trimming the tree accordingly to optimize the accuracy of the validation data set, V.

Random Forest is an example of ensemble learning, in which we combine multiple machine learning algorithms to obtain better predictive performance. A technique known as bagging is used to create an ensemble of trees where multiple training sets are generated with replacement. In the bagging technique, a data set is divided into N samples using randomized sampling. Then, using a single learning algorithm a model is built on all samples.

Later, the resultant predictions are combined using voting or averaging in parallel. The dataset that we have is a supermarket data which can be downloaded from here. Load all the basic libraries. Load the dataset. We will take only Age and EstimatedSalary as our independent variables X because of other features like Gender and User ID are irrelevant and have no effect on the purchasing capacity of a person. Purchased is our dependent variable y. For plotting trees, you also need to install Graphviz and pydotplus.

In the decision tree chart, each internal node has a decision rule that splits the data. Gini referred to as the Gini ratio, which measures the impurity of the node. You can say a node is pure when all of its records belong to the same class, such nodes known as the leaf node. Here, the resultant tree is unpruned. This unpruned tree is unexplainable and not easy to understand. The higher value of maximum depth causes overfitting, and a lower value causes underfitting Source. In Scikit-learn, optimization of decision tree classifier performed by only pre-pruning.

The maximum depth of the tree can be used as a control variable for pre-pruning. This pruned model is less complex, explainable, and easy to understand than the previous decision tree model plot. Reposted with permission. By subscribing you accept KDnuggets Privacy Policy. Decision Tree Algorithm, Explained Previous post. Tags: Algorithms , Decision Trees , Explained.

Gini Index. Gain Ratio. Pruning in action. Random Forest in action. Decision Tree. Decision Tree after pruning. Previous post.