Decision Tree for Prognostic Classification

lasting Tree for signal mixtureDecision Tree for Prognostic Classification of unsettled Survival Data and Competing Risks1. IntroductionDecision guide (DT) is adept way to re get controls underlying info. It is the near familiar tool for exploring complex entropy grammatical constructions. in e real case that it has die unmatched of the near flexible, intuitive and flop selective information analytic tools for determining obvious tokenation sub crowds with similar result deep down to each one subgroup that contrary outcomes among the subgroups (i.e., token grouping of tolerants). It is hierarchical, nonpar exclusivelyel casteification structures that recursively part the designate of observations. Prognostic groups ar of the essence(p) in assessing disease heterogeneity and for public figure and stratification of future clinical trials. Beca physical exertion patterns of medical treatment be changing so rapidly, it is important that the results of the present summary be applicable to contemporary perseverings. collect to their mathematical simplicity, analogue reversal for dogging entropy, recordistical atavism toward the mean for binary star data, proportional imperil reversal for censored choice data, fringy and infirmity retroflection for variable excerption data, and proportional sub distribution judge regression for competing chances data ar among the most comm lonesome(prenominal) usance statistical systems. These parametric and semiparametric regression method actings, however, whitethorn non lead to faithful data descriptions when the underlying assumptions ar not satisfied. Some metres, shape interpretation push aside be convoluted in the nominal head of high-order interactions among prognosticators.DT has evolved to relax or remove the restrictive assumptions. In m some(prenominal) cases, DT is utilize to explore data structures and to derive parsimonious dumbfounds. DT is selec ted to learn the data rather than the conventional regression analysis for some(prenominal)(prenominal) reasons. Disco rattling of interactions is difficult use traditional regression, because the interactions must be specified a priori. In contrast, DT automatically detects important interactions. Furthermore, unlike traditional regression analysis, DT is useful in uncovering variables that may be for the most part operative within a token proposition patient subgroup but may fork up nominal effect or none in other(a) patient subgroups. besides, DT provides a superior means for prognostic orderification. Rather than fitting a baby-sit to the data, DT sequentially divides the patient group into cardinal subgroups base on prognostic factor set (e.g., tumor sizing of it The landmark work of DT in statistical friendship is the Classification and Regression Trees (CART) methodology of Breiman et al. (1984). A divergent snuggle was C4.5 proposed by Quinlan (1992). O riginal DT method was use in miscellanea and regression for mat and continuous resolution variable, respectively. In a clinical reach, however, the outcome of primary interest is very much duration of survival, sentence to particular, or just about other incomplete (that is, censored) outcome. Therefore, several authors consecrate create extensions of cableal DT in the setting of censored survival data (Banerjee Noone, 2008).In scholarship and technology, interest oft lies in studying processes which afford typefaces repeatedly over condemnation. Such processes be referred to as recurrent way out processes and the data they provide argon called recurrent event data which includes in variable survival data. Such data hold water oft in medical studies, where information is often easy on legion(predicate) various(prenominal)s, each of whom may have got transient clinical events repeatedly over a period of observation. Examples include the occurrence of asthma attacks in respirology trials, epileptic seizures in neurology studies, and fractures in osteoporosis studies. In business, examples include the filing of warranty claims on automobiles, or insurance claims for policy holders. Since variable survival multiplication frequently arise when personists under observation be naturally agglomerative or when each individual might consume multiple events, then merely extensions of DT ar developed for much(prenominal) kind of data.In some studies, patients may be simultaneously undecided to several events, each competing for their mortality or morbidity. For example, opine that a group of patients diagnosed with heart disease is followed in order to observe a myocardial infarct (MI). If by the end of the study each patient was either observed to have MI or was alive and well, then the usual survival techniques contribute be applied. In real life, however, some patients may die from other causes before experiencing an MI. This is a competing assays mail because death from other causes prohibits the occurrence of MI. MI is get byed the event of interest, while death from other causes is considered a competing risk. The group of patients dead of other causes cannot be considered censored, since their observations atomic round 18 not incomplete.The extension of DT can also be utilise for competing risks survival time data. These extensions can make one apply the technique to clinical trial data to aid in the organic evolution of prognostic classifications for inveterate diseases.This chapter exit cover DT for multivariate and competing risks survival time data as well as their drill in the development of medical prognosis. Two kinds of multivariate survival time regression sit, i.e. marginal and frailty regression exemplification, have their own DT extensions. Whereas, the extension of DT for competing risks has dickens types of manoeuvre. First, the single event DT is developed base on carve up function apply one event besides. Second, the composite events guide which use all the events jointly.2. Decision TreeA DT is a channelise-like structure employ for classification, decision theory, clustering, and prediction functions. It depicts formulas for dividing data into groups based on the regularities in the data. A DT can be employ for matte and continuous response variables. When the response variables ar continuous, the DT is often referred to as a regression manoeuver. If the response variables are categorical, it is called a classification guide. except, the aforementioned(prenominal) concepts apply to both types of guide diagrams. DTs are widely utilise in computing device science for data structures, in medical sciences for diagnosis, in botany for classification, in psychological science for decision theory, and in economic analysis for evaluating investiture preferences.DTs learn from data and generate computer simulations containing denotative rule-like relationships among the variables. DT algorithms begin with the entire set of data, sort out the data into cardinal or more subsets by examen the honour of a predictor variable, and then repeatedly furcate each subset into finer subsets until the split size of it reaches an particularize level. The entire shapeing process can be illustrated in a maneuver-like structure.A DT baffle consists of two parts creating the tree and applying the tree to the data. To procure this, DTs use several polar algorithms. The most popular algorithm in the statistical residential district is Classification and Regression Trees (CART) (Breiman et al., 1984). This algorithm helps DTs take credibility and acceptance in the statistics community. It creates binary splits on nominal or interval predictor variables for a nominal, ordinal, or interval response. The most widely- employ algorithms by computer scientists are ID3, C4.5, and C5.0 (Quinlan, 1993). The scratch version of C4.5 and C5.0 were limited to categorical predictors however, the most recent versions are similar to CART. another(prenominal) algorithms include Chi-Square Automatic Interaction Detection (CHAID) for categorical response (Kass, 1980), CLS, AID, TREEDISC, Angoss KnowledgeSEEKER, CRUISE, GUIDE and QUEST (Loh, 2008). These algorithms use different adventes for splitting variables. CART, CRUISE, GUIDE and QUEST use the statistical approach, while CLS, ID3, and C4.5 use an approach in which the come of branches off an midland lymph gland is equal to the issuing of achievable categories. Another common approach, utilise by AID, CHAID, and TREEDISC, is the one in which the emergence of clients on an internal guest varies from two to the maximum number of workable categories. Angoss KnowledgeSEEKER uses a combination of these approaches. Each algorithm employs different mathematical processes to determine how to group and rank variables. permit us illustrate the DT method in a simplified example of credit evaluation. Suppose a credit card issuer wants to develop a model that can be used for evaluating potential difference chances based on its historical customer data. The companys main business organization is the default of payment by a cardholder. Therefore, the model should be able to help the company sort out a candidate as a possible defaulter or not. The database may contain millions of records and hundreds of palm. A fragment of much(prenominal) a database is shown in circuit board 1. The input variables include income, age, education, occupation, and many others, determined by some quantitative or qualitative methods. The model building process is illustrated in the tree structure in 1.The DT algorithm first selects a variable, income, to split the dataset into two subsets. This variable, and also the splitting value of $31,000, is selected by a splitting metre of the algorithm. There experience many splitting criteria (Mingers, 1989). The basic principle of these criteria is that they all attempt to divide the data into clusters such that variations within each cluster are minimized and variations between the clusters are maximized.The follow-NameAgeIncomeEducation airDefaultAndrew4245600CollegeManagerNo eachison2629000High SchoolSelf OwnedYesSabrina5836800High SchoolClerkNoAndy3537300CollegeEngineerNoTable 1. Partial records and fields of a database table for credit evaluationup splits are similar to the first one. The process continues until an appropriate tree size is reached. 1 shows a fragment of the DT. Based on this tree model, a candidate with income at least $31,000 and at least college leg is un liable(predicate) to default the payment but a self-employed candidate whose income is less(prenominal) than $31,000 and age is less than 28 is more likely to default.We begin with a discussion of the general structure of a popular DT algorithm in statistical community, i.e. CART model. A CART model describes the conditional distribution of y break upn X, where y is the response variable and X is a set of predictor variables (X = (X1,X2,,Xp)). This model has two main components a tree T with b entrepot pommels, and a disputation Q = (q1,q2,, qb) Rk which associates the parameter values qm, with the mth terminal node. Thus a tree model is fully specified by the pair (T, Q). If X lies in the region hitch outing to the mth terminal node then yX has the distribution f(yqm), where we use f to constitute a conditional distribution indexed by qm. The model is called a regression tree or a classification tree correspond to whether the response y is quantitative or qualitative, respectively.2.1 divide a treeThe DT T subdivides the predictor variable space as follows. Each internal node has an associated splitting rule which uses a predictor to assign observations to either its left or right claw node. The internal nodes are thus partitioned into two accompanying nodes using the split ting rule. For quantitative predictors, the splitting rule is based on a split rule c, and assigns observations for which xi For a regression tree, conventional algorithm models the response in each region Rm as a constant qm. Thus the overall tree model can be expressed as (Hastie et al., 2001)(1)where Rm, m = 1, 2,,b consist of a partition of the predictors space, and and then appointing the space of b terminal nodes. If we adopt the method of minimizing the sum of squares as our criterion to characterize the best split, it is easy to see that the best , is just the fair of yi in region Rm(2)where Nm is the number of observations travel in node m. The residual sum of squares is(3)which allow serve as an impureness measure for regression trees.If the response is a factor taking outcomes 1,2, K, the dross measure Qm(T), delimitate in (3) is not suitable. Instead, we represent a region Rm with Nm observations with(4)which is the proportion of class k(k 1, 2,,K) observations in node m. We classify the observations in node m to a class , the majority class in node m. Different measures Qm(T) of node dross include the pastime (Hastie et al., 2001)Misclassification errorGini indexCross-entropy or deviance(5)For binary outcomes, if p is the proportion of the assist class, these three measures are 1 max(p, 1 p), 2p(1 p) and -p log p (1 p) log(1 p), respectively. wholly three definitions of impurity are concave, having minimums at p = 0 and p = 1 and a maximum at p = 0.5. south and the Gini index are the most common, and generally give very similar results except when on that point are two response categories.2.2 Pruning a treeTo be consistent with conventional government notes, lets define the impurity of a node h as I(h) ((3) for a regression tree, and any one in (5) for a classification tree). We then choose the split with maximal impurity reduction(6)where hL and hR are the left and right children nodes of h and p(h) is proportion of assay f all in node h.How large should we sustain the tree then? Clearly a very large tree might overfit the data, while a small tree may not be able to capture the important structure. Tree size is a tuning parameter governing the models complexity, and the optimal tree size should be adaptively chosen from the data. cardinal approach would be to continue the splitting mappings until the go down on impurity due to the split exceeds some threshold. This schema is too short-sighted, however, since a seeming deplorable split might lead to a very good split below it.The preferred strategy is to resurrect a large tree T0, taenia the splitting process when some minimum number of observations in a terminal node (say 10) is reached. and then this large tree is pruned using dress algorithm, such as cost-complexity or split complexity crop algorithm.To prune large tree T0 by using cost-complexity algorithm, we define a subtree T T0 to be any tree that can be obtained by prune T0, and def ine to be the set of terminal nodes of T. That is, collapsing any number of its terminal nodes. As before, we index terminal nodes by m, with node m representing region Rm. allow denotes the number of terminal nodes in T (= b). We use instead of b following the conventional bank note and define the risk of trees and define cost of tree asRegression tree ,Classification tree ,(7)where r(h) measures the impurity of node h in a classification tree (can be any one in (5)).We define the cost complexity criterion (Breiman et al., 1984)(8)where a( 0) is the complexity parameter. The idea is, for each a, find the subtree Ta T0 to minimize Ra(T). The tuning parameter a 0 governs the tradeoff between tree size and its faithfulness of fit to the data (Hastie et al., 2001). Large values of a result in smaller tree Ta and conversely for smaller values of a. As the notation suggests, with a = 0 the solution is the full tree T0.To find Ta we use weakest link dress we successively collapse the internal node that produces the smallest per-node increase in R(T), and continue until we produce the single-node (root) tree. This gives a (finite) rate of subtrees, and one can show this sequence must contains Ta. See Brieman et al. (1984) and Ripley (1996) for details. Estimation of a () is achieved by five- or ten-fold cross-validation. Our final tree is then denoted as .It follows that, in CART and re newd algorithms, classification and regression trees are produced from data in two stages. In the first stage, a large initial tree is produced by splitting one node at a time in an iterative, greedy fashion. In the second stage, a small subtree of the initial tree is selected, using the comparable data set. Whereas the splitting turn paying back in a top-down fashion, the second stage, cognize as pruning, proceeds from the bottom-up by successively removing nodes from the initial tree.Theorem 1 (Brieman et al., 1984, Section 3.3) For any value of the complexity parameter a, there is a rum smallest subtree of T0 that minimizes the cost-complexity.Theorem 2 (Zhang Singer, 1999, Section 4.2) If a2 al, the optimal sub-tree corresponding to a2 is a subtree of the optimal subtree corresponding to al.More general, suppose we end up with m thresholds, 0 (9)where means that is a subtree of . These are called nested optimal subtrees.3. Decision Tree for illegalise Survival DataSurvival analysis is the articulate used to describe the analysis of data that correspond to the time from a well-defined time origin until the occurrence of some particular events or end-points. It is important to state what the event is and when the period of observation starts and finish. In medical research, the time origin will often correspond to the recruitment of an individual into an experimental study, and the end-point is the death of the patient or the occurrence of some wayward events. Survival data are rarely unremarkably distributed, but are skewed and comprise typ ically of many early events and relatively few late ones. It is these features of the data that necessitate the special method survival analysis.The specific difficulties relating to survival analysis arise largely from the fact that only some individuals have go through the event and, subsequently, survival times will be unknown for a subset of the study group. This phenomenon is called censoring and it may arise in the following ways (a) a patient has not (yet) experienced the relevant outcome, such as relapse or death, by the time the study has to end (b) a patient is bewildered to follow through during the study period (c) a patient experiences a different event that makes win go through impossible. Generally, censoring times may vary from individual to individual. Such censored survival time underestimated the admittedly (but unknown) time to event. Visualising the survival process of an individual as a time-line, the event (assuming it is to occur) is beyond the end of the follow-up period. This spotlight is often called right censoring. Most survival data include right censored observation.In many biomedical and reliability studies, interest focuses on relating the time to event to a set of covariates. cyclooxygenase proportional hazard model (Cox, 1972) has been established as the major framework for analysis of such survival data over the past three decades. But, often in practices, one primary goal of survival analysis is to extract meaningful subgroups of patients determined by the prognostic factors such as patient characteristics that are related to the level of disease. Although proportional hazard model and its extensions are powerful in studying the familiarity between covariates and survival times, usually they are tangled in prognostic classification. One approach for classification is to compute a risk gradation based on the estimated coefficients from regression methods (Machin et al., 2006). This approach, however, may be probl ematic for several reasons. First, the definition of risk groups is arbitrary. Secondly, the risk score depends on the correct specification of the model. It is difficult to check whether the model is correct when many covariates are involved. Thirdly, when there are many interaction terms and the model becomes complicated, the result becomes difficult to interpret for the purpose of prognostic classification. Finally, a more serious problem is that an handicap prognostic group may be produced if no patient is included in a covariate profile. In contrast, DT methods do not suffer from these problems.Owing to the development of fast computers, computer-intensive methods such as DT methods have become popular. Since these investigate the significance of all potential risk factors automatically and provide interpretable models, they offer distinct advantages to analysts. Recently a large amount of DT methods have been developed for the analysis of survival data, where the basic concep ts for emergence and pruning trees remain unchanged, but the choice of the splitting criterion has been modified to incorporate the censored survival data. The application of DT methods for survival data are set forth by a number of authors (Gordon Olshen, 1985 Ciampi et al., 1986 Segal, 1988 Davis Anderson, 1989 Therneau et al., 1990 LeBlanc Crowley, 1992 LeBlanc Crowley, 1993 Ahn Loh, 1994 Bacchetti Segal, 1995 Huang et al., 1998 Kele Segal, 2002 Jin et al., 2004 Cappelli Zhang, 2007 Cho Hong, 2008), including the text by Zhang Singer (1999).4. Decision Tree for Multivariate outlaw Survival DataMultivariate survival data frequently arise when we faced the complexity of studies involving multiple treatment philias, family members and measurements repeatedly made on the same individual. For example, in multi- decoct clinical trials, the outcomes for groups of patients at several centres are examined. In some instances, patients in a centre might exhibit similar respon ses due to unison of surroundings and procedures within a centre. This would result in fit outcomes at the level of the treatment centre. For the situation of studies of family members or litters, correlation in outcome is likely for genetic reasons. In this case, the outcomes would be correlative at the family or litter level. Finally, when one person or animal is measured repeatedly over time, correlation will most definitely exist in those responses. Within the context of correlated data, the observations which are correlated for a group of individuals (within a treatment centre or a family) or for one individual (because of repeated sampling) are referred to as a cluster, so that from this point on, the responses within a cluster will be assumed to be correlated.Analysis of multivariate survival data is complex due to the presence of habituation among survival times and unknown marginal distributions. Multivariate survival times frequently arise when individuals under obser vation are naturally clustered or when each individual might experience multiple events. A successful treatment of correlated reverse times was made by Clayton and Cuzik (1985) who modelled the dependence structure with a frailty term. Another approach is based on a proportional hazard formulation of the marginal hazard function, which has been studied by Wei et al. (1989) and Liang et al. (1993). Noticeably, Prentice et al. (1981) and Andersen Gill (1982) also suggested two alternative approaches to analyze multiple event times.Extension of tree techniques to multivariate censored data is motivated by the classification issue associated with multivariate survival data. For example, clinical investigators design studies to form prognostic rules. Credit risk analysts collect account information to build up credit scoring criteria. Frequently, in such studies the outcomes of net interest are correlated times to event, such as relapses, late payments, or bankruptcies. Since DT metho ds recursively partition the predictor space, they are an alternative to conventional regression tools.This contribution is concerned with the generalization of DT models to multivariate survival data. In attempt to facilitate an extension of DT methods to multivariate survival data, more difficulties need to be circumvented.4.1 Decision tree for multivariate survival data based on marginal modelDT methods for multivariate survival data are not many. Almost all the multivariate DT methods have been based on between-node heterogeneity, with the exception of Molinaro et al. (2004) who proposed a general within-node homogeneity approach for both univariate and multivariate data. The multivariate methods proposed by Su rooter (2001, 2004) and Gao et al. (2004, 2006) concentrated on between-node heterogeneity and used the results of regression models. Specifically, for recurrent event data and clustered event data, Su Fan (2004) used likelihood-ratio tests while Gao et al. (2004) use d fat Wald tests from a da Gamma frailty model to maximize the between-node heterogeneity. Su Fan (2001) and Fan et al. (2006) used a robust log-rank statistic while Gao et al. (2006) used a robust Wald test from the marginal ill-time model of Wei et al. (1989).The generalization of DT for multivariate survival data is developed by using worth of split approach. DT by goodness of split is big(a) by increase a measure of between-node difference. Therefore, only internal nodes have associated two- assay statistics. The tree structure is different from CART because, for trees grown by minimizing within-node error, each node, either terminal or internal, has an associated impurity measure. This is why the CART pruning procedure is not directly applicable to such types of trees. However, the split-complexity pruning algorithm of LeBlanc Crowley (1993) has resulted in trees by goodness of split that has become well-developed tools.This modified tree technique not only provides a co nvenient way of discussion survival data, but also enlarges the applied orbit of DT methods in a more general sense. specially for those situations where defining prediction error terms is relatively difficult, growing trees by a two-sample statistic, together with the split-complexity pruning, offers a feasible way of performing tree analysis.The DT procedure consists of three parts a method to partition the data recursively into a large tree, a method to prune the large tree into a subtree sequence, and a method to determine the optimal tree size.In the multivariate survival trees, the between-node difference is measured by a robust Wald statistic, which is derived from a marginal approach to multivariate survival data that was developed by Wei et al. (1989). We used split-complexity pruning borrowed from LeBlanc Crowley (1993) and use test sample for determining the right tree size.4.1.1 The splitting statisticWe consider n independent slips but each subject to have K potent ial types or number of failures. If there are an unequal number of failures within the subjects, then K is the maximum. We let Tik = min(Yik,Cik ) where Yik = time of the failure in the ith subject for the kth type of failure and Cik = potential censoring time of the ith subject for the kth type of failure with i = 1,,n and k = 1,,K. Then dik = I (Yik Cik) is the indicant for failure and the vector of covariates is denoted Zik = (Z1ik,, Zpik)T.To partition the data, we consider the hazard model for the ith unit for the kth type of failure, using the distinct baseline hazard as described by Wei et al. (1989), namely where the indicator function I(Zik Parameter b is estimated by maximizing the partial likelihood. If the observations within the same unit are independent, the partial likelihood functions for b for the distinguishable baseline model (10) would be,(11)Since the observations within the same unit are not independent for multivariate failure time, we refer to the above fun ctions as the pseudo-partial likelihood.The estimator can be obtained by maximizing the likelihood by solving . Wei et al. (1989) showed that is normally distributed with mean 0. However the usual estimate, a-1(b), for the variance of , where(12)is not valid. We refer to a-1(b) as the nave estimator. Wei et al. (1989) showed that the correct estimated (robust) variance estimator of is(13)where b(b) is exercising weight and d(b) is often referred to as the robust or organize variance estimator. Hence, the robust Wald statistic corresponding to the null meditation H0 b = 0 is(14)4.1.2 Tree growingTo grow a tree, the robust Wald statistic is evaluated for every possible binary split of the predictor space Z. The split, s, could be of several forms splits on a single covariate, splits on linear combinations of predictors, and boolean combination of splits. The simplest form of split relates to only one covariate, where the split depends on the type of covariate whether it is order or nominal covariate.The best split is defined to be the one corresponding to the maximum robust Wald statistic. Subsequently the data are divided into two groups according to the best split.Apply this splitting intention recursively to the learning sample until the predictor space is partitioned into many regions. There will be no further partition to a node when any of the following occursThe node contains less than, say 10 or 20, subjects, if the overall sample size is large plentiful to permit this. We suggest using a larger minimum node size than used in CART where the default value is 5All the observed times in the subset are censored, which results in unavailability of the robust Wald statistic for any splitAll the subjects have identical covariate vectors. Or the node has only complete observations with identical survival times. In these situations, the node is considered as pure.The whole procedure results in a large tree, which could be used for the purpose of data str ucture exploration.4.1.3 Tree pruningLet T denote either a particular tree or the set of all its nodes. Let S and denote the set of internal nodes and terminal nodes of T, respectively. Therefore, . Also let denote the number of nodes. Let G(h) represent the maximum robust Wald statistic on a particular (internal) node h. In order to measure the cognitive process of a tree, a split-complexity measure Ga(T) is introduced as in LeBlanc and Crowley (1993). That is,(15)where the number of internal nodes, S, measures complexity G(T) measures goodness of split in T and the complexity parameter a acts as a penalty for each additional split.Start with the large tree T0 obtained from the splitting procedure. For any internal node h of T0, i.e. h S0, a function g(h) is defined as(16)where Th denotes the branch with h as its root and Sh is the set of all internal nodes of Th. Then the weakest link in T0 is the node such thatDecision Tree for Prognostic ClassificationDecision Tree for Progno stic ClassificationDecision Tree for Prognostic Classification of Multivariate Survival Data and Competing Risks1. IntroductionDecision tree (DT) is one way to represent rules underlying data. It is the most popular tool for exploring complex data structures. Besides that it has become one of the most flexible, intuitive and powerful data analytic tools for determining distinct prognostic subgroups with similar outcome within each subgroup but different outcomes between the subgroups (i.e., prognostic grouping of patients). It is hierarchical, sequential classification structures that recursively partition the set of observations. Prognostic groups are important in assessing disease heterogeneity and for design and stratification of future clinical trials. Because patterns of medical treatment are changing so rapidly, it is important that the results of the present analysis be applicable to contemporary patients.Due to their mathematical simplicity, linear regression for continuous data, logistic regression for binary data, proportional hazard regression for censored survival data, marginal and frailty regression for multivariate survival data, and proportional subdistribution hazard regression for competing risks data are among the most commonly used statistical methods. These parametric and semiparametric regression methods, however, may not lead to faithful data descriptions when the underlying assumptions are not satisfied. Sometimes, model interpretation can be problematic in the presence of high-order interactions among predictors.DT has evolved to relax or remove the restrictive assumptions. In many cases, DT is used to explore data structures and to derive parsimonious models. DT is selected to analyze the data rather than the traditional regression analysis for several reasons. Discovery of interactions is difficult using traditional regression, because the interactions must be specified a priori. In contrast, DT automatically detects important intera ctions. Furthermore, unlike traditional regression analysis, DT is useful in uncovering variables that may be largely operative within a specific patient subgroup but may have minimal effect or none in other patient subgroups. Also, DT provides a superior means for prognostic classification. Rather than fitting a model to the data, DT sequentially divides the patient group into two subgroups based on prognostic factor values (e.g., tumor size The landmark work of DT in statistical community is the Classification and Regression Trees (CART) methodology of Breiman et al. (1984). A different approach was C4.5 proposed by Quinlan (1992). Original DT method was used in classification and regression for categorical and continuous response variable, respectively. In a clinical setting, however, the outcome of primary interest is often duration of survival, time to event, or some other incomplete (that is, censored) outcome. Therefore, several authors have developed extensions of original D T in the setting of censored survival data (Banerjee Noone, 2008).In science and technology, interest often lies in studying processes which generate events repeatedly over time. Such processes are referred to as recurrent event processes and the data they provide are called recurrent event data which includes in multivariate survival data. Such data arise frequently in medical studies, where information is often available on many individuals, each of whom may experience transient clinical events repeatedly over a period of observation. Examples include the occurrence of asthma attacks in respirology trials, epileptic seizures in neurology studies, and fractures in osteoporosis studies. In business, examples include the filing of warranty claims on automobiles, or insurance claims for policy holders. Since multivariate survival times frequently arise when individuals under observation are naturally clustered or when each individual might experience multiple events, then further ext ensions of DT are developed for such kind of data.In some studies, patients may be simultaneously exposed to several events, each competing for their mortality or morbidity. For example, suppose that a group of patients diagnosed with heart disease is followed in order to observe a myocardial infarction (MI). If by the end of the study each patient was either observed to have MI or was alive and well, then the usual survival techniques can be applied. In real life, however, some patients may die from other causes before experiencing an MI. This is a competing risks situation because death from other causes prohibits the occurrence of MI. MI is considered the event of interest, while death from other causes is considered a competing risk. The group of patients dead of other causes cannot be considered censored, since their observations are not incomplete.The extension of DT can also be employed for competing risks survival time data. These extensions can make one apply the technique to clinical trial data to aid in the development of prognostic classifications for chronic diseases.This chapter will cover DT for multivariate and competing risks survival time data as well as their application in the development of medical prognosis. Two kinds of multivariate survival time regression model, i.e. marginal and frailty regression model, have their own DT extensions. Whereas, the extension of DT for competing risks has two types of tree. First, the single event DT is developed based on splitting function using one event only. Second, the composite events tree which use all the events jointly.2. Decision TreeA DT is a tree-like structure used for classification, decision theory, clustering, and prediction functions. It depicts rules for dividing data into groups based on the regularities in the data. A DT can be used for categorical and continuous response variables. When the response variables are continuous, the DT is often referred to as a regression tree. If the re sponse variables are categorical, it is called a classification tree. However, the same concepts apply to both types of trees. DTs are widely used in computer science for data structures, in medical sciences for diagnosis, in botany for classification, in psychology for decision theory, and in economic analysis for evaluating investment alternatives.DTs learn from data and generate models containing explicit rule-like relationships among the variables. DT algorithms begin with the entire set of data, split the data into two or more subsets by testing the value of a predictor variable, and then repeatedly split each subset into finer subsets until the split size reaches an appropriate level. The entire modeling process can be illustrated in a tree-like structure.A DT model consists of two parts creating the tree and applying the tree to the data. To achieve this, DTs use several different algorithms. The most popular algorithm in the statistical community is Classification and Regres sion Trees (CART) (Breiman et al., 1984). This algorithm helps DTs gain credibility and acceptance in the statistics community. It creates binary splits on nominal or interval predictor variables for a nominal, ordinal, or interval response. The most widely-used algorithms by computer scientists are ID3, C4.5, and C5.0 (Quinlan, 1993). The first version of C4.5 and C5.0 were limited to categorical predictors however, the most recent versions are similar to CART. Other algorithms include Chi-Square Automatic Interaction Detection (CHAID) for categorical response (Kass, 1980), CLS, AID, TREEDISC, Angoss KnowledgeSEEKER, CRUISE, GUIDE and QUEST (Loh, 2008). These algorithms use different approaches for splitting variables. CART, CRUISE, GUIDE and QUEST use the statistical approach, while CLS, ID3, and C4.5 use an approach in which the number of branches off an internal node is equal to the number of possible categories. Another common approach, used by AID, CHAID, and TREEDISC, is the one in which the number of nodes on an internal node varies from two to the maximum number of possible categories. Angoss KnowledgeSEEKER uses a combination of these approaches. Each algorithm employs different mathematical processes to determine how to group and rank variables.Let us illustrate the DT method in a simplified example of credit evaluation. Suppose a credit card issuer wants to develop a model that can be used for evaluating potential candidates based on its historical customer data. The companys main concern is the default of payment by a cardholder. Therefore, the model should be able to help the company classify a candidate as a possible defaulter or not. The database may contain millions of records and hundreds of fields. A fragment of such a database is shown in Table 1. The input variables include income, age, education, occupation, and many others, determined by some quantitative or qualitative methods. The model building process is illustrated in the tree struc ture in 1.The DT algorithm first selects a variable, income, to split the dataset into two subsets. This variable, and also the splitting value of $31,000, is selected by a splitting criterion of the algorithm. There exist many splitting criteria (Mingers, 1989). The basic principle of these criteria is that they all attempt to divide the data into clusters such that variations within each cluster are minimized and variations between the clusters are maximized.The follow-NameAgeIncomeEducationOccupationDefaultAndrew4245600CollegeManagerNoAllison2629000High SchoolSelf OwnedYesSabrina5836800High SchoolClerkNoAndy3537300CollegeEngineerNoTable 1. Partial records and fields of a database table for credit evaluationup splits are similar to the first one. The process continues until an appropriate tree size is reached. 1 shows a segment of the DT. Based on this tree model, a candidate with income at least $31,000 and at least college degree is unlikely to default the payment but a self-emp loyed candidate whose income is less than $31,000 and age is less than 28 is more likely to default.We begin with a discussion of the general structure of a popular DT algorithm in statistical community, i.e. CART model. A CART model describes the conditional distribution of y given X, where y is the response variable and X is a set of predictor variables (X = (X1,X2,,Xp)). This model has two main components a tree T with b terminal nodes, and a parameter Q = (q1,q2,, qb) Rk which associates the parameter values qm, with the mth terminal node. Thus a tree model is fully specified by the pair (T, Q). If X lies in the region corresponding to the mth terminal node then yX has the distribution f(yqm), where we use f to represent a conditional distribution indexed by qm. The model is called a regression tree or a classification tree according to whether the response y is quantitative or qualitative, respectively.2.1 Splitting a treeThe DT T subdivides the predictor variable space as fol lows. Each internal node has an associated splitting rule which uses a predictor to assign observations to either its left or right child node. The internal nodes are thus partitioned into two subsequent nodes using the splitting rule. For quantitative predictors, the splitting rule is based on a split rule c, and assigns observations for which xi For a regression tree, conventional algorithm models the response in each region Rm as a constant qm. Thus the overall tree model can be expressed as (Hastie et al., 2001)(1)where Rm, m = 1, 2,,b consist of a partition of the predictors space, and therefore representing the space of b terminal nodes. If we adopt the method of minimizing the sum of squares as our criterion to characterize the best split, it is easy to see that the best , is just the average of yi in region Rm(2)where Nm is the number of observations falling in node m. The residual sum of squares is(3)which will serve as an impurity measure for regression trees.If the respon se is a factor taking outcomes 1,2, K, the impurity measure Qm(T), defined in (3) is not suitable. Instead, we represent a region Rm with Nm observations with(4)which is the proportion of class k(k 1, 2,,K) observations in node m. We classify the observations in node m to a class , the majority class in node m. Different measures Qm(T) of node impurity include the following (Hastie et al., 2001)Misclassification errorGini indexCross-entropy or deviance(5)For binary outcomes, if p is the proportion of the second class, these three measures are 1 max(p, 1 p), 2p(1 p) and -p log p (1 p) log(1 p), respectively.All three definitions of impurity are concave, having minimums at p = 0 and p = 1 and a maximum at p = 0.5. Entropy and the Gini index are the most common, and generally give very similar results except when there are two response categories.2.2 Pruning a treeTo be consistent with conventional notations, lets define the impurity of a node h as I(h) ((3) for a regression tr ee, and any one in (5) for a classification tree). We then choose the split with maximal impurity reduction(6)where hL and hR are the left and right children nodes of h and p(h) is proportion of sample fall in node h.How large should we grow the tree then? Clearly a very large tree might overfit the data, while a small tree may not be able to capture the important structure. Tree size is a tuning parameter governing the models complexity, and the optimal tree size should be adaptively chosen from the data. One approach would be to continue the splitting procedures until the decrease on impurity due to the split exceeds some threshold. This strategy is too short-sighted, however, since a seeming worthless split might lead to a very good split below it.The preferred strategy is to grow a large tree T0, stopping the splitting process when some minimum number of observations in a terminal node (say 10) is reached. Then this large tree is pruned using pruning algorithm, such as cost-comp lexity or split complexity pruning algorithm.To prune large tree T0 by using cost-complexity algorithm, we define a subtree T T0 to be any tree that can be obtained by pruning T0, and define to be the set of terminal nodes of T. That is, collapsing any number of its terminal nodes. As before, we index terminal nodes by m, with node m representing region Rm. Let denotes the number of terminal nodes in T (= b). We use instead of b following the conventional notation and define the risk of trees and define cost of tree asRegression tree ,Classification tree ,(7)where r(h) measures the impurity of node h in a classification tree (can be any one in (5)).We define the cost complexity criterion (Breiman et al., 1984)(8)where a( 0) is the complexity parameter. The idea is, for each a, find the subtree Ta T0 to minimize Ra(T). The tuning parameter a 0 governs the tradeoff between tree size and its goodness of fit to the data (Hastie et al., 2001). Large values of a result in smaller tree Ta and conversely for smaller values of a. As the notation suggests, with a = 0 the solution is the full tree T0.To find Ta we use weakest link pruning we successively collapse the internal node that produces the smallest per-node increase in R(T), and continue until we produce the single-node (root) tree. This gives a (finite) sequence of subtrees, and one can show this sequence must contains Ta. See Brieman et al. (1984) and Ripley (1996) for details. Estimation of a () is achieved by five- or ten-fold cross-validation. Our final tree is then denoted as .It follows that, in CART and related algorithms, classification and regression trees are produced from data in two stages. In the first stage, a large initial tree is produced by splitting one node at a time in an iterative, greedy fashion. In the second stage, a small subtree of the initial tree is selected, using the same data set. Whereas the splitting procedure proceeds in a top-down fashion, the second stage, known as pruning, proceeds from the bottom-up by successively removing nodes from the initial tree.Theorem 1 (Brieman et al., 1984, Section 3.3) For any value of the complexity parameter a, there is a unique smallest subtree of T0 that minimizes the cost-complexity.Theorem 2 (Zhang Singer, 1999, Section 4.2) If a2 al, the optimal sub-tree corresponding to a2 is a subtree of the optimal subtree corresponding to al.More general, suppose we end up with m thresholds, 0 (9)where means that is a subtree of . These are called nested optimal subtrees.3. Decision Tree for Censored Survival DataSurvival analysis is the phrase used to describe the analysis of data that correspond to the time from a well-defined time origin until the occurrence of some particular events or end-points. It is important to state what the event is and when the period of observation starts and finish. In medical research, the time origin will often correspond to the recruitment of an individual into an experimental study, and the e nd-point is the death of the patient or the occurrence of some adverse events. Survival data are rarely normally distributed, but are skewed and comprise typically of many early events and relatively few late ones. It is these features of the data that necessitate the special method survival analysis.The specific difficulties relating to survival analysis arise largely from the fact that only some individuals have experienced the event and, subsequently, survival times will be unknown for a subset of the study group. This phenomenon is called censoring and it may arise in the following ways (a) a patient has not (yet) experienced the relevant outcome, such as relapse or death, by the time the study has to end (b) a patient is lost to follow-up during the study period (c) a patient experiences a different event that makes further follow-up impossible. Generally, censoring times may vary from individual to individual. Such censored survival time underestimated the true (but unknown) t ime to event. Visualising the survival process of an individual as a time-line, the event (assuming it is to occur) is beyond the end of the follow-up period. This situation is often called right censoring. Most survival data include right censored observation.In many biomedical and reliability studies, interest focuses on relating the time to event to a set of covariates. Cox proportional hazard model (Cox, 1972) has been established as the major framework for analysis of such survival data over the past three decades. But, often in practices, one primary goal of survival analysis is to extract meaningful subgroups of patients determined by the prognostic factors such as patient characteristics that are related to the level of disease. Although proportional hazard model and its extensions are powerful in studying the association between covariates and survival times, usually they are problematic in prognostic classification. One approach for classification is to compute a risk scor e based on the estimated coefficients from regression methods (Machin et al., 2006). This approach, however, may be problematic for several reasons. First, the definition of risk groups is arbitrary. Secondly, the risk score depends on the correct specification of the model. It is difficult to check whether the model is correct when many covariates are involved. Thirdly, when there are many interaction terms and the model becomes complicated, the result becomes difficult to interpret for the purpose of prognostic classification. Finally, a more serious problem is that an invalid prognostic group may be produced if no patient is included in a covariate profile. In contrast, DT methods do not suffer from these problems.Owing to the development of fast computers, computer-intensive methods such as DT methods have become popular. Since these investigate the significance of all potential risk factors automatically and provide interpretable models, they offer distinct advantages to analys ts. Recently a large amount of DT methods have been developed for the analysis of survival data, where the basic concepts for growing and pruning trees remain unchanged, but the choice of the splitting criterion has been modified to incorporate the censored survival data. The application of DT methods for survival data are described by a number of authors (Gordon Olshen, 1985 Ciampi et al., 1986 Segal, 1988 Davis Anderson, 1989 Therneau et al., 1990 LeBlanc Crowley, 1992 LeBlanc Crowley, 1993 Ahn Loh, 1994 Bacchetti Segal, 1995 Huang et al., 1998 Kele Segal, 2002 Jin et al., 2004 Cappelli Zhang, 2007 Cho Hong, 2008), including the text by Zhang Singer (1999).4. Decision Tree for Multivariate Censored Survival DataMultivariate survival data frequently arise when we faced the complexity of studies involving multiple treatment centres, family members and measurements repeatedly made on the same individual. For example, in multi-centre clinical trials, the outcomes for groups of patients at several centres are examined. In some instances, patients in a centre might exhibit similar responses due to uniformity of surroundings and procedures within a centre. This would result in correlated outcomes at the level of the treatment centre. For the situation of studies of family members or litters, correlation in outcome is likely for genetic reasons. In this case, the outcomes would be correlated at the family or litter level. Finally, when one person or animal is measured repeatedly over time, correlation will most definitely exist in those responses. Within the context of correlated data, the observations which are correlated for a group of individuals (within a treatment centre or a family) or for one individual (because of repeated sampling) are referred to as a cluster, so that from this point on, the responses within a cluster will be assumed to be correlated.Analysis of multivariate survival data is complex due to the presence of dependence among surviva l times and unknown marginal distributions. Multivariate survival times frequently arise when individuals under observation are naturally clustered or when each individual might experience multiple events. A successful treatment of correlated failure times was made by Clayton and Cuzik (1985) who modelled the dependence structure with a frailty term. Another approach is based on a proportional hazard formulation of the marginal hazard function, which has been studied by Wei et al. (1989) and Liang et al. (1993). Noticeably, Prentice et al. (1981) and Andersen Gill (1982) also suggested two alternative approaches to analyze multiple event times.Extension of tree techniques to multivariate censored data is motivated by the classification issue associated with multivariate survival data. For example, clinical investigators design studies to form prognostic rules. Credit risk analysts collect account information to build up credit scoring criteria. Frequently, in such studies the outco mes of ultimate interest are correlated times to event, such as relapses, late payments, or bankruptcies. Since DT methods recursively partition the predictor space, they are an alternative to conventional regression tools.This section is concerned with the generalization of DT models to multivariate survival data. In attempt to facilitate an extension of DT methods to multivariate survival data, more difficulties need to be circumvented.4.1 Decision tree for multivariate survival data based on marginal modelDT methods for multivariate survival data are not many. Almost all the multivariate DT methods have been based on between-node heterogeneity, with the exception of Molinaro et al. (2004) who proposed a general within-node homogeneity approach for both univariate and multivariate data. The multivariate methods proposed by Su Fan (2001, 2004) and Gao et al. (2004, 2006) concentrated on between-node heterogeneity and used the results of regression models. Specifically, for recurre nt event data and clustered event data, Su Fan (2004) used likelihood-ratio tests while Gao et al. (2004) used robust Wald tests from a gamma frailty model to maximize the between-node heterogeneity. Su Fan (2001) and Fan et al. (2006) used a robust log-rank statistic while Gao et al. (2006) used a robust Wald test from the marginal failure-time model of Wei et al. (1989).The generalization of DT for multivariate survival data is developed by using goodness of split approach. DT by goodness of split is grown by maximizing a measure of between-node difference. Therefore, only internal nodes have associated two-sample statistics. The tree structure is different from CART because, for trees grown by minimizing within-node error, each node, either terminal or internal, has an associated impurity measure. This is why the CART pruning procedure is not directly applicable to such types of trees. However, the split-complexity pruning algorithm of LeBlanc Crowley (1993) has resulted in tr ees by goodness of split that has become well-developed tools.This modified tree technique not only provides a convenient way of handling survival data, but also enlarges the applied scope of DT methods in a more general sense. Especially for those situations where defining prediction error terms is relatively difficult, growing trees by a two-sample statistic, together with the split-complexity pruning, offers a feasible way of performing tree analysis.The DT procedure consists of three parts a method to partition the data recursively into a large tree, a method to prune the large tree into a subtree sequence, and a method to determine the optimal tree size.In the multivariate survival trees, the between-node difference is measured by a robust Wald statistic, which is derived from a marginal approach to multivariate survival data that was developed by Wei et al. (1989). We used split-complexity pruning borrowed from LeBlanc Crowley (1993) and use test sample for determining the ri ght tree size.4.1.1 The splitting statisticWe consider n independent subjects but each subject to have K potential types or number of failures. If there are an unequal number of failures within the subjects, then K is the maximum. We let Tik = min(Yik,Cik ) where Yik = time of the failure in the ith subject for the kth type of failure and Cik = potential censoring time of the ith subject for the kth type of failure with i = 1,,n and k = 1,,K. Then dik = I (Yik Cik) is the indicator for failure and the vector of covariates is denoted Zik = (Z1ik,, Zpik)T.To partition the data, we consider the hazard model for the ith unit for the kth type of failure, using the distinguishable baseline hazard as described by Wei et al. (1989), namely where the indicator function I(Zik Parameter b is estimated by maximizing the partial likelihood. If the observations within the same unit are independent, the partial likelihood functions for b for the distinguishable baseline model (10) would be,(11)Si nce the observations within the same unit are not independent for multivariate failure time, we refer to the above functions as the pseudo-partial likelihood.The estimator can be obtained by maximizing the likelihood by solving . Wei et al. (1989) showed that is normally distributed with mean 0. However the usual estimate, a-1(b), for the variance of , where(12)is not valid. We refer to a-1(b) as the nave estimator. Wei et al. (1989) showed that the correct estimated (robust) variance estimator of is(13)where b(b) is weight and d(b) is often referred to as the robust or sandwich variance estimator. Hence, the robust Wald statistic corresponding to the null hypothesis H0 b = 0 is(14)4.1.2 Tree growingTo grow a tree, the robust Wald statistic is evaluated for every possible binary split of the predictor space Z. The split, s, could be of several forms splits on a single covariate, splits on linear combinations of predictors, and boolean combination of splits. The simplest form of spl it relates to only one covariate, where the split depends on the type of covariate whether it is ordered or nominal covariate.The best split is defined to be the one corresponding to the maximum robust Wald statistic. Subsequently the data are divided into two groups according to the best split.Apply this splitting scheme recursively to the learning sample until the predictor space is partitioned into many regions. There will be no further partition to a node when any of the following occursThe node contains less than, say 10 or 20, subjects, if the overall sample size is large enough to permit this. We suggest using a larger minimum node size than used in CART where the default value is 5All the observed times in the subset are censored, which results in unavailability of the robust Wald statistic for any splitAll the subjects have identical covariate vectors. Or the node has only complete observations with identical survival times. In these situations, the node is considered as pu re.The whole procedure results in a large tree, which could be used for the purpose of data structure exploration.4.1.3 Tree pruningLet T denote either a particular tree or the set of all its nodes. Let S and denote the set of internal nodes and terminal nodes of T, respectively. Therefore, . Also let denote the number of nodes. Let G(h) represent the maximum robust Wald statistic on a particular (internal) node h. In order to measure the performance of a tree, a split-complexity measure Ga(T) is introduced as in LeBlanc and Crowley (1993). That is,(15)where the number of internal nodes, S, measures complexity G(T) measures goodness of split in T and the complexity parameter a acts as a penalty for each additional split.Start with the large tree T0 obtained from the splitting procedure. For any internal node h of T0, i.e. h S0, a function g(h) is defined as(16)where Th denotes the branch with h as its root and Sh is the set of all internal nodes of Th. Then the weakest link in T0 is the node such that