https://www.edx.org/course/the-analytics-edge

'data.frame':    566 obs. of  9 variables:
$Docket : Factor w/ 566 levels "00-1011","00-1045",..: 63 69 70 145 97 181 242 289 334 436 ...$ Term      : int  1994 1994 1994 1994 1995 1995 1996 1997 1997 1999 ...
$Circuit : Factor w/ 13 levels "10th","11th",..: 4 11 7 3 9 11 13 11 12 2 ...$ Issue     : Factor w/ 11 levels "Attorneys","CivilRights",..: 5 5 5 5 9 5 5 5 5 3 ...
$Petitioner: Factor w/ 12 levels "AMERICAN.INDIAN",..: 2 2 2 2 2 2 2 2 2 2 ...$ Respondent: Factor w/ 12 levels "AMERICAN.INDIAN",..: 2 2 2 2 2 2 2 2 2 2 ...
$LowerCourt: Factor w/ 2 levels "conser","liberal": 2 2 2 1 1 1 1 1 1 1 ...$ Unconst   : int  0 0 0 0 0 1 0 1 0 0 ...
\$ Reverse   : int  1 1 1 1 1 0 1 1 1 1 ...


#### CART model

CART实际上就是决策树，停止的条件可以由叶节点的数量控制，在R中对应的参数为minbucket。

prp函数可以查看树的结构。

#### Make predictions

   PredictCART
0  1
0 41 36
1 22 71


#### Random Forest

Random Forest是一系列决策树组合而成的模型，在R中的使用也很简单。

ntree是决策树的数量，nodesize就是决策树对应的minbucket参数。

#### Make predictions again

   PredictForest
0  1
0 40 37
1 18 75


#### cross-validation

randomForest很容易过拟合，为了防止这种情况，可以增加正则项，正则项有一个对应的系数，R中可以很方便的求出这个系数。

#### Perform the cross validation

CART

396 samples
6 predictor
2 classes: '0', '1'

No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 357, 356, 356, 356, 356, 356, ...
Resampling results across tuning parameters:

cp    Accuracy   Kappa
0.01  0.6209615  0.219308080
0.02  0.6308974  0.238358325
0.03  0.6258974  0.238018757
0.04  0.6386538  0.269706555
0.05  0.6436538  0.282648474
0.06  0.6436538  0.282648474
0.07  0.6436538  0.282648474
0.08  0.6436538  0.282648474
0.09  0.6436538  0.282648474
0.10  0.6436538  0.282648474
0.11  0.6436538  0.282648474
0.12  0.6436538  0.282648474
0.13  0.6436538  0.282648474
0.14  0.6436538  0.282648474
0.15  0.6436538  0.282648474
0.16  0.6436538  0.282648474
0.17  0.6436538  0.282648474
0.18  0.6436538  0.282648474
0.19  0.6436538  0.282648474
0.20  0.6257051  0.238726905
0.21  0.5707051  0.092505117
0.22  0.5604487  0.064158660
0.23  0.5428205  0.001593625
0.24  0.5428205  0.001593625
0.25  0.5453846  0.000000000
0.26  0.5453846  0.000000000
0.27  0.5453846  0.000000000
0.28  0.5453846  0.000000000
0.29  0.5453846  0.000000000
0.30  0.5453846  0.000000000
0.31  0.5453846  0.000000000
0.32  0.5453846  0.000000000
0.33  0.5453846  0.000000000
0.34  0.5453846  0.000000000
0.35  0.5453846  0.000000000
0.36  0.5453846  0.000000000
0.37  0.5453846  0.000000000
0.38  0.5453846  0.000000000
0.39  0.5453846  0.000000000
0.40  0.5453846  0.000000000
0.41  0.5453846  0.000000000
0.42  0.5453846  0.000000000
0.43  0.5453846  0.000000000
0.44  0.5453846  0.000000000
0.45  0.5453846  0.000000000
0.46  0.5453846  0.000000000
0.47  0.5453846  0.000000000
0.48  0.5453846  0.000000000
0.49  0.5453846  0.000000000
0.50  0.5453846  0.000000000

Accuracy was used to select the optimal model using  the largest value.
The final value used for the model was cp = 0.19.


#### Make predictions

   PredictCV
0  1
0 59 18
1 29 64


#### Penalty Matrix

Penalty Matrix主要是给不同的错徐赋予不同的权重，这样可以定义新的损失函数，来看具体做法：

   PredictCV
0  1
0  0 18
1 58  0