From e2dbac6ace174c3c222668691c47bdc88f5ef0aa Mon Sep 17 00:00:00 2001 From: KelvynBladen Date: Fri, 14 Jul 2023 12:30:52 -0600 Subject: [PATCH] vignette and project name --- README.Rmd | 47 +++++------ README.md | 49 ++++++------ man/figures/README-unnamed-chunk-3-1.png | Bin 6116 -> 3645 bytes man/figures/README-unnamed-chunk-3-2.png | Bin 5718 -> 3943 bytes man/figures/README-unnamed-chunk-4-1.png | Bin 5745 -> 3763 bytes man/figures/README-unnamed-chunk-5-1.png | Bin 8878 -> 8761 bytes man/figures/README-unnamed-chunk-6-1.png | Bin 7412 -> 5921 bytes misc/pdp_compare.R | 19 +++-- rfvip.Rproj => randomForestVIP.Rproj | 0 vignettes/Vignette.Rmd | 96 ++++++++++++----------- 10 files changed, 112 insertions(+), 99 deletions(-) rename rfvip.Rproj => randomForestVIP.Rproj (100%) diff --git a/README.Rmd b/README.Rmd index 7a97c02..1339389 100644 --- a/README.Rmd +++ b/README.Rmd @@ -68,33 +68,35 @@ You can view R package's source code on GitHub: + ``` r -rv <- robust_vifs(medv ~ ., data = MASS::Boston, model = lm) +rv <- robust_vifs(medv ~ ., data = Boston, model = lm) rv$plot_nonlin_vifs ``` - + These functions assess concerns with collinearity. Notice that the VIFs -from `robust_vifs`are all less than 10. The partial correlations with +from `robust_vifs` are all less than 10. The partial correlations with the response from `partial_cor` are a type of pseudo-importance assessing the importance each variable does not share with the others. Now we tune our model by assessing four different mtry values in the @@ -97,13 +99,13 @@ Now we tune our model by assessing four different mtry values in the ``` r set.seed(1) m <- mtry_compare(medv ~ ., - data = MASS::Boston, sqrt = TRUE, + data = Boston, sqrt = TRUE, mvec = c(1, 4, 9, 13), num_var = 7 ) m$gg_model_errors ``` - + ``` r m$model_errors @@ -115,26 +117,27 @@ m$model_errors ``` According to the accuracy plot and table above, our best choice is when -mtry is 4. However, the accuracy for the best model is notably very -similar to two of the other models. We now look at the variable -importance metrics across the different models. +mtry is 4. However, the accuracy for the best model is notably only +slightly better than the models with mtry set to 9 and 13. We now look +at the variable importance metrics across the different models. ``` r m$gg_var_imp_error ``` - + The top two variables are consistently identified as more important than the other variables and their order remains unchanged across mtry. However, the variables ‘nox’ and ‘dis’ switch order as mtry increases. -Common sense suggests that pollution (nox) is correlated with distance -to employment centers (dis). Our common sense leads us to assume that -most home buyers consider distance to work more than pollution when -selecting a house. Therefore, ‘dis’ is likely a more causal driver of -price than ‘nox’. Consequently, the model where mtry is 9 appears to be -superior to the model where mtry is 4, despite mtry of 4 yielding -slightly more accurate results. +Pollution (nox) has a strong negative correlation with distance to +employment centers (dis). This makes sense if the employment centers are +responsible for much of the pollution. If many home buyers consider +distance to work more important than pollution when selecting a house, +‘dis’ is more likely to be a causal driver of price than ‘nox’. By this +reasoning, the model where mtry is 9 appears to be superior to the model +where mtry is 4, despite mtry of 4 yielding slightly more accurate +results. We now take our selected model and build individual importance plots for it using `ggvip`. @@ -143,7 +146,7 @@ it using `ggvip`. g <- ggvip(m$rf9)$both_vips ``` - + The plot above resembles a standard variable importance plot, but possesses superior tick labels and editing capabilities for the analyst. diff --git a/man/figures/README-unnamed-chunk-3-1.png b/man/figures/README-unnamed-chunk-3-1.png index 240aebfa95c0c3075cb63727feea983fa4b64bc1..c423d1af0db11bb4e79b9e5fe956a0b3c413810d 100644 GIT binary patch delta 3309 zcmZ`+c{rQd9{!RLp`wx2QdAO}N-Q5>S}QpA)xpTGh?z{_^p+^ZK|@#!9JNkBQ+oU zSF1a{7&98at)KF~0KR-uxIt&M2{nAB(qT`NOfYm{htJWY6YB?pI_tf4iIdR=)iS6> zA~8GWjACeKc0A#-uWA_)Rbu{Bg`JFe14y??8 z#N)V+U=gQN`c&9Va5z0cPH(^gPp1|75&N1>z(#7ms*WOY`h>2(@-9!4A*nZ8=9}mp zaL~0_?5t`o6yaN^fd==LUBm6kb@l_w%;G-&r6JV^T>8er#-Tzf595jX2M1z1$Lsy9 zwhlH$^bf{YWGa^LaWs_T&`-S#qw_#})$603W)p6qt8HOB91F zxXRdRyO$|P78uYRZA%E+-s{)Z^vQ))a*{!AoUq?JE#KE~O{+0cck6r3%cm!#yo%!R z=M7!Ir?JFCV;Ia-H4UL0{@rRvh0lpiP#Q~w;)?JrI} zOpI~9Ce?O)=*B43``S0n-@+c|RYHnC%^iVdT}oTb|Ac*1upG=+E?IS<+IU0q014zCAjIY8_k`g52zdy2|A-tDh$UQa8 z(+k?_N0@{p1r5Z35g&!~x29j$Q5}gFo=aA50qCQl&3mBmkovG3w}2!^o3<^wI(f!3 z*vvCi2;0mKTa2lG7Yy&IIdrzy>+Y6)?RgVXbEjUHRdZ5|$K(jy3h-);elPEl8A@a& zML_0*5VEM)A|P@0L#PbxusO(JI}WV>H1|O%g~PAtra~AuqP_b@0n$|nP0*;4a6m~M zFqQ=pjsYl70NeFDl<;+(>oI0|*zLgR&=0l>hArNPo>1;5xHQtQq#G`u`+C)Pb{J!4 zB(cLpPvccGU_H!adh4u-UIIFu83!qAM*2gUEz<{Rx4Iz-2#_PijNa%^Df%Dj5SWqm znjbqgg_v$6`%J8Uia&vw>%NoWf5N%nkQs7lnnK!ENw(~>ls1pvj5VcNv){)Se zvbK-}n`BTo=O?K0%kqtc(tqZAv!1{xN15gplpFB!<58G8$s<&mk6U?XCRA(xNKS2Z zxxe;oI1{F*!H8SPB?i{*B1=Z6w>YIf!bN_mruV({bvgw%m_mwILZ!EToG+$p#$Q zE}J_!dLI8uwN^DslZ%pN1{Hjz{Fy}pj1Xds`u~Ywzbxzchj2+WiAUNgoRx}4s*}--P~qCt zm^(DNCvYDN3SWOps6b{AI2|}LK3A>wmVk~s44^w?^O3hA?|)tcN{i|Bf3WpKW9yG` zO+rBhq<~(*K%%BAidOZ|*(h?s2|=4@8;X;BGZ`+8sX{d%V|C>UYvxxymX3vK<)$qh zlb=4`D=PzzBM6QM7U8CqSv|A_O_f_Ig#WAegB$@#xH^ zmLBt_o3PIWPF)I{!fMVGf8D9>OIq5}cH_?b$0k#qH+N2#7_x&M8PPJEmei(KE33U$?6FojM}|t%xk2zhWypM z((fVz;nfrWnet>WxTC{9k&t-VC9jeylaCl|0v8 z`Z#5L4HLEc^11*!u6B`bo4!-bO~av?Aj9`p2wmSq4%QWg8w2S=6Q zZH>MPPwgwU3A%tLVD%OjIK+c`e4opLkw|Uml%h!;1y=KmqR(>%`cO&x7=(=rq{QI; zPG5$meeZD_Mk(;tfRmcj>94<$`$xt8>F+pj1hUW?3$U$84`i>MP1HZUP{!sqTV(q8 z`%NDu``g4DX5m|JgYfU zIwO@nJUEK-64mMsZw#z^me;NbQfPq`57AlOyF@LToqW&oMDM*LTjWQyvCGaf7q*V7 zS-uewO#2TO;Fn^nJ&BqcIG895=|5qzea5{dIN$|M2~NnTY|~y}9DCg6t3PG5O@@wi zOqA*GjJ- zf{(lqNO4$vQ-M+VdEpigOlQt^rSGojiioyBgWMS)_}^U7-p8| z!a88wBiOv1E|Wg1eYmQ75CKyF!N6sCHZ){_k6UJ)M|Bo=kv%W>HX@Skvm-B ze6idhz5_wc!4O;uAcr*153=&__j6Q8+dR-Pcl&{FB`!;=_I@qTdUPsHBX_4_YqQGh z`MJEjxomm8>-guBWAOonZFs8gw)4gsG9k+A74cZ@T};+iy47wlLpA<;*izqH-_wOc zyk2r;wv!!E(?m4boa$D`MB6nX(S<=jr0Dh^iK!~w+JtKxM9+?0d*#$_O6P;7&ubI^ z%#42ep(W|vsuYTUx?X8W4C70RpYkO4#^98=BqcNBvE|lNILra7R6`?9Rwr1 qM*1LM>Eb>tv*y3<4ZEdpV;(RcuGQK{ycFDE0JBrp##Kf?T=^HH2+@rI literal 6116 zcmd5=30PCtwmxB&u}(yWz+GYf%IgBt*f00y4!Aq?Rfr!6Jen zcxo!3DWZa5P9D-QD^V205CaAfL`WlH9?sjL?Y*z>>-%2s*Lz>@`Sw29d!M~l_W9RZ z|60k#eeNU!T~l2Mf(&-;{PeF7qy|F}s#`}LtdReT)dG`!`}Q8#0j3Z{fFKG4ArOH8 z5hxHL6{7I$?CiiQm=hq1YMDSGP(Vv1PSa9qLFDdG7EGX4cxgj7K_H^(zv~>pXI?^^%$(OSY$9^fvcgH)qtcDpi~NI zgjAJEf=X*DFO?@w74uTX2$x3YE~w+O$SQqY7Po=R?L~MLh?h#>rBZlcFHg)9gNE=B zq=7|lV9^?SxeZ7|1Jc`oAmUVlIF%yiftD%;T2*_&g@EngCPt86F0Hp0sO;@UkfEU= zpc-faVu+Zhdfv-L78C_h0TEp;Om3EdXFn}f%K2evFfW$G=_z@KQVJHGZ_`h|W3 zDICp7bW=mM>VK1KLKk!d2AGvye#w5ztj%_B*<_H*nn7)9iB#X0_w;}3c1Cch;~@QQ zvq_QTOf6n9HBv8ca=A7unG!IH+NK7yAJu2lS{gP-h4<@}w8ao)krPFzSYB4OG#?-T zwVo+9=vt*iI4X;}t3M;hKLyF}&FG`Jm!{nb&nX5T2{s z-MRDitaeYP>Fdo^0gCXgOXh9jiyqis&WSEfCq}Q1N%W8gh6+pK`(x6snAd+ZHRgd( z>^Y_{-Mb!t?Pse;HGJ3on)@D8e53VXTQYOiX32^mEZ>`&TgXt7T(WE1{IfQNYnX0a zo@_RHzt&ciTvVokb6R~v<3!I`XRB4fbX&?I7%klp)8`t^kqdHb?E>E0wFZl}R@)-g z0;Ck_EYnavTeqgj6_L#pEr$bTv9O}((Jk!6RFr?eK2F{4^lgU?C*GMx(9sA(HlN)X z9=QlNfv!iJ+Ic8ei;m9^6_xL)SZWsqH|8{zdOXX%s!wHTGX|F{dOKV#-6gBiiD}v2 zVP%tI$$Qh1kgtm~q^DLpyq}-1oSgN0u+BfP&n^fSqT_bMQyqyLt3TeW!0zfPh}t}+ zdF##A>>D!6y0(w1d%2E4B>hKwLn59MR(;{LFm`w&uf+>`421Vi^o8Kjn zvYV*gYE(3pF5gvhSEW+qo?oo22P03DffLILF@~jSH?*=npOfQXP~gGy_XAxFnN2Bo zh)Iq~il?#DT+6XX$?tWh9UHAL^pkZXIxUqo4_uDUkhHSyeO+%g{zb6ZLt*ZkEh*5& zxx2q-$C`*3Tg9aq>kjxa?SNKR%b-jawq7!FHGk4rec{x1)W~7|#J1YOVhduhKC#cO zZzVO>mraTbP9k|KJDX{f%Zcj@!TM^rgZ;~?|2Po;Fmk^w;AJSv&X~sajoOC7iLHKj z=ll^X&yvWjS%}(qNO#|{4TtwTTZU^_bf%Evh8mm4%7%x~8;Y-tC0`YOqnCY>xbzBV z&UbpHw0=dnM&TxN*9E)Y{pn-m&9ANekom+ynnv~cqdxH4Z}NM+?4%lIII=hg4;9)0 z&l^i%d_)qNG&?{Q&)qO^?eIWx#^7Ql?oxYQy{9zI%PXKgb|rJ;Lr-mL{JP5~Ov5Nc z8E5QAQdC5x9-j>ZU)YY!}! z#Vsie{^tkMKMtbFfRZQdjr!j2dzYVpTS3})*6x}aUL7+syI)y=?_#y{=V$v~^&GXk zR52)D!s+SD@$K3a5VwMu@#-Tc&BfW-M=wZOL%|#B4L=x-t2lIgo!_}|QU`Y9N#x@p z^t#qaVx@vAz@L8M(AAjmrM-WjuA5Z)Kxn`9e1(~s=r5Zjyi62%!Zuy|=?^0qttR_| z%5PB{*lsF#irIM-2$ps6P?61P{t)5o=jg(Td6Po@VP!D}tR)r>+1HWh@YGe7AZqlyHLk2>H zq#;Wikx91j2k>GAh7oq&QE8{DO-5$vR^J=)J)D_$>_riIy;Y5CA)Jd$W0EnGdIty3}%AbhTw<(FvaOAjwa zzucIo@3R&$9iicQU7s(Mq{44|(DjetA*oJT=kV zq`~GM_X_ypetJrejt%~u+A!(q;fgmKN7PF`gc*PZ6owX+ygbU>HCR7NafV|dQq~vy z#u(^|cB=!FjHvWww@T$4r;#^9s!R|Gf6n@sGJcC%EPrMhtal0+twn8`p$;eiG&B{= zO=5`XPC$y8n#wu1Y0TZ6nNX-A6F7NY%Svjs!s%kWihh1d(k&XXyr#8`Z&ijpL~M<+ z%#}Om@H1M+la>XCUNRY;fDQ+M%@mXY&d8$9DsAEeloPaFi%>mG8DVoYQ@BjN&qa?l z9*Q~aBfc<#(-1xShU2RiQ8F>P0$xtij6LS}vdfaBXIjjx)4u)WcF~vPmX=n8TIalw zQGc(-i0R4fIp6qoRas-n>25xP$z!2Mu4mo$*nVcQsb$1Sb=D*&?2sw;mLww@n!bzNj-*C2a_exr6K)H{@3l}4jcV~r2;qDE%xE*kDQ_3r1 zZLA}+-5ex(KqrGwHi0q7_NsCF4loAqmqknxH~P?(`A{Jqq>D&4Y*#n2>Tt*JP1FC9 z>hWh}#+X;7!TKV?Ld*`dfLO@G-4!3Ovj+!FXBw7rNYyoOu^oH{W9utq*RoSNYh0kL z)`yk+JYYkPuIQ;R-uc|D_%Us(kUicmtLcwr@Pg+2j|6((EMr_=OPczDVmx|b@3AAg0_h$Pa)y;C11HC}YK*tTwq6uJ_u zT2aH$@`FQUs_bLE5G@6IIG_I*{~jN$cti!W?6tn{7jq??u`^xr(jJA7rdy z9>i!4khu+6AC6?HMO@jqcxt%+px5fnzWnISBJIFA|688z*muq5E?<=^_FvBFYD?cM z>r1XV;qW)YBZ$!TX~faK1mT-DfFvvY}ox|(>_EM@G_2KC-G!}mx?lFz{adt?b z%BS0QYfw`F6Qw2?M=3hjI{WZnKK4*Tg#_U51d&<&pC6Zt29!<3|h*J&{pew>zB&)kykCu^BBX^eZgiDR=nnSbt4QcE;7wt%pUa36$XBH;fM- z+b|zU?)Gmx8s+n%d@*&`W(Of!5K!q;t;*##YLB;wQ#=$QrCj`x#HJK)jBD+Xm&L(= zvg8c2Aa8DPbml^QuiqW;F}c5&AoF?`a@3_dIjHuG%JT|zBc_iM z?0=3|HzyF$7qeF~52`qCnk3ru2Qy*jN>Om$S|m8mqFz}ta2krZRx8W2VD^K1uCbFL zb7Oe4d0T*X)232$=H?WO|JHMbs15vbS{WuqusvpKKSRl}&hoPEi(N+F^5Amhdi{YD z^RuKl37wr5u*Nx7FckaIQug^)w=ET{9jIZK>9_CYJ70csX3#M`V@n^&a@cQ{cA%cq zU3}SH6Qk!MOp0_r)|na9#LgW_+l<-LM5+nmL|flsT~IezcZQDaYGPX$Pn5CD`cxw9 z?*mXEHJ4c6yLC)0>MyWq6(3H%UW~)MjiiRNU~>7p-Sx^F-^FymgE)w)e%F;0=iNTj z{)Us3);M74gi~o>X?dvLKjBMD@??`=v^mMHefI{$^T*BxxAq#rD%)SZ`QK64c!Ga! zn4vLj&fFOE#5tyqp|dcA4u?a835S_pIU$EFT`lk$jvyGxiskdC^3&$;WR6PQgKp7H zUU$!%N!4jeC}Jd>7mGZe8reO|kex6WR3rT%k%7}XhpRSM>AO6`AEg)(-#Q0zJ@%&} z6C2(Aj0ITfUy!j~TSIa+YCOnB4>i?A_PW0SWR^xF+&!iq|PK~W^tp+3i)2gV< zxf=ZUJ+YR_mzGVOd~XEXYs29;-d~xQ;BO1kOh_WdT4&Vt+&8G$F>sRoU8^+XgSZYN z<&_&Ar^9jlU1|d_RO+V#^TPZXsYep#YwOxtq^|i@p9v-d*D!sZpq!cRQ@LOdIHd!a wfMRYg>6TK@V-JH^_;;*+?LDgXTtvGWb?@Yfmb0Czf9rScaR2n~Cr8i!7su@Id;kCd diff --git a/man/figures/README-unnamed-chunk-3-2.png b/man/figures/README-unnamed-chunk-3-2.png index dde23a14f32ca3e1ecece067a0bd299ce49691d5..529bdd749589e4259cae1d763ae10d312c2ccca7 100644 GIT binary patch delta 3600 zcmY+Hc{tSDAIHD5Fk^&CmM*3lYD$I#{WfEnaG5)bZk8rX zB74duOJ#|aMrEli*|Uu`nGAm1`rY5tALlvGInOzt=ly&>uk$|Vbf}GPQ@n0%X)1=1 z!vFvvW^P8c1ppXfqoEN10Bk?yT3VtQ1_Q;)%&C;)L8((|%iyQSo1gC1@!Ox>({Zdc z@x#>PM$69pfJuXWxuM#n-`F2{HW!>;2TV-vziIzLa%~MCRmT8!vrS{9glq%vSXqpq zq59)FykyVnb#pCN$VPlF6z4sruweb zC5)8uX9H@}RK7a-E~`v!@8Pu+KkV(;g?-~6!AX!yvIQ!{QCFVegQ%Te3z3yUWVKm@ z)sEiO58~$*?`{VwTE7)~;&m~oP|x(iGH73O7BMbhQUOgqnmr&bj>fd<1Sx*cAJz)* zx?sF2?>k-h4kOA9`0TTiR3`~S$1MU@20POl6J+e7Z;jZWayJU?>92Gu8q;&h*d$@1 z6@W(EUWthezoG5d0GBI&8h3ZdEK!(TTTwX-)w_Ki!cHY_Lk>fNTMyHj!NsgfV8|^k zV%qn#9io4`R7e>p3mP1=2S`kuS&-BnA!WisxJ*ElTTuVcKGUh zFElLMC2SUDmV0dM{ZKtqKw1}&RhajFseLU>Fd-A|`Cdz4xaagk;7hN>YPzAH5Mm%o z-)BK6(n$hQKgm%Cf!tep_s+*0d?Y(g2=-S?Atv2c_O{gQY89#T4((SO5mTHA6!Z!! z5#j8Dn>MIZt$Rtm?L_9V$vFs zgwzl$i#nd$sWeX5f_?X76?f>hptjG-GtZkJoS>XuT1+hgGNohet>Ks^adFu3EZ&$gN!>4uJ zuR~gi>@PHwQ2m^Z;x6>q4U56wa2fB$;cqj)6%WC$bbZazK?~040C^d!gsC-=quW5% zTzPNWN&2DtFHO8nWa?p;d|5N1_OjfVal&Df5JsL4<$+hj-@D9$7*}vqhz=d(iH+}@ z&%L60*Z>^M-oEY+RuI+zX1stzw{7`%uwPipph{&kccB@7Q}x`pw>5l<24Z0!47Y98 zEyYye-Xq`y$y)7f?w4Snw%^*s`1I;ApCaSP0+7*s{?ql<6GaTV40WDC1Df@iBd7-W zKtWgP2gqiTH;~XVQ{>wA36aG2A zODK{yAep)358u~KPAk-)Nw1bZT4rG85WAbdNZ8_wmgaTeuceE}S+~>{IXkif5W(AP zna3pb^^=wu9j+)7C+x|2)O<>Jj2F37e4w?X=O}atqJz*-9B0CE)WFkX`vvC``j(7z ztzQB9sjVCG@VyNXfzlRwz13V2@YYcAf`D?_Hkt^9-C-H5a_Bu2euu={?IEE8ih~nt zY)fe)-op;(u@rG|4rOxP4fUsyvU7wL^T``;uZ9>)X2{zOO2MC){=#gL7GZ0-EZdqM z9!ze0PBFqYB{aSilB)HH6@y+yG6ICZhgM3iP)weLD-nx~m=a&!pr8pJ^)K$+^n=4P zSu)}wb8Lhj?@HOBbNm;%HaL$x3iUwREhBHWwFCRlH>`)C;;5?}PE#ZUbnO4Ws_WaJ zcBw)p)p{StN&R(AiM*h$f67q!T)5})))Z8eU-Cz%PI}l)PYTVA_qq2gQn|>Hr4B#t zV2tLN$Hv^kE5W(>ot~ttRF``$i}y7}=j>^kM&6-=k&nGsprHfpaI()kd37lb@Cd8- z_g6D$(%OOC`Rv1U6NL;gDz%m7Yi1?#RzFTeRHYezJVqUjoJEx!X_D9bBm4j>T{0_h ztLh`>2O8y4Mh|!wmaMp~s6pL%SIi{Gy=aWe)Wf;)ftMHClb4B05|P~q>{0dOhRu(?yCyO7GUKU4P05ksQEwofAGX657Z z;B;KOk-?f@j^CAcyM5*}nMKD{M89CQ$9hh`Fev(aOnt)tB9fT%@|G7#Aqy$a6dl{*K30#Fsr^yc9 zM{=HkiAMeb?tjJi$WEur;VISDPufO=rD{)_?+0*3VZn7q&M+O;D*DLu*~#Gr)Q2H7 zo85p%;tQUhB%GRHCUD|#C6Q_?HR~2bhKl0VfZX*dgLXf7UI=dCio4_b*1Q9vc|ocH zfmLV2fng9SeV?H%$9?DaR3~a)v)2agp9jf3Z{rtNa3Qp_ls1Phhx-Diihu8rA>%_kpv7tU^vs>McNz~g~ z{Vg0N;0!L&0mIdFM4RpzQPJUk{B19JUi0fG&5%!BHlZXS*KKqdsn{?7eJ*V8fcV4X zs|3*%!%OTQJkrEFOZ(~K3G?(jYh8iLK7R|X_2T?@E1^bqiTa>+wfOi;(tE^% z9S`-sR2KU>h%$>U$)Sz!>&FJK&4^tMF;u2SIgtPZJJCNU#L9wH@1=yV`=`O!Ki9`E zqSpuCPcG+6a%h0brmArLoII^x_uB97Bvn6jS>_{`d ztWJX0H>4_Lr5a~0Ie3%!&Y+R9`On(@;Lc5iKyupEb2dGzb`BleFM^d^SGUpCR3fI? zKW^`hrzuFF&LL=-SI79Qv(DEdlErxw@niC4F@!@(I^0ghgWd2ajI|UhRs1pn4*oe~ z)uLHIe(s0*Hh3-7EJcaF>3TQe3QpkAgsYOjuC9WrfL%klRRO{Gq|KJ(gQ9G7_GTZ@ z_-9qnslv>gLD6lkZefAT#v9@8Ji0j00&c`>869;=B399GlqjkXwn{t+>UpM4v(ar5 zLt8jz?R--P7n#0x3_oj|v=Hx){7tfg@>yKK?0KfSGTE!?wQLtY^vvKyz`!FK7(`PzQd$h8O*e~;1*b^#(#P= zeNEA@R&PV+(=foMxLAJ)%D8NO+{i5d(XASM{d4*18);b><34hfMW@7o)ea0M0JRp_ zX1~A~-}AqB{g0eFZQ(c-;R44NUmGe!#Xr1yQz?74BT_G|dpdd#z0qIpcYm1@2`+2& zMNiG`ueW#Gf=7_`Ljh~Y7FRQLQ4EL-)4e|^az*VEp@jCi~%Cw@ks$LLAHlSB~c;z3Ad^R0S z>E2^OCOIfpSgTgJ$o896sg>^Y+A6w??`C+Z_)ru;&dxVziQQxWtC36zky=ca6Ycpw!i!rp1H|(-Dcd&YHp&oVi4aH5T&GOgMGZDYv8!%Eg|rg9 pqEnpea9yzxVF_?w|MWd(NIcv)0U-wP)5^d)Assa&oW{ z7LXHwKp?_)wwAv^Ap3|A$WNd6_k%6)-?Vu_wBQ7}Lo}I+?IX1ttKb1xsL-gqocN z`OVG(z~bT}$QK+8Qtz>uB>^BmU~e=?2mrt>YOX6N+5jEyhU%7QA9#wYSR&;fQ*F8560EThe4|QLRIFy@x`?wivQ2B^%n)P_%t$~zJ5n>0A<`U->lxLqRtEG!z#pnN} z!i&P1v?tBj3Vx*J>0VLnr!+v5e&SNp!zlWq%h0xswsz@6NfUE676j1&^^3fr`4bll zv)rLfUI%Iu^Pp8@Yfot0OAJgnPBr;6=?*k+W<=Q^@PnQ3H{`5a0bgOyib6YN4J(Lu z!j*qrFoKQm!_?Zd@s;*juWpF}9u~{sI3U|Pj(sJWQlFgrShnFZERn?#q{*5IK zfb+a5x3DoJh5Q-0nX}B-S1Mshte|>LM%x@L><*Fxjcj_@~m)geu`PADg;%HE$ zT%BIj$r5)MusS_(Kn~;3U;UW7)`ccX=+kR3k|7Q3-xUROCSpx?FPVHXne-482dX4C z-HLQ!5|$mR!>bzQoAm&7Yi(GqVeRs_IcL*4)xgQhhNrLPt4t@5^+6p9n*NlM269Y# zrF$o%ZDT!cNsZHVDz;xFr+(R;jJCC22r9cl;XU=7UKiqZO%qb6ScUx=Sn|{CT;*g1 zu9Op(Z6^hCLIT2LAEuI3zFnezo6P*O%q{TVY0Jro^e>rLEDQP=r_6|*;yQ{B=8T9B zVjGj#F>^-b7npCkMC4&pJP*A7yxTKUwacRDdt!ILmWw^ zys?m7WF#49TV@T*DEj{76vL#cyLalLefYN=ebLqS@ zZd69(tR;7&-Ug~*oXRq=&rO=jXz6zxaa1j*PdAIYEcwl=Z<=zNO<87O9IiJ%ww|UT&=8XyN=79J@a+c_xIw_*CV;K16V~R#*=1L;YW*uy53)Chf3C`hRtj85V-x z{XsyA29H8+t(c^BgR)x6GnurFG0A&wuvjIf(m5RY-s<-wSS{U|B%l6j`<(7F|F6bqwzF7EuQeZSLTSRS!5PPYY1XgA7-lWnRtt2#Z2QTKgp^+09l?}s6N7$BYPTu47V z*T9ji-B?!*sGdj&5^ef;hWfMY_3`uz`8YRN)JEaseoZ%xn%fbWmBsxY?<||Vd{=4O zqrq`x`tg24@`LUNI#&{6=gp*m%+^PJKF~b1@@AP_ve#J&tU-^D@YV?b_@=%W_NZFQ zWtTsQ-E!cU{#(h`QrO0#Xb+LVv`3)4vu`T?;01%C>=6hA`i#=J9=_1KpVL1_L9M6> zGDQl8Y_6yE3<9bU=oNS8!#TP<>I}X09>B93rTks9woeg09{= zab|E@Mct$2QSI3Ab2;kf+=YyFYShIY7xFhDYpOalBGEoCIGM#vs4Dkz!F?#=Fir|#e0XUhpaGcbM5`Y7vC0Q!u_Kt1 zX{f9wbQvZKTfClfXsU1HmTa0Ij4fcjARXkJaxt<0`>oqzgIJ_g%82&+hdf%|C%jvX za$f(MvemvNlR|XydxcHkY9E(*4#TAp*EPqPKA8>|gPazm5o!W*ZaR%*C;T)b+GoW< zTquAS`-Xr7zxJbmav_yvz5ujEt&AcMlbL|f)Oxi|A~%0xP2-LQTndb{cq_9Dt&2cZ zRwJ|WXC?u4|_Ls5?wFA!tDpEv|Fu;b%uDH-W8Evs&Jg2@&i0=-g@Tv-Ds; z(J5eQM4{cqN3bYdG$<(oh|@3%PgJ>K^=G%xAk=!AZYQe%oNlv!y#Dq0$?Ig`pnhSlLJlS0dX zubx)btxqY|dA`aOLrQwZ=btuF$`^*aXKU4etb3KDs|b7Q9cOw$%lvWScO3GdTpo{R z=gd^5zwC91rlBnL-3A@3xx9B-wr9kLN~UC}dj~v)f6Mbyyt#A%Hs@pTm9c7l)KyC- znGzR{asbyEc7uNPg#Yl{dqkacWh>gaM|J_j+WHNVF(y?u@foS?S4l%xz{QrFY?H(s zI&WkfV=QHHKnBf=u-2xtYtxt)JJHn#2pwuTo({vaZ;X%Fn3WZvm&h>e_I;TLF=Xlo&E zY=2QR-?4|@eWK1NzVMd{f270sFIS}GSDb19uJAX#IKlC4&dnLaG7I6hW_h;mC>ELL zBh2ay(|ax0NKfWf_4pE|8E`0@zGdYuB{kiI9y%95Z(Hi*sxoKaCu4eKVA2mi-+_7^ zhFy-=(D^5i7tj9pBbxt79Apkn)kGywkBjY^=+7M#(Q zi`!_6kDQ&Ewp8{cN{0N=t;0DsJKKFVW95z7mG%q$=s#ubPQx|~t560@DJXkpu;W?N z7CNlj@Uw~P9Wy{mk1#54OP@}RYoopr;(5xEZm*Rhdr zZqAr0S>E9G@9_7?UAXV;PK|@s%>*F7?xYfRj>#S$C3-|Y$>v`$pRDa1C4#@Y63fnjNbqoe@rOR)CH4Y|uWOL!vaZnO}?1>FZYgna{lYB$^)S>yNpbZ0owsAySArLXIl_EatmhK~Ods zpY5p)N-%`~Nz+O&Cc6U$t)_ZAitS)LH7GO?|3f`Yv{)^zrl8Y zE?-Cxg@Ly3`cb1MsHz>K5&~7o4ZiYDr9AQK@hps(OwsBQ#+R&W+KwMi&a)x!M&tH$ z?L_EzTtqG18D9Tlm5TB!<2Yz$PqRb^W`g2rEnN$sJ4W=$iK;NF#A_F4T^#>&Z~T;Z>_;`pISbcVt2wAjVPPFlwEIVt_KNtoOrd|e>Tuw}PO=ia z3y!IduE__&cVE~Qja)a{=`0ssvZ>h*Z&8diToN3nE~dK;!DkXNFs9OQXFI&RZYRM# z!{T{RW(*B)TrnPX^_s!)^*iUyy+whl!n-!f6+8yWirZRib;!4;rEl=L3m@a*~HrT%=oFcc0K4zl467#Az z;YymCT;)d0`?NV37;9tWzQ>|>RH_-*!T!_tE?k}5_P>s-#Ko5&F8`qHnoDjVY`2H#?4Bf9!?`HsK!{pVx!Uzp!H zz@J;)v`o1btUZAJG}kU!yeQ2YCe^c)(k22z=6N+|47FXv!b?q$FtXWCx7P|WwZ=U2 zb};xdISM(}diMZ!Ak&7YnI5@6pBC8F+{w-*N`!b8Dl^V^LzzlEe<-L`JhIr@YGTB^ zlbtgc7Dm^z0RLb3--q;<JvbzNkgCz(Bpc?q(+kb%^P}`_rqRB z{K*`5*qZ0gtfv0Vn;$$Pjom-c7?wNuesojP0xfBw!T4Z3+MP6j zkY-VQMH|EWzHHBzsp;+D^Gm4- zT?wB3M!iRraMY_foR+`#S|n0T|BO>XL;gZD{5LiJh2xR3rzbof7he(^aa?*@n1~*a u!UvafBhm_|TA(neqYBXD!~as{0esKI1CRGDUcb6`KWt~^U|D&_2m2r4jzXva diff --git a/man/figures/README-unnamed-chunk-4-1.png b/man/figures/README-unnamed-chunk-4-1.png index 7f02e5fe7f420f869e3c0584c0869be652d4280b..f5238be3ee0263290ccd997a7017d411fb6bfffa 100644 GIT binary patch literal 3763 zcmcgvdpJ~U+h1#07zT};ZR}A@&Lky6j59?h>7dB$4uddMlAKx?L}GS6(RNmt-RR)Z zB$C;NXwy!V%CJM)*mB6uX(#Jlqk7+cy@%_)zVDB3UH4ko{oK!e|DNZ$pL_kDwNj}& zouwsJB>@1Cc6D*^1ON~b-Z&A=lBME!5o^d$-Mk#JCIFBDfDVw!02zC-06KTmrcIa$ zYm))G&@8;@WI84+GM!6iVIpuBUJQ5l40i^{J)@nDL9BWU{pi4|FN@9+(79YX7ki?Z zE5n_^aL-^cFySyb?F`OH#%g?qdqxJ<(T=jn0E>EiV9d{0hcb|vOf0+P=NqNM>vcT6rGru zz=AP9%t;tA!a=c6R2YQ0peP!5bfg2zheVi*_bC97;R|oDEtGu}OXCh#2V1Ym{Kz7}oz?umwlspoi{NpQUF zr!%Rc9Ix%Xl{3|QHWZ9{@<#pB2bu^rOZ)v7Ub}+BjmAcMaf2CA!DvCueV6x(0~a3I zs0b`Ow@*x>`2mX2Htu&j(%e(k$ZT`#-i4L5a})16CR|g_G&;GhILicH?VYWX(`yx7 zUZ$vJC8ksdE^Zi+b^cN9*pv4+Il%>60h9mkk148n>fF13*mGUx?2$oM z=##tZ&(5kd(cF=_(Gp&)X^yOgmt@xOBTqH(Eba5=pX0@;=GNUm ziaHb(wi;^eepIL$1Rrg&sz1Q1#OIL2pcaSDgOts1>XZB}z1LkM4^g(jWfx~(pVQhM z7}c3p@?fKfzfYk{QKf8`|2e}R+)=IKrUics0|U3_l;N^`=2GkI=uUrhgq)(UE zH!OtcJYI-C`pN<9E4!(J_-ddq40Z?KlBSRPDz5`Doa))b>x=(LS zm^H*GOwB8Am$)g?Pf)dI={Y?|08tQ(FoRjc1OOxqUO@m$RmkOihyqV}J;wl+S0bWs zboU(71fctC2{h*%JK#3}pWpzz){R0mvB>a1cuW}su_aOX0IY+^{82c*k=y>a_D!WT2&hN!LMH|Px67=^SufmmIfYAl{P!3)n+{D3GxN4w7! zM{ZVzb%cfXNrGU0GH)f|#5ek4pq^cm_{;>-HlE6I6*}My!H$hTiotALCRDrkMG}CU z*|8{3_}SJA~~!LzF7CU6?@bA6lAcH@Btn{JvV0x z2$U*bt-cyW&XytX)xE0$P! z?RlB7jTe(N4ISHBem4qd>NnKjqQV$ARHD`M!b?o^yM=It0AIBHLiW(5U!y=e-c8AX zuPH<@k~BF$s}E3_3c#6Mmr(ZwK@!$!#8>eZ1;sGUn6DrWI&Chws;^nR2=#~^+ghs0 zW0S(eH^cIWXnohD%)=dFd0$!|TWW(Dto)praWf4`z`NyT4<#f6SAXbwnk`EWjt}IU zs&ze=rDA#OF&ztZhN9B%M2o*53>H!)$269^k9&|e--6N%OhSCbEG98>d;S|*y0TKX z4-{3*-C~^h!)Gi;a&e(@g42!U*P3h|)T4Ne(33$LvVP;;vqGS`%_Kjbrs&d+q+hQ( zt3M!G9`bW>*@d;k?Z2QaSG}NcrChFF;{Wun-*0_Soej5p=ia!Is7Z0Xy~q2d)?JZo zgkkvmq~E-ipLnMp`;Kv*X)yjpfd3~`FUaZrp8XN3diy-S{noswpF9oL7x7D_2Ac2m zy&5=8SuaPNE;ZX`naU`PlBHgv#+J`9xAU&`j7=A$#k-ImTL%=(1J52mNY65VfA;h@7yT|+ZdD#|bTz*(kw4k+EGcN9SxZq!RRJM)9{bypm|Ub% zoC0W<$n#Q&-}ig7iYx%g;c}cDQ8g|qVr0kw2RXEydAb8y_7SI0`b89BgEqD*MkiiS z?~0xn6d+NCw7M!o{jz%C^7cr3aRe~6)20QkO$N>|>%}RExt%sq)v877-36+I@*QQ; zcYe%j@!*VIn?Dl|RwP?t3IIovh`fwsz!R%&0J4(9cnhmbtn>k}@IO#FF+M?q=5z$N zt`#eJB5%{bq0kc;`Qv8XFn;^L4^#PzdyVdtwg_0ue1LG|J^tMTjjC0{;CZ?SmeMT08JV!$ic7+a2W1 zJP$F8Gn^Vk8g%ZR?=^Hg46bu(1EY|HE=ATHfGgIj6{X39*$3cr3)P1V+QAY-;04IQ z#e(;g_lSJv_J2OkvGsS_Elvh>86Sw1e^4rgo7uA!hKcmbpP)XZ5H_cbUnvijvMNf3*y~2u$!|QR!MVfqS!ATj=YYd6i?+FdCp8B79^=l%+wsa)F@Ar_vb!Nt!D!jH@p?KGnNRP3 z49LumgRd7O?2hB9fQ3?yRW{+a-dh{->vhPX_2JT6Dw?5@x8CEszXJubf8B^xJs%aZ XFF)?%5^Jd={P}cs-05)Bj+XE@N#Px1 literal 5745 zcmeHLc~leGny&;R2%>Fk;2A*L1?fftihvrn-~zG)f&$7GwABzqz$F5*C4hp0U>;!f zfj~4OXoR!~K|!ExMOj)B7DWh)vPb{{LH4a)RV6)h&di&?<~V=6GdY#I-&gl{f8SR3 z)~&iqcC=AZ!YKg&pkjMq-$4LCa{vJKQBe-MLON)$3{qB-9bEQ9DgY1wfC>-@00BC) z0IFcejvde?NGAYP_%i&X5~$F@B2Wbc7IcV7@Gp==N+t#JNXgYyhyp>wI4Xc(SyYyo zDiBZw&>4iFfuz7dQgUD*bnpUs)q%X;WMq6YDLEPHs0LXCfJKE40*eJ1VX*{amKfv( zl6dfdWL`Co*W1hM1qD<HNn~S%h}7G52c++sz)A;^M5KW0lRko3@{bZCOpOx*+dzRukpwW%HkE zW-0M{)Y#;VjpY(GYkW!*iA1E09Uil=j)DW}cT^D|F_;R^w* z63fh}j?=+muiEdt>>OEDo^N?CmNL$3IUL`b6COhEtxUMNLM=S8%QHdPtS~n^J?`K3 zm13o{hNWFuiwzkI(HdF16Kv>r10r*8w$J*EjZGe%%HaxC`EvISObw=jUu(ubr)Hsa|c${ zjQnx7Y18LbZ!2uppY+L&9RBTQ=H8{n?F2(p@QY_3?Hnax9*V*cWQ6h zetU|&p0{>dxB^Pm?!Py@Y5kO%UHPpc z5$}*0W+On0iq$w>se!_eJZSZ`9#!aDWgGj;Yi!BPLMr$Alb3IN!qT*6cxpWcn&oZ> z3Xi|>c^caO<;t_Y$|B5g_o>}z553;(W=#u@5s>b~OtwB;D~P^tX`yjdL(@304UZz2 zc7MLZi*^h|+5O}P7%|ATm-Nc2`nTM_Cn7SmXWNpDp&x`)913#0PlrP$;3O8HD4zUB!zb5Ho zO{HymUgq;~BQUvnru^=hv(2Hr)E1R_`($;k?F>l(r%LS5<6~+&A&4aUp^d_SYRxut zUzHow?8`@WEmnkbEx>lu>+B_7y(cj@{yKy5T)ea--#TN??Vg>Fr;M2_4EIoN+RY|R zM2Yb=_2A0U_CeP@2O=2h4>Bzocj;*o=F$XN_;rJ}A*Z#jUz?$5KbBF=qzVs?a)nk8 zQmdJ|cmBn!e6V~Xz}AlO_*|QxA8*T)v$)IIy^M*w6y!oo@=(ktT4S2?E{l{-dPZh&`GYR3;XBV!a+vVy(>}+G^jk87+q*9&xjxv!=djNFqhIY|Q1n)9;@jzD=Jt zetcai&sq5nwmOfAEB?VF`r7|SHAIn8Ud`Pr<6n(TQ>2agh)pa8U54Dpu(7N@QPTn6PHfhsiw=xvYurI>cM~?(H$;% z%rM+Z`O-UN)TEK!a{7)t=s~}MaZ*eI880PEv&`FOas28s$` zLpTl5vd2E;D@`waDvyl*_;71pZo|g{PSWR7%i%!I0{Pkq99@)*z?DTf*kPIxk*5p; z#(;bzLM7T;cI~Gp=ygeoG*{R&qb(flWhIC$kLk7s&P5N}!xqj#(h3?D0hn;GmtcT| zS%4%9Q{-MZ_tic6+G_=RK4NTx=mKKw8nFdQt1}vpAbaFJbP=zViX0G1EDgz;E=RpM z?UGRGgpmh_hZqlyE?5g2-?XgM4neMyC?UpAaq1Aj(`bDx48Rm!X6GROYe)u* z>Rz06N{*5oX4U%+Q0uF2uxk#2|*ArY@~)@(v{YOlTra z>>@eMFD@y(c}?_1I)gaAm_H9IXnI*mb$DT(R2j3I%c|7W z8p_;jE5e#2mWhw_sCdygr>w502_}nCc`5g9u+PruQ&|l_0cZbx{@#pwSGz{Lx}PK5 ze&V7DH=*v0u!sW8!@q7RUn-jwJ8(Cn2mOrz&g?vAI(OALssG`Y)#St6`e^<8173Yl zr=JT^?cidio)6AqE6W+>eFcfKiXD7u94eBMB!%rJlM2};GO3ZRDwD?9Z!b%63G5`9 z^pH*ZOHxH$*sjaj>S?mJ))!-_%i(#NtdUCEc=hFd(2ax@6|zZEGMZoAvOTkppmnb3 zdCoJk)SafrBR^-uGrz!ZHTetF`1|RmLA7@wwg`-5tV5pZGA<>2l8oyHdy9q-(tT=qJj5U)@kg+-@GJ`F2 zImI5Q6s2K=;onK4hNp-O6lm`%GhLQ*S7y5FAPKffJ3}&aG@QmJ~dWuzt^ zY+0|H2V3Stu!pKF6ZUvzDOFL9DoW)yvn`yz^kTNBtf!oPQ5Nj4b+=`~KIywJlZyM| zWKzk=Ez$~!16GSzwd#PmZOUS*DL)oNK(a8nLeh{4jtu)~j$BfvBlk$Nnkcb+@eA1a zVyf;;w40LZ znxF@(U)bzD@FVF8ag&oesIiE8+jTr0Z9c>}bDLsmdcetY#cG=R{>kn^r8c_=R5_tt z^R)Tfpo#sI4$!15h7xtwtTZJ~zNq*xb@In?e^K8U#z=E2C3Au{6~F)7t^VwCTlmSo z_$842j_-)A+L#lxqmL+D@4)=*H@7IN4{?PbxP-D8eL3i)M2q0oGpQvrX6K#8j#Vtv zE<&%H#Lz!E#eCkz&o*B<`b$;=lpZzFg0Kw_sEkLFt)W%Khsa+Ugz|Q1dKTLXYN`N|$(=%sQCkDR zq*&Ahj#cfS2xJ$xc<%Qz_K$|Qy02Y}2B!C#cDZ)6yt2UkjxDO5xIp`ln^?B}h90KA zt2*zsFj~Y8XOovf?>|}-OA}TV@XC#BAU6P+_WyOzci6Lkzqf(^Yi6TeoP@>zIwoLy z1owrvRYraB`9`<`|9*4(`$dF9i~f3dgKux-L9te}%SH|eo2@zI{6h_f^)|K+z(t@9;N?K!=i_A!sT5YohgV6g;1E+w4#&yvE;v~-KGr|`BD0hYi zaXp;2I=&v7P+sZZu|_XY_(Yy%;OO89hc#bEi)tSz(hfkB4x&=+^MXg|Hr`34nxL(|CMmzX>gXI+Nm#4QUTe713i9)Z% zRKlGNw1w677HXr0tr1Ir=GtKFpflS>9Rl=UbTg70D!Yt%65 z*@@}W5lXCXg4KWjQm2VB7CIS-e1AA6-%NNpopsk~L$6}q1$jDGG0!S)H1eH&O2@IO zV~isUlFpkoQ=`F!K7>}QY_FrM$+0WKLnygAwoAw6Pex%hrwnh#4&NMj_sL?e7oEt& zo%kTmjtCtcP^3-uK$|6c=%}`|<6da%?pZQ`r^HiBLOMPTSrb(X(UPy7)1@z4lAv)I z5%1-7dOTkpWn8Tx>dxtxB1_5^QCUM6Le^}$%BHA|Lkx!JeOB;Wx`Qi|m>;PS}$|&?68i9a#kOE)ffe@Tm5HXb+dL0C-rq zEP{WCMT5k_BI4lg>&V?KSG_D)3PCT6oRu{`o;AK(WDkn$u|-7qg$E=S6%E4VMT2m; zTeMqLv^!X|yIcPZkO_L^-Q7VVc93X4SOmYsL0BqG1wXqmEnM!7XSt5=?#|B6E-x?t z@p9R+>4l~5k_j*yMGQ=ETVBGACXXx?y(Yns3&tcK|5HWpQ#>VOZtqay}lU7nR2J$ ztX@Os)UJw1X52aP%Go=!ndUqGKV=HLGxKK7Rp&l1HasRqE+2iXsN7tGP(Pg5MmP-cW91y3Zv3AST7K!B35M7g4 z%kLGMsaB6hRWtq}Z{V3a?=XFB|5)g$JLTv%B9XGzG^q|A_vOfLwj=I@)GcVQ3 zzI-8SctYSn`6QkE;FYL3?b5uTo?G53v8e?vzLA+hV$A0fRLx(X?+twJe>bhFSmsXw zGNZYqSzW$0;KWwWc)Iv@-imaXoaB#%7NH!1gWZ)Qgyg*D;cDWNXM)e7P@1=gEzV=hrZ+w#e}H_g`)mEZwODuZO(cSE5g?_UCCJ3)upN zW)vTxRt$^gd$bwMoWJo_8qZ0o8PUl{IV|Yn3oDv?Y~M=EPG;8r$?0pq;GMl7S$%IU z1*y|gGIH&$`(U`O*)OTu>YIPo>@uS_nd z9uvD9X#41$M@~;PqB*KEo|;{SrsIv>MBgQCO_%c1=l=e2+ZnVi?vn~Y?iv&ZQbSFL zr@fJc?jLJ*MO(Q8X9rV~Q9j}8QAe+|IA6Lgu54Em@MQk0jI9W;3n{~Q|ht9Q>kkjtCzV|IYmphT_VB#-JyulrK{L#yB$=&9I?3@jSd+h9PH79={k3pt**LNy%39-{B!6DSt;n@= zD(o9?hOZfm{1p%LHe=Pya45bQ6>Alnk?Xu>Z0S^faAi~-aAW7^Y;4tFNcoZXZ^&l#9Zzxq2{H(oMuMp5)|tbN10dwG}s z(~-N5=~CS+7qs8l(O<{&JV+Z*Ez-y_%ouK|nB_+UbK$`0q2PW{sJwBXCF)DvYc2)T zMm4XSkuLphGR`xF7grklacj2P3#e{H=H`$|o-B>dN&3$fkr(G?GVH8Q*!cP+Bv$h# zn)o>+Pbu1_TYNHUEAesdt+pno<-|E}{4CFsM$1c%DN<#4o(f6O^ftfNEgCvd*;>Ah zd8B|atg}tbX?%U`xt~o-g|O)uj*sMYMONw(Ub-bH7OSR*PZ9I#)$TkExxMOdb-r|h zr}0LeMfL6M^yA8oel7aRb$=R4a~M1A-4b+KG$_igy> z?w2+7z_cl9yPHn5n(K?g2ULd7DZ?Dk__1_?p)n_$=7@>F*F(^c(SSh1+KB*f+nMqu zzV#lA>&8$jHPp_sp4501&^Cw3M*QYt1gd7%TXqk}jZjoBCI5$g-3XtH?JB*a(?iE= zxOj%`a@ZYOdIB4WtYHJC+PcZX-wuGpaG{%^eukBU_ZZ-~e+;-ijKfk_9|dv;bo)Sf zw}9xWb3hb*9*iG3=1Cu*4QvJD@)4|A>bTqS0LPdB$Y;;ZaBLF*SwqMMg)x0V8Wly5 zxFirnzxXb(KYR$>!;}i2#hKz6mOVZcjS32!_5ZoEQsc~mV7)&ff$&}Ksau&YPeozS zEs3ZV3*m%X=y`gq@Z7;Jh&SPVc94e1=A~2whPlND89zPk=ftJOb`o6kq}A8Q_J!E( zq|&Gu1;&Oj;V<(EqecDi-Nq4qAaFRC+L-;n#|C1APAwYBW>&2+Kti&Yk(hPSw?>O+ zkQny;VS|IysPNbFsi0p5hUuf;RFJSB-C^(!OGT|s2R>b;!w;psx>&P7i@%_;ulfI7 z5XMGEQu3(xBT_y#lA@xz@9jB)ACsPuDA0Hu-0|-DGhv*tBVu2rQ!@}(FLvi&HNSm+7u2Fl5Z2s zM46Iv5g^m#3m-J$Gs3 zUTk)Z`YZ<#P~pH$yl|r5fsd5QKK-_#IMR)?a%-o$;*!Qov1SaW`$Gir~$ zEagP(OD`Oc@FeHD*vLqvhi@N@V9Q zJ)O3Osh5z2KjW!gxJ&kg_z$?Z1Q#;iq>R&iWd5|hsu^0DvWo1AYd>E_Lxqv-B)2(g zyb>?$1&*W8Kp!9QIR#G`F*W?kc@;peqxfwJVO;J(`;UkZdXf4AHZ_uo)R4mf#=paB z_B9zejcg*JBNUfM(v#M5jQ({z1=oC7xj~1xAig$Rkac8lrO(w+CCdi_;sZ(?ULrF5 zCb_`!Z8(mG5@5AeiRzym6lo+e>z@h+pi5^~E1MO*XmqJl^iL|7} z%H>B0Tfc*i)r_#Go>5_b_Yz!`p)Z&2Y`0MSCuH*Cybk;Y zyT`UZecQ^qH?}FH>-CG%`bZ-k@v}3FB7)RoDfTSulve?8yt5@&*;g9q9={a3qq2LZ zNikR7>wQeI^d)coPbp}scPQKSfmSACY=vFDjZ)Ycz;+R8a)10evjT*Ub1t>XIlp#IHVt&xwGcAgnl_7@YoI00U-28G z&Y)`KPrU)kPM3L#JCk^zV(IPxMePE44o;}&mOp`&WEwNp)+PibsgJj4F~#p#&Wri! z8X#?$}F|vq4hrpz3z_IYr`$Q1o`Xwij zyxD?<+_(a>=J>hsHMH$1b$6bzVa9$f*&{S0PU43SNj;^)4Bj~iJ44jwKx#lMTNe@Z(`;MGF%o0LqKteCTvUSN-rO^40^uUD?;$Yy6{&pAR;c*bm z^gK*GQn6fm1`=Z6^mw&ck$}SYfz8RCl_h5uON5|V3F-82>Buc+goFcKd*^yp_um4h z=eFOGE?;H)C$58f>h?#)f8ROQrnK1JDF11m^33hFPB!M45uP!1jvGvUUY8FadM5KS zlmR(Ztj1(4`+KS*E&oJ2<+#h_bYVZ)T}$1|!J>3GA*j)f4S(e9S2{!MZ@du>pR7uX ze}DSD-pju&th@H)$7xv_e9GULqmVHJXGrNP9p25&DHE8d@}{>8?v|c9r1$i0yU&L{ zzMG%Jv~8sHxH@X7jc-{ki=m#|DV@N5iTra3tkm2P|69J{v-_KS?*Ja{cEi~7o`flT zd&C&2@#rlmi&MG@WHuGHD==`zTk^vji3}RoD;iyZc0+{Qe-F<5$p~*n7J$v4)cxD~ z0u}4$T56m0Km_UgEVLndlLzkzM=xOOl@WJ|lC)RehLTYV3F)6l}}z_qe@%FcL_hQfoatUX(C6f9GR zU;7@MGWZMLmu!r{%gZICTRmus09HIL%#ymgHw+3B9Lf+}RV_zh0Th-Ox(_H@nHPR- zZEkUzAh&|m`Ob$R9)|pcKuoI-8s77+fcB{*AK2be`u5EbyAMZ@_&0Z-pn6~NEVR~rR-fJ(v^C>cuUhwE#eY74FHu>%%3vrq~ zonjmn>5DRP?rdt4@XZf0W_b=gc;S1gT!4o+)Z_&5kF%NNRW5cYo8fzichW(~gkR?) zy9ji0;XHs1xd?LfyF}+&y*Ls?L#fF#{)b_KHFTIOcciymHmVL$G}gx7v|gw9ssvF+ zM!UQen7#L$Rku&JBAye3B6TECk|UxX?p>O!nti=%jdbv-w1_NJZYnX8EO^nU+1vM7 zWiERY0pEryz(n2MQro*kgQtTGv+$u(M+KUiMQ9;{bT$fBrUFFhVqP-Dzj0X6yWHVN zxj?~91KM6e3A()=f_eGF)|Y8uJ)R=ip5(5-KbNt^oUPs0J<12)oZxLxbUnY)YL8$N zK@6B@g?s{H+G6SE&D z1B61Rb1xfILEj%@N$Zi6a^ZxZBliiIPzCytQrY=_sb)V8!XU$1J@h1;48ynYJ`TKE z)t<$Rs6z9;Z_W8Wb_{U)Rho>SJU=i{(EjLs2=TC`H^9>2vHlci4+6tNNXB451er2%ARN>T%_J{=}X>(xi! zsqnQ}S8=G)pR`Svvizcyez7p6sB^P`PnIn&Bsp%Mr_g z;@24kC`%QIgY)DonGQQOZ=V|8qHhmpWH@WVL=h;9FVnJ#;hCCi(=LM$u6B$v6iG@z zd=oy3s&}uxqNjJ0&VERdGKwPdhDNjmCp#;JqNg$+?6o0@ACMz4x8x~YC;3QOhxY72 z1Iuo1imkuF)Vb@llx+h|I{t|j&`;ckxAX7Vqxf7Qbs$Qt+wkF3l}AlEZJW}l9%4Ur zk)?g32*jJWrJ*5ptaRav6YXi*4<%3uQl^XulGNkTsVi8EG4xc2Nb0?7y2MpR1z>D+ zI7#&Z8PUAhVds;?2oGzxGs<{cg9Z0yrcK*_Cgy9%3rQMBQnT#sJ}UeStEUrfwrtp3 zxmnr(ExxseTU>n{Sc-WuVrC)>Mn$h@6gSJc9+u=d0j-TPKCVHLHlO=H7&)k?5UqGW zX%y=FXW@#uDj<1Ku zmYdj<#>(Qb`meUxcUn=THb(p#zg0)%Ogo%OXiI? zcm-#<9oH7}c@FxdW$0y)$9s|^=}7M!5_7=0YM($Z(&Vkr*}`&<|03zKLQpSJQx=!z z56|Xqz<9>?fO;BFh9}3Cz&S~QuT3|qAJzB0F++@!4}6lP3Ms?w2;zm_w|tHsb_`v)alznZ1k8tH4*o1@j}WCgu$h}if>ZKPm5}v*K@2Pdjw1OSEv9X;}s2KAPu_z;X|RTt%WCyA1)?Jh7Y>vKs%YR5FLU-zepo}|VJ)beE9T~XV1P$-o+ zhY!)>-z3|`bVwjx&Z0ikU<6QjH?yk>`gpRIjXV|ksi1r`Xg`E7UJYs$<0;>y#TU_$ zeCgf8(c(ve0xj{wNji)`RA$EEqo8mY=$aXHeWV2ll7$z?V=e>+ylRRQEW6?a|CeSG zDP*S^dpr6UT15n^e};k#-*A`^4h)adz~EBc8yS4YG}kAlfIT=a@?B=e;GG|bxA$S8 z(oHcQH;-3v<^)4}%J^4UF&@vtyy!jX#2&4asT=zujY$iTAt4)m1TkJK_Hi2bzbcy8 zWiw0762?=v$31rg5m!**mLoR)ZzOt@_TUzOS?$oC8iwgIo+~dh2nT8=ZtmlP$F-#+ zY59|=_W-bb@0B#2#th+VwSIbF@cOogM|#G?1m*m)aXx#81AADRLaBj9J551Qj-tP_ zGo0)LoY5NV5}~%WiJk(D&BR%nDD21_0UfVN?h0=leevxyFVCyaIjR{@0UHjj1i_HB zPTmuT_fan@caP_3_QLxv$tSqFcWF`YBW|O8A^Y|u0-;wXnpTQxZu=~^vF26P8P@*+ zr*Tm2H|L+3-2T(8vERH+z$y}W{{d@f*7RhbZM&N9=?a~{PJnR-vL&^)5$Dlp9O|7J zbC>oN94a=Xau^((`9b>+UG4Z>&~V@%qMG)^uH!7dAqBah;G;XY!T}}|zI%8u5oxqR zxHyct`mi*F9;Kjq-(;8s z-8<@?bnT+YqqHw&VlawAfod<&d@|6_sTY9Z+}G*q*6$nkKINgn%kwo0TnK0i)d6!w zBQVjmWLVPwYZUXp4@&<31jGK<26UyRFYs2Z;dWH_VBOJC2SDOi@mKL*ln3=Tr_bY4 zs@7dL;j<74L)&KXshIO%y3<>wP5JDTh}SahlW+M+jd^1D5S;+Uj35enosk&?aE>r3 zjJl@6(D>$3XR$2(^@5~hPziIr!)blI(|-%H{9l_iE*^Fx@~|wsT$tKLQ#2UQF`(ye;D%ntw{z%d*t^=s} zosuyc9gmlRt7eSgb!S(|0o2MUG&N81A?j2#+MatX^5Z#OfGbSsS}S9r@K(?;etB#DreR;nF`Hq zha|jzrO!SZd&^LKXW$&3(5>RF?~X5MQ`ByphQU80PS>%*nCAK;#_$&eyeZ!WfU&oS zo7zaBEmgDMp@&+nfDWS2_pfu|)ioqUPg-q~ZV&Ev9z?`hrrr|`j3rUO?=3>bryMJ% zh6<#a%$yhyGwcmlrI65TskL;e8ew{HFzyMbfHNgv|C^2fsw=zM2QJPIC)~OSs{nL0 LFKXnfT@C&pSyy1s literal 8878 zcmc(Fc|25a*!MXz#=caxBD7f|{Os9Jge;TET9y`DmXPcsQ=*tGSwlEwX_%xkMr5fU zvPE_wL}ZYVeQVw``u*Pb{d}J1`Rje2&pUI@?Ov|?x~_9w_kEq~{+ejqSpyDM0agG2 zI1JC6HUj_@3;^I)W;D`+HIrpT8g@7n3q7O>0BQi>0;s70YDk#?xF8n=pv(gAk`)vb zkTGOH4RE24(?3WhLCwWQ%>^kbY6%Hy2}n7FrGLDz*f^{g2^-gBPDVJ=nYsW7DKgo` zWw(a>aY;yUNf>gWP+TZTNkis%VdK2Kkc;F+YVsmY#O-p4!^XwY2S{;Ev;;LE0g>tg zB)A|Y!T=GAloaHfKp9FHq9hE_@?&TuFDwb+5{D%)X8=uc*d`inC_!x~!DWboTnR%6SA+`r&=6Xrq)m{#CTO(T z+1X#eeys<1Ze8g|xYL^AkZE)_2+0JAhH$0PIU(dU8tsPki3r5HEZ5Im@C5+QI{FPZ zdJ~-h;1FPV`jkb`G56%(aKEzXVeajEqP#9rK1s6B%hGVO#&`eVaM4yuQMkQqp~X4&Q=rtZERgvr z1M~#O{1tb<4vrD?}FbtGIu0TfcL?ZrDul_=t`4s)$VD(WKm4 zA?XKeWI5CHWH_XNHbq=teIw>6*v_gccc0+5xapn5?`<8=2iC{{ z(hu0;&E8E|SJXb-w#fReGRY;;{bPH`Wc;ZH;bpMTweTd7-zgfkAMEbevCSXhb}Qs* zsL|RRnOU%Hy>atw4U_%8FGhkL%f^W--y+@D`ov{VJk6TN+~2s_Zy?J|PP|mX;3xX~ zevh{nX1RBwy4aNOwdGr_5G8dxo-CtBcY_vRFpwuw$|=UYuLbNZgKBxbd%A_v;!{Ox zf~-Ls`4<#wOx9rfiKV)R=}Ud%l=LFVQ|#9_vWD|#cOPP^_y+2=3o(_jX_%~V*y4Bq zhInQ}`*pMJ4ds2hc^s}bCyax~$EQvVZck{eSr(GkUc^7uuCb#j54Mf)Ymd>IHkl5< zokNSd;Hdj6U&MMOPSX+78pjRZ7?Fte?~)hajt3sPC>pu(mueoL#(Z0|Mxh_6!6WSg z;J>`m+4pRbOZ;4RVWQ9XhZu_=$L91OR!!vC@QUmy*x2xXyLsV0d`~@xE{Ct5Wa*0-=Ot z@T|n>l6~(Rs$LpcEFRzcKwp%9ZdaXT5ZkK2VJj77Ji50V*?vYHXT5xTcFe zDE0QQGY;LlK(b_e@mJ9PxzsMv+>tY9HF*u5p{qj^EL3FlO`RdaL`AK;#V&!4VRcWcGI_s?qLNfqM%{wvmTu#uB|}k&;Aci#@$s`en8ny6I8eDL zrrs*QXYxbYU41*5M~vMXk;-t{xCCc{^oqVr&sas^3I?RgZ?;~x4dRCS#lshT`hwr` zBnjwUXpM7Ty=WGD%>H)Qq3Q0E2NEncVM~Z0XDKIn226={8I}^vdmg8JhaVfNs@;05 z=HLLofp|n-`6O(dW!?Lxy?t|yZNsx@LX@_AeFb#O$f*^B1se{NKSmkv8?xDz{aZ(@ z5M5MoA+I>4&{8s=UbgMjr)O*6Ci_)thF}q{*3K4{N2O zMjz>*jlWG*?N^`rkwbe=wplnnW8UANWSrzsBW~2#eYa4JW|c&4YbG^^_%Ii;-bmkm zGgCptDx@I$iOGV`Yjq&NT$Wze1_`;wZr^psrIi&XU1I5p3Ab-cXcbQt}at(4m( z60iQAVQbfhv5pGRY{TBP5Uv~B!wCe3rx%0x)9Jg@KU4l$;>M zJse7mT0D^tLaNAIA~jrVGkueguLwxoYqSGmbA%2~l$rp38p5R@`ABLP03Q?vdTi)| zxd8WKIvEpCW+eb5*P-z?QXD|b7zh=oFanbc0NiJU#@`46a2~td0b<$zx9K?=14Bcl z2lwu(D7j9?O6)4_S8__%5rya{=LqpW;5egv$^-QmeWa9ute+m7GJyFD=xy17&99!F zZOqF@k#8N*_4wL$T{`FGok9j^1MB+NV^f86(8MW=*F+R(aeHd+j8Wk#vY&oUAHQJ2 zZqxl0C}zcnoN0A+82Q9@5Vw3vGtXIWS_vh$ys&RiZqSt5T*k8C!OY-wc4jE+S(%f3 zf{AoIfI{J>-0o$3KAg15EPe>^&!*HyW95f5blB)rb5|V7x~?rUE7C{$>hH#X{QKw+ zJkS$tqv?~B&Vo-w!I_ehSpHr`KokQR54yI461?$r6(xJS9;0!R!cvcne3^O{i?7RQ z(EYN&LhPd+#G5Uie{Yc?0*fu_1IaOstOZ?<0i_%j6l|ek^;e1u=sMpV02nnmV$9tJ z@toD{3{YXv^UIQ0UYvT`ZD3bKelsYr`Y+p;tP+j(LjSNj4@&1Y1zIx!`Qxwd6e^lQ z=EoWXr5>}z2crZYnaPJup~m}(9=L!fjDHBuLTmelO}-raM_@9(vNCNyK9R1Xz+u>J zm<6)OFu*Q@rI(*e{X=sN|J~jcn2|gLDBo{6q$ZR^)caTpkzHVhI;cz~?iRvrn^$t8*@ z*pR6Gv#fut#wv`6-kM%audc^vNA)qr%R>*x^f-* zkqgwbKn8VYrl_qcQU^o!H9QFTb1%#krQqm%Fyx!fVu1gi1xa(lLpBTxu|3fNXyHS~ z8N`6$fiZ|8pndc!U=144UnXyaJA0U{ZR zt7uPv=mQ-+p+xrr4;D?*Vgw+}~Yw6(G(9w`R*T`1x-o>G24O%O=wFwgyJGNXu<2hCH65pR_cDncy8^ zWo=$4^F^3Ko9$p+KZ@!wC13NoCOj;8A#GcGN8M=DF6W1s_fNYQ$P4|b67veD>uB2^ zF_i0M!Qt=0#zTvP0B_mUTsQ6QgGpLvxV~ztI)rX@S(XIvx!Yv%F-xt*YUS8@` zM(XCV_sScmSM??{$50uH^X7~eZ*QtvLV7B`vm%L$xC1!VHgd|EsnWY8Mrz0KI11Af z+M}QZNw;#tb%6vnwhl1Y2QX^kg82dnY$%*y>1iN2nF}iZ!8gT?BbBBhEhcCVX@R)7 z3;w|1;WfO?pme(stYK%Rsl$0v7Lw|Z(_u`Wlypi0B#t-Y2ZyBNXp+Zr4E2zy=8)%4 zzNxR0$AJ8AN=d(J+XT?CRUpCFhj$;c2gDeOlf&atdA&a`qHvNUWT33t)-Ik5+C}*x z;eIPQYGrtQxjV}5KvyIapd=fcDww1SmdRu3@u3FXSzL^g{!fx75OK zO(uG82bBaG868MF@#@K`E{uimq#Q_9U)y(vNqKSkb=!L-iE>kxS7J(?d8Lgd3K!vcB8qWzWupL~=4Gb+dlJtFaI4y+oo%<{uo?jg4SIIHM;8 zOi{ntBF&SA&Fr8|`0U9MeCG8!c|S7!Gk?d@lKd0Q)lNd7Ge_kEKZ?3AGxcQ`QB>Jx zniFB7$haJ;D++yD<(9|5{ocdzvXyHOxR`Kt;&SvLUfc(xTFG2S#Y@b4aNn4A1AFo6 zsYxo;M0t@E*?z0>HsJpIpdj7az2`#DMa0yms;A`wiapPK(Kz{wA`n4~i1$5+U%kn` z9|`guGVm<7QjE5OP=kTdS#WMES&eSrMsjbur(%w;!>;FFRF!%v|H#@+L7Fx0V0!t7@JEo%l9Tsb%=aPCD_)wqEz=(GWK~ z4nvK6ek@z`&E3rurijZ2~`vBA72i^$X zs~5g(wROa)CKwlsF`x@_g<>20MWc6im40^|Ra9}28=*JZk9&1prkZqX4?Nf!jo7I1 zE~q(_lB~SfJ>lj~?@wf^V-s3bKfh{?K(Pfbb(qO;qawuwl-IVR@}4I#lxI39&YMgj8iwd!mOCc4|$sqUR?W>;&O zj-wFQPNN3p4MsVL@r^)CE*jk*>q+lsbe5Ne z2+obz5;Y*dfSOs#4562q89S=0HQ(ZkvRs#$3;{(b9tWh{EK>m`t@n$+^c>~S{pGk} z#D+7H4n9~f4md9^a8gI2B=Ig%4@&bNyKfvsUSzumf*>YxE@PY3{;TK3-k>^QtQecC zhlry6mB%vbe8ttma1BLg`Ju8z(lpBG!FZZ_+EW=#HGOXFGxXrf-El4QmCb1IR?lS~ z@3AOnsb}qv5z+xLqxdgA)0Y+Z>s!8kzd8VPKqlWY#8*adLnp#IZhIu2c&z3-2%7*z zF}V6g1G-~Dqkv69^v`*Qo3W>>a(!%<6E{%AMa=SFt&j2mO$1xV*IRvhxG>WebMr&82LDynYHj*iX* zJubI|B2D(wg^z%u9vu$3XDe77mYkosK*hx~JulAC_pSt9dG(aBtx$=!8n5wpKX5L= z?PuBUocF+>VoXUdd^LQe6Ej)WGhn}~f*g>sLf^Qwa|XOEq0-qdbmb?w_$Y zufIhvvs~Dvgh8d$%(v+_yW9Y>wVR@49t`_;3y8u2FHiIcb}%q{LbmTqwYl7(<2%DR%`Og!V5$%|teW=3j*xb!2dus| z+uzw&_{M&Nls2DAQU7f;Tz~CAM_parhqV=n-pZkh4fN3;A$5XUFh|>)KuCKC$I_9T z)tdh$+qdjw9APew;CyHkucdx6=o`pE9lIjo_b^!Rdi^MfvlY`rsuZga@F&pSislblb^5maI?eM+#t~PTm;{c&>HbnWS78l z{%L8ZW3Jz2ds>wZjizd6!O&)7^RZVUM2w^s&ai9+2AU6-@dT`u3YvSjYN? zdP#WU#q#a@McN+SmE@`6_I#Qg;aP^*?3b~m)3^KK@YDz64OWc~UxcR+F$Wt)Lt$pm zb_a7q#5JW7Oj;>D(Du&#UszAGx~=lt=)slef=8im^FRI3gUi1C`9167<^?l#-64$A ze1hLWB^0)ETwTWQt zIe(gCllt}DOD>xb6@I{$FhAO{5iWt!g#n8Lq}1*eVJ-GKD=EMW!EI~ z(#AAr$R){T-&UME<6_TgDLURzz-p9&43=*zDLQ(j)bkCfKddo-C6vJdo2Bs8PI~k| z?U9GuBwqV_8IM;Ew+j?6p)8Ef*%;c9@+&3gwyFinKDCMISKmFWT9JF}%%qwwlutFz zJFcAAMe#YGb~|!)fcj(eibu76UlaL2u`fmu`&|8sjqdXn0bA=%ow>f-#WXEl{^B|I zPbl2+QscZ65?u$#GHq|cA1e)1Z-7vLe+bFl&e7V7( zWPVt*-#ceBwCbB@ruVcJyyjd4@Ln-nIyE(Ss&g+EMcTHe>x+;tsF4gmdvv`7Js-~f zo~trgBo7{!_HAyQmH<6jKBw6}mF)zny}i9XV2XPIc~ZX69`}sFl~DOcRN^PFJX2)i zx;i~G`F3dZ)^euA;3fU6Ex$MhTp@?iP`Ay5A94mh^DM&?z3h%;fBcP$n@%blG?|U6 zs7_GQ@+sMOzjI{w1}dX9fFnbCn1|*zT%{oi<@;GDx(L)C2g-(-vOw5u6Ynz9cpSk4 z{6QoD=J3LdE5yVxh<@M)B%a*)DE!FXnuxrVm>`a>)xXI}<_HQgizdD*K)Dv(dPMPM zMNU5e`RZWyBjxM6bwAu-&7)LR3<9SB?uR|SO;lezR%XSw3v^h7$?=MPd3=@w*P6iu z7k-qJPJijc3K_PsbXbTzqKwG*_S>}{eMyIS2E{A;jcl;emDoEe~KezYJ;tokGd0U=%WluWkLgZ|8mT>!8tj%rQy{X*vS zllxs(*P`G2Pb6>LYUH5+uXEJz2-gdy{RI>o8v zu}#EQRsxP_#S}dgV^bXuv)V^=W)HYKER^rzChmQBZ6`msfxty9cux5PXA5_FO!{QfVCTypbv&tc0B_XxbsRP>?a90Zfryd=G3!{d4 zG;#YIY^7qGQ5i;Dh^bleH!}UMAT|PVDq?ewP{`Bv zc{(iaMgCx9MDR$*K74@##Ip+n7Rh`cdTtEvqLM3G#8@kl-R{8|Ily8U+H`$p5HIPU zjn~P}hgPz<`gJ&FhUzYc`e+Q@}6&vr+{Ro?^dzT9`OCk`SvIKYFTEyyeGmyAMK9wHy z=E?xO#h5pqAhE)0)jWHn&TDhp^(UL}Ym$$ISQJ8mo4R{ce=~9lQ@4`W4SNI)BCk z=Z)72@fHZf4b;~PYrvmI7tXc^O)5myk!1X-~C(KuO#5np~x8ObXQUEJP z`aj(6M3I`sEd6yIW8?<{Kj^;&hzMrEXSn%q^#t;LM?wy=LC*tZbJ9@GiKnjNDcinP zi6uX`)dqie4^QM0>_^ixs{q$$R)nn9^&|Ue*GBo3sO}H1SA3}vDKU);yXy4|1!vhv z$MtHLo0>yO{rv`wi!<$E|4ef#H{Ur5rtEa&1skw^k_83}`5^1S8oht({o=LIpE@i2 zdN%#6lKpAdu?-BQZV4)C5dsvT$@44@e88OagE4z%{AyoA&HTZSrvUP^iDk`tng8&x f|HlBuG^Q`u!d^w6_(|ls0YkmBr(ft?zVp8T214F` diff --git a/man/figures/README-unnamed-chunk-6-1.png b/man/figures/README-unnamed-chunk-6-1.png index 7cee6f8d90e73f6bb054ac6d615f5c5b5a5535eb..1fc2486f3d070b22ff46e231d366c02cec1e2973 100644 GIT binary patch literal 5921 zcmbVQ2Ut_vmfj~Z^e#=piZ6DS6I1!rJD zFu>8TLIVQ*>csxG!2&E+H33U62?OYT5a^|IiqJ_*Pz4El4bvf2>8J_hgait$5NA+G2O=2k!4wh- zI|~b^rV6KMWGoT{wwA8@6?Lui zDbb$A=;xTZ78>o?{cZc`W>2h+{nXOg_~!>i*lx(5z#SHW*Yh38Uo{n8u4M&B_<@cL zNWcJO*i_OFu%T;Z@Z1gv^7aCjL+PkL>E&ERYdfXCDiLfo*m`d6x zQ?-<0#z8;FiK}&1=`a_w)X|)2tE+u3y5{Xa&stcALm$PK)75>&!Tmtu16b;akDI0`XiND++lyVlo)1at+EzBTE|Xl`yxoJhO7@>+2^&+sd=KrS9h3JW2wiQ-?H2y`(ZQ0Zk> z`BDbW`G&UAUsQp^A2=%t(AoH@ULdq6 z);1qlr&-us9CT+E_W7F?BU0b7{&VlFSz)Er-7rR%>~jhdkLgg8YN!W+M(52XN*mz* zS_^N6_3ZSP+}9Z_G0Z>|?H6_;O3UZ4T-R$Br8I%1s(`U7o zl09sLJx^f7>zZ|W>TW=0|Huns#)?*tnRSvjJKpH3_WY@p$qEL*4FfRtsQ(iU&EkX>chlXUld|KwWJT%xrNib*<=LT#LUQh+ptUw?P4dnL$#{n?W#Rzt| zp(ZX6Z34$jKdS_(r6(4zq@q0?z_Jcl4Lo-VDo14Lmmnsvr$08nQj<_mjcWazW^;63 zQ?Pd-tNo*|)k_;R|Me5CpSdM_-jN|*iDBr*#&>U@wQ!sI%znF9h~OXX0w#mSDh|(M zLR2ugNb-15D8EZguvA;fLA{R1YtnI z4czz&2fNY0o(Z%Op{5j0uz47UKUfQhpDo}(%GhpaQ4Vh^=?0HpE-Z^|XGr}hE(D@i z-Q5nLYI)b7Bg=*PUqPkBEH&Lx?=y_6A#7X7fJ@|^ z=Y~{osbrs}aTL_{O9es&C*tZ3Q`1-W8B6YYKoSg)#KC0JKA;JZ7yln_6bH-NJ0gJB z5Fv_KUcULtHgfK+62kasDq7azE8^mnZUcy^qVbn#z)9idvx_Eb;G-z#Gb5r=SA6LS zaiQAhMY3gTjts|Kd7Gqi1??KdrNvYHLBrz7k4D=AzMyjSWANGwO>*jJx9oHFV@adl zH__ueHOnTJ?W2vN^F1P)by)ZON`$gyqW%bM#(mleUp|Hlvyipl`u0*5NOpkliHdT; zW;sx<5ZuZGii2CbLaT(Ax@hvm^hj-vHMif9>o!Bxnv*86xF^7P@~JxGaD*#<4MrCExHvur`#z zu6%hLYTF#dw7mD%8d!E$95?Co748`b1#wh2o-IVxmWZ)9kd-|{?{Y(zm71F6?`bL)rM*~S3bb-Uo-4DqnyY!WnsDk9W~ZyHqcO# z$%M+K^7$P5k-ogL94)J9OS1D@+Wl#>>Z_u=QQ+!ANi&1UceNz#)w zx!a zsXVS>4d8=Jc#yTNA2%wCx&@K$A2qSYTxo=#Ed6W0{8bYEW2l=tP_mytcWj_2%=mA+ z51_k)S%*AAZiNqkmW4;vshUn?XyW^rIz6gqVDIBUHGFEkD#j7zl2kTI)) zdZ0V}JHbw#Q^*#qF9}-rpSxqK$a|1xDYaf-M$AlP|@c-bOC)9``ML{7eYHJZJA7(08oe zS52&%kSjcJ1B}Y4Xfh*Ld5Oy3>Ut9#Iy#>i94FNI<*=M7z;eiD`G1|<$sk?M@^**b z7k=qcf!{bQQs9&<;w7;rXbk#|@GPnbscHq3zfULsvSEe|hNtS@*M1mLF03?qEr)cD za9;r#RGrKN6c|YLc=y~s+6X*`>;>=g5I0M^EATq)QXEa1&z<_o&wGz2c? zkV&^cLHzlKr!RwCF+Ou~FlrtJm`>nT*!Dg-g!aJz@X-5LNt6)~Ke+D$R+qhCsP@Dg zQDA1s8Nvs^Z!%_+oQevb9O8kZ*W9g&EuSI4?7L?ET1~7xi~J|>A+l9{>}bp7)T9KYD_+hf`u-Pc^=Lh-jqiKQ768|;pAiZ zsQB$i-5N9Es(ug5i|j5x1yXyfm&}$QYIvPJ#B7hL0P_4BojNGX0trA<-hW%f|Kaj~ z8m$ik&p@fiw3@=Am*+a+C`&ic>7rZCfM30o$W9J&NF2@yPPXomiq+b};Wiu}ovu!2 z`)DfUvC(p*f}iV)MUoLKv387=^X*MGrdx$?oZ0A7tcr{R-ncsbELMXKQ5JcWfoskp zNfiHK(YDAz5<=GuPC-Z=nV8WQdaM8bTJ~oE+ptfB*X9&$-#Yt*FSg}(A}|;a@Aks& z=kskCNaQD(98DVoaTjIUSQ)dY_nXqEvV`yGcasD-K5$)_Lv$7d`}7D%+|%rVTX%2& z9GMEQ_4%;lR{n#jz%{H3mS)4{r){YaB^0`LW1V%@A}NTK=p1%lc0ygE0JyaZynIQ0 zyHpZ_=yYTwx`&;2YQk*7nWKa(BP6as!Y8kTtP<*0cepK6#XU{-Ma@U0H2;$UGp~JB zd=~o{>HT7UXl3WI*@ESEb93IyYO1BN>~;k-@jG&5;Y9a;oAx&!L&iZbekT#E$tkRZ z9+|HUVH0F$Ci|_$pxTkx53gRcph{ev5y<{fhadvr=y={5?V=%r6CFAIe1^v*ERa$R z+IXIxebD(1@pS~&vd~w9Ai-{Y;%Eqn<3pu7p@`Hi0@UV4mCq^Z37=f(LORh@NEuM4fOzw6thR~a-SfvZ3uZ@_kZ4H{zZWP^VT#DO6$0On;lQ>ObiBB z*`itnR$QK&LQ)(L-f-c^c2{<}Mqu`|9y_sOGw*qwr9N(nliL-Nclz!Y9-7x$4uSqz5(n(|2toGE3Ff;w**qI6m;pBW9YCQJ)*;llEqQ27ioi?%MCai^o@HY|+|wtqrR zblismD36T`aZ4=t^z~^ts`rrG+8@z!QqJ6wQy?pkdH`Df?Y{ISyvzULzMZQOQ`wWA zapa^#Vo~O6{z3Yw0XvA<`^#)j@1K_a^Av&pQ3B!HOc|TTei)Z5*J}B7-hA8I;W+kD z@@;GXk;O^!vrJ#d*hA%|78PsLlQTtYg8H&7Rz!D@#qX^y98WTDPKaK3keR_nyq!sv zdeC5gcHNG>W9axm>%D7s#SPgWX(!&FO1i&iRgh1&A{cT^H#gmn^x#*x% zRyoTJlpKB_c3#ZIsj(5i7!2KACNYmM6Zg5HQ~ z{CpKd`K$s;)M46irA{y}8y@*3AcK14c0~5)hqCvHwft3Sh&b5*QpMK>Yon|onVI87 z?(s8)au@25qs&nTMMaZk&&a@<_Z>{eZKMOUlAtYBZs9v%&(FInikiD@A7+rqmNA}W z6dPI5fAJG^sFfjl;xZvlvyy8OxtDb|pE5 z#US?i+av|na%S)fRGf1hz>oK_DCHOM0Hxq}#{gy_!0X7fJjECGCuO@9^gtVu6TeoT z`ZqpKLBxn4V6hhT36g(ro&rKII`xLGPwmgYqkK)TloxQ!sX<(F6#oyPfJyRjg zmbvGbnhw)WtzlndT;9b{dDXaBw-6iq~k$nux=fluDPh&oz>n(<=lrgeFzP2U_ zc{csY)!L)pDa}H_GU~dsBoyr;RimE1RGfEs5H%aD;|2Zl5uQ}8Ye5aM*uJOK`rlRm zwGb@frt6Rm%c4n)aW3BJ4p$bxt7}&iG%nJ#XhhpC?Tr;nJbsF~$(4)%0TJfl?i(=> z>rCr?{fQme59O0RAksKumRu}|A6z=W@a~}AzGBFwmeo7M#sPVvx3n`$?ZtjU!v7 zZLAJ362H6AhSWdCp=RZi-TITNtC6Cv<`iN(A0eMvLF^m4QvO;$s7SF?Pbpx0o+o8| zYEU`ZDB;`6rJ7U68647gioUuM|F*ya*J{tS%F9-J5okFpvN8WiOX#v?o4J<8=nj|k zJI}Q~Mqw%!<;lqNs~FCK-6)J?yy;KJh|o*r@6@$CdOd8gMRz>bjkv!iR$|;wZFkhf tJuaSp=j;`FDeOT6n(4nJ4xQ~*Xz>;}{gxu=@!q~uSIZDxa=||QKLAC(P=Npd literal 7412 zcmc&(2~bm6)_!3LHmeIDyRF@VY@%#p5C|d&vPIcsM}-hUg32DY*bLBsED8Y(0s)#e z0wOW&pdyR1B*;zxMLL?FtV|HbL4nwr0=duINr`ER|X>fLwFdG~(j-0z%wUmU{X zq5!W1F8}}nCdL=8000L803aXsae_VOS5%9c~&Ddo+0hSTE)O3RUR zdqLrk7ISD5DuFyvI-#rWoV)gh9Q38-j5m+#f*OdJN!`tB!){w*cS)v!CbaxoOrCYOa^_^*blzD?!lKk#Yx9VC zi>l;}YpACCdmLV>@x6bUdlM}tcVt=cVM2+DW&K>B9qW1V*QBL6EizV?b|av$BU$C) zf|hz$bX@am?c>*jco@L;#R>6Jf=s-#_ixyuZGHmt#{esHs3`T;PrE6J`qHUFt<;Ux zC6_=31{LksEvV*t18t<%{*j6i-{l_Y4p68Z58vzkT-IMIdu-xHuo+;+#Jg*s_SO=h zyGYSmV#w2SD+2ot1w1<3ww^N+b>LEU&@nCfCwx}${%|OD-PB)H1B;Z577FBlb2|E( zlHO)!p7P@Yn38nv?%N&zZOL73>uVX6r{zY+(_KuOr1#RXz?u3vv?%z6vM3o%tG z2|dR6n$uU$TcRPj^gkV(DVddgCdW(+^oU+m7~!VBDi)5i#b_uKL|R5AQ0)h1SQ&L_2Zyh0~h{kI&u37SKI zLpQ!p)qK(t(& zye`ekX{e`V&jtT+gNCispyyu|gx5ork+KcAlU)7aXnL0K+!26T_{QC-hw7-fMj`R? z1im?xpkmnUDcV>49L!y#id}3 zOt}LrNO7yvj@+AWnaheC^-THZQzKzw@;9gNg%luTZ6lIK%U&c^X?-8#^9-}fc9|w^ zm7kk#C5H$(jA%cl@ONRyk~TvT+ty#+Y+u=2)G0A>j}q4A?)hM9o;i7jrv%d(vQs&K z!<2kYuUlwH&>;CoyCK(%a(&a@(^(3OVhIKdH`XN($){_O3!0xMcibfpahLRqX=<(W z>;p8a9uS|j{G44~owzBKRQ3{z#gBB`hK}>VOoay z+CI9u@RI99%cVWMj8UI)dW<&)azd^%Y>px<+cwaO&~hy}qEgUoI@4H zs2An*IDL7jC;t?je;UO<7o>Z2?bz_~H%i0%PgJ`1TL{DZnS;w1C@H)SUR*JeP*K7? zWI>WZ=*=pH_PI<|_`(g857SFexlN2byPKXfQO-dMC>1TPIEqAAvPGI-@=GWXgq}h- zta_gvS;}&Zg7Tpb=MU<(yS%lBP6`s7<88O-rH?&6ECFMMsOQNM?C#!BaMqGr^cm0Z z-T!K^{P{Hhmqq_I?5F1yfWHp~rrv8|l#rvj#dtvreW0(H5K$Y?%R$2B3eYkd`MIdu zc74@qHe<`h8FtkIb~VJ4Su-cbN<2NWvg8ORG7VS1AVc*~X$7n&KAoYkf=f(Ex9aOB zV)!UNswa~Vsa^~zZK(CuYyEt8lb=7K{zAfALRPubK{V9nDqaKi$>WCJ8-fJ2KuCxO zRqKtg8bzmN)GtU;5yJ!_y#BVLgI-jbvW@u@Klt|0q?-UNeDe@ibD>U<8YbXnIsrj> zta-tgtj@9fM5&`DTf&ibkBYxdKL7er2j%XqH1V$j4Uui5zxI(bJyT^s4nKFcDn$LMtu)-dffC z-Sm4QwO_2m()ezTYrN)XREbH7`}Yl}&vIl13EUQ7R6RcKO?}5-MjkJ#IvcQ5T9U~> z_C9b}qa>6axjG`w^93ECjhZuD0|x)l6Dj0yFcCU4@Vq!*Al(0p+UG!E>B_%FQ~yTf zeTND40gL2z1O`G-Wld$ZHgK#(a>V`o2Pa`qBMam2XMPEsfZ#J4c@HA=B8E_vN-d%X z39%jZ<2{9YcUW6rE4395qdE@Q_PuBu$;^GZS<8#CDmAp(Z)M|9F~M7*NteJtH$KXN za3MAfa0l@bgCU!3PlpK+JMIGYjHH&GWy4`C)0uP`8n6ydo(H~!zlm92D>c>7WFuG3 zjWPw9C9sYpOm=HiU8CQ*P_5T60o9W%B-F`upPGNFBqtlO9yT z9sb7fT?vwr`d_o+Nyyl}zd2$|nGjIWhkabnv?k>x1NBAXaGn21CjLY3`Hz-uIs{gy z9>8j%BKE_Fg`4gi+8D1U@-w8V2i2lE=?4(b$_QM?ue#erHQ#8-A)J8U!HtQ2Z)mp6 zZl?GqGHpn#F7BK#`h=R57IqJKKFAS}nC|Jx=-A+Uo_qJ6`S}bNbsKcrgUh@nWjzCv z6ra<6vu9;B734gNJ$damE?+FrM&>PuBB@eu+~X~4;x(it!ZpOz`K**b)1#p4Ho~Pa zgL25kGYynrNrY2b3jz=^_7B+HGe}Bf9TC=p=uxg~GePSb*7nOC5N^B%0-QRLn!ADV2oM)3l-5H`r!}A zi{1`#-ifvV${9Zpns`XIQ<6aWTD0O5VL^f?JfP3eD)zLZ?=X~#DQtbkHDGnNixiH- zy@D+M7&QjF|Ftr*azX96LQAdk28ZdZptD^*&+hIDS9aMObp(2X*jFrQdQwL$&zh>z zRS88&DP=zoiBij5qPFBcL68q19Ai&6gLD%LZ~7{H+#$NSl1<6lfY-t)knwakF25M^ zv%H*dgWXc_6b}xyf_uts+tmwBzt6d^dhvI!5P=PwUe(XcB5b-*0fbHUD07wru9{u6 zGzgbZIWMqnve`%=oQtZ_12F9W0FM7aZMq~y-E`G@oP8f4=~Fh8$d#gE!J*ackjLYE z0aE*6wCB8}jmZqX-{CSR!d||zg7{ULy|?!0)C@lCCu3Qwu=)+3yb*YTcF{<{f?7%J-E?1pzV(^n~Vh=l#y&+@I*frJ$)JM9J?RZq z9?Ln|$76aeoFbAN5C+BvO|ySZ=VT01f<0`C>N^t9m7Mf7p`x%T&`JfVfgGlm~y)j)OoOaz?l;9P@lD?q-H(b@JOaK z4i)&N$5*#jtbW5AL5X+UcvKALYEMxR^2gJqvNt{79&Z z11EJT0t%8S!cpy%s~UvP$k70NmcZ>oPjRrL+U>x1JMztcFZf9ZCHntc;+0&5AC4cd zO`Iq!K5@APn;xnl^^^x#SB=j<++T2{fVAHzE-%63)p+~zJF7I5ved%2((BR1uOOi* z{4i~uEE$=OGZekoif_5mq2Vu(Rg;z?!T2_qWJPlIv-7a@2XayPL=3yvoeG|QVAijd z=RMKiFe)#ZF6JE~Yyjr6ztyJac^j~OHuSN&5R5o&0B>dtI8=Hn!yUIk67YksxE{C6 z?sElDi8%q^ZY_XtIydo@i#)u_xBwOr79is64cmWWKmX6fA9_D^G2=Rf6i89v4F@AW z^dfesEbOOvaLMm-T$<0a0k`q-Ioal}d(Thq*U}l#L+7JdUbwssuh=+t?1=!HOe>zY zR$B;6Tf7b2jG=hBCL=?O(0gF4t#0rQI@q2}xpdE_Z`@Kk6W{iW08&-KTIkZabu+T+ zFXzMu&}l|~dI2f=b{+^DnWt{_!kqgLAI*=BUNW!;!M+R6+AfXKKxOk|SYBz9--?cX z_A(fb`8geSp!|WpBUssmPH52AXD!4LJY}jk zJgrQqPQ4-bgFs20aaU|t7h%${LHytnu{i{_Uzd-fF5UzEUBA8 zcM&<**>e5CyQALYtRkZ0o4JoL^9J5mxkl^S{)jY`n5vP8di)41Xmsk8OquUD}_u?;nok*=KaWLf2V)%9{(*w5g7o zE!HTbuc5ApCbuFgR67wK_F$)0``b*oRPxeMCjI4fLIWiuX(YzjN9pw9vfIVoOcd?Q ziCY=LmFI8XbpEM7yXqlCBPJ>6U@CKfsa0dHf#WDkY@9F$2Akxfu3<>=aU4OkW7So{ z)Yk+J>`9DXrMD2h`|2vTEUO>yA9PAd6UVWoIOw-Sw49U^H&{Arm#Ye2>RpQOZodL= z`PQ}#!-_WM_V-`{Je~-{eS=a%+)I4tFY28O3EEy?+30?DI5{=6+U+6!PN4rv(`y1v zPH&OX+P&<~%4jjGXEC>rrE|lhEc0-h>0;?uG0mP>Qs{c#dz#PgPa7_rA>J2`uq^nN zvWY)EoEguqIuB^bwP;&==Ql)MTtNo~bz3YYYs~Mgwsq8Kb{B-Y^{AHpT$Z`C~{Ma zkrsV^T^G)JTRK=O2j=CF3U%GFf@+oP%M(+My$M3p`Z2-C_+c_CbCKcF zB5_$RyxC4&-u10ef=gk82w~d2s`TBcG!c-sX5MU{^`0ikB~T=x=8WUMUB*u7rDQ&9;%OY z9eJ3xqxE^%jO}~mq4%5d!)D2oB29KHPn&nn4Qp8{Cu^&Nvv136kZjsh9J6gIS?O)B z^X*d$`=-K3IIZVQkG)`dU@LU1j^5jT{eHkN6BUt+$3a8WJ$25kDq~Mg(Uu!_sda^{ ns(tz(QELC)x%c0`gsrk>PiK831Vb_Gua_o<78fecyZ!P%&wG95 diff --git a/misc/pdp_compare.R b/misc/pdp_compare.R index abae453..849a0e9 100644 --- a/misc/pdp_compare.R +++ b/misc/pdp_compare.R @@ -41,18 +41,22 @@ pdp_compare <- function(x = Lo.rf, vars, im <- im[(length(im)-2):length(im)] } - # if(inherits(x,"gbm")){ - # n.trees = x$n.trees - # im = gbm::relative.influence(x) - # } + if(inherits(x,"gbm")){ + im = as.data.frame(gbm::relative.influence(x)) + colnames(im) = "importance" + im$var = rownames(im) + } vvec <- colnames(model_frame) pd_num <- NULL pd_fac <- NULL for (i in vvec) { - tmp <- pdp::partial(x, pred.var = i, which.class = which.class, - prob = prob)#, ...) + ifelse(inherits(x,"gbm"), + tmp <- pdp::partial(x, pred.var = i, which.class = which.class, + prob = prob, n.trees = x$n.trees), + tmp <- pdp::partial(x, pred.var = i, which.class = which.class, + prob = prob))#, ...) names(tmp) <- c("x", "y") if(inherits(tmp$x, "numeric")){ @@ -70,7 +74,7 @@ pdp_compare <- function(x = Lo.rf, vars, y[floor(length(y)*trim)+1], sd = sd(y), mad = stats::mad(y, center = mean(y))) %>% - arrange(desc(trim_range)) + arrange(desc(trim_range), desc(sd)) if(exists("im")){ imp <- dplyr::left_join(imp, im, by = c("var" = "var")) @@ -197,6 +201,7 @@ mtcars.rf <- randomForest(formula = mpg ~ ., data = mtcars) # p <- partial(mtcars.rf, pred.var = c("drat")) # plotPartial(p) car_pd <- pdp_compare(x = mtcars.rf) +car_pd$imp car_pd$full car_pd$drat car_pd$cyl diff --git a/rfvip.Rproj b/randomForestVIP.Rproj similarity index 100% rename from rfvip.Rproj rename to randomForestVIP.Rproj diff --git a/vignettes/Vignette.Rmd b/vignettes/Vignette.Rmd index eefb32d..8496778 100644 --- a/vignettes/Vignette.Rmd +++ b/vignettes/Vignette.Rmd @@ -18,28 +18,29 @@ knitr::opts_chunk$set( ## Overview -The goal of `rfvip` is to tune and select a good Random Forest model based on -the accuracy and variable importance metrics associated with each model. To -accomplish this, functions are available to tabulate and plot results designed -to help the user select an optimal model. - -This package contains functions for assessing variable relations and -associations prior to modeling with a Random Forest algorithm (although these -are relevant for any predictive model). Metrics such as partial correlations -and variance inflation factors are tabulated as well as plotted for the user -using the functions `partial_cor` and `robust_vifs`. - -The function `mtry_compare` is available for tuning the hyper-parameter mtry +The goal of `rfvip` is to tune and select a Random Forest model with high +accuracy and interpretability. This is done by tuning the Random Forest +based on the accuracy and variable importance metrics associated with each +model. To accomplish this, functions are available to tabulate and plot +results designed to help the user select an optimal model. + +The function `mtry_compare` may be used to tune the hyper-parameter mtry based on model performance and variable importance metrics. This grid-search technique provides tables and plots showing the effect of mtry on each of the assessment metrics. It also returns each of the evaluated models to the user. -The package also provides superior ggplot2 variable importance plots for +This package also contains functions for assessing relationships among the +predictor variables and between the predictors and response. These +are relevant for any predictive model, not just Random Forests. Metrics such +as partial correlations and variance inflation factors are available for a variety of modeling techniques (not just linear regressions). These are tabulated and +plotted for the analyst using the functions `partial_cor` and `robust_vifs`. + +The package also provides superior `ggplot2` variable importance plots for individual models using the function `ggvip`. This function is a highly aesthetic and editable improvement upon the function `randomForest::varImpPlot` and other basic importance graphics. -All of the plots generated by these functions are developed with ggplot2 +All of the plots generated by these functions are developed with `ggplot2` techniques so that the user has the ability to edit and improve further upon the plots. @@ -54,10 +55,10 @@ library(MASS) library(EZtune) ``` -To introduce the functionality of `rfvip`, we will look at a modeling problem -for the Boston housing data (found in the MASS package). We will attempt to -build an optimal Random Forest model for accuracy and interpretability. To -begin we will run some preliminary diagnostics on our data. +To introduce the functionality of `rfvip`, we look at modeling the Boston +housing data (found in the MASS package). We want to +build a Random Forest model with a view towards both accuracy and +interpretability. We begin by running some preliminary diagnostics on our data. ```{r, warning=FALSE, message=FALSE, fig.width=4, fig.height=4, fig.align='center'} set.seed(1234) @@ -70,7 +71,7 @@ rv$plot_nonlin_vifs ``` These functions assess concerns with collinearity. Notice that the VIFs from -`robust_vifs`are all less than 10. The partial correlations with the response +`robust_vifs` are all less than 10. The partial correlations with the response from `partial_cor` are a type of pseudo-importance assessing the importance each variable does not share with the others. Now we tune our model by assessing four different mtry values in the `mtry_compare` function. @@ -86,9 +87,9 @@ m$model_errors ``` According to the accuracy plot and table above, our best choice is when mtry is -4. However, the accuracy for the best model is notably very similar to two of -the other models. We now look at the variable importance metrics across the -different models. +4. However, the accuracy for the best model is notably only slightly better than +the models with mtry set to 9 and 13. We now look at the variable importance +metrics across the different models. ```{r, warning=FALSE, message=FALSE, fig.width=6, fig.height=5, fig.align='center'} m$gg_var_imp_error @@ -96,12 +97,14 @@ m$gg_var_imp_error The top two variables are consistently identified as more important than the other variables and their order remains unchanged across mtry. However, the -variables 'nox' and 'dis' switch order as mtry increases. Common sense suggests -that pollution (nox) is correlated with distance to employment centers (dis). -Our common sense leads us to assume that most home buyers consider distance to -work more than pollution when selecting a house. Therefore, 'dis' is likely a -more causal driver of price than 'nox'. Consequently, the model where mtry is 9 -appears to be superior to the model where mtry is 4, despite mtry of 4 yielding slightly more accurate results. +variables 'nox' and 'dis' switch order as mtry increases. Pollution (nox) has a +strong negative correlation with distance to employment centers (dis). This +makes sense if the employment centers are responsible for much of the pollution. +If many home buyers consider distance to +work more important than pollution when selecting a house, 'dis' is more likely +to be a causal driver of price than 'nox'. By this reasoning, the model where +mtry is 9 appears to be superior to the model where mtry is 4, despite mtry of 4 +yielding slightly more accurate results. We now take our selected model and build individual importance plots for it using `ggvip`. @@ -117,18 +120,18 @@ We have used the `rfvip` package to tune a strong model for prediction and with reasonably useful importance values. This was accomplished by assessing variable importance and accuracy metrics across the hyper-parameter mtry. -## Example: Lichen +## Example: Lichens in Pacific Northwest ```{r, warning=FALSE, message=FALSE} library(rfvip) ``` To further demonstrate the functionality of `rfvip`, we provide another -example. This time using classification data. We will attempt to build an -optimal model for the Lichen data (found in the EZtune package). The response -is presence or absence (coded 0 or 1) of a lichen species, Lobaria Oregana. To -begin we will run preliminary diagnostics on our data using `partial_cor` and -`robust_vifs`. +example. This time using classification data. We look at modeling the Lichen +data (found in the EZtune package) with a view towards both accuracy and +interpretability. The response is presence or absence (coded 0 or 1) of a +lichen species, Lobaria oregana. We begin by running preliminary diagnostics on +our data using `partial_cor` and `robust_vifs`. ```{r, warning=FALSE, message=FALSE, fig.width=4, fig.height=4, fig.align='center'} set.seed(1234) @@ -151,12 +154,11 @@ rv <- robust_vifs(factor(LobaOreg) ~ ., rv$plot_nonlin_vifs ``` -These variables have large issues of collinearity. To illustrate this look at -the pairs plots above for 'MinTempAve', 'Elevation', and 'AmbVapPressAve'. Most -of the VIFs from `robust_vifs` exceed the standard threshold. The partial -correlations with the response from `partial_cor` are a type of -pseudo-importance assessing the importance each variable does not share with -the others. Now we tune our Random Forest model across four mtry values. +These variables exhibit high collinearity. To illustrate this observation, consider the pairs plots above for 'MinTempAve', 'Elevation', and +'AmbVapPressAve'. Most of the VIFs from `robust_vifs` exceed the standard +threshold. The partial correlations with the response from `partial_cor` are a +type of pseudo-importance assessing the importance each variable does not share +with the others. Now we tune our Random Forest model across four mtry values. ```{r, warning=FALSE, message=FALSE, fig.width=4, fig.height=4, fig.align='center'} set.seed(100) @@ -168,10 +170,10 @@ m$gg_model_errors m$model_errors ``` -According to the accuracy plot and table, our best choice is when mtry is 19. -However, the accuracy for the best model is very similar to some of the other -models. We now look at the variable importance metrics across the different -models. +According to the accuracy plot and table above, our best choice is when mtry is +19. However, the accuracy for the best model is only slightly better than +the models with mtry set to 5 and 33. We now look at the variable importance +metrics across the different models. ```{r, warning=FALSE, message=FALSE, fig.width=6, fig.height=5, fig.align='center'} m$gg_var_imp_error @@ -182,14 +184,14 @@ There are 3 variables to focus on. 'MinTempAve', 'Elevation', and appear to be the most importance variable when mtry is small. However, as mtry increases, the importance of 'Elevation' drops off a bit, and the importance of 'AmbVapPressAve' drops even more. After seeing these changes, a researcher -might consider how these variable actually affect lichen. They would find that +might consider how these variables actually affect lichen presence. They would find that 'MinTempAve' informs freezing which directly contributes to lichen presence. They would also realize that 'Elevation' indirectly causes lichen presence since 'Elevation' drives 'MinTempAve'. 'AmbVapPressAve' can be assumed to be a byproduct of 'Elevation' and is not a feature that should have much of a causal impact on lichen presences. While it is highly predictive, it is not something a scientist would prescribe for inducing the response. In this example, as mtry -increases, casual variables rise while collinear variables fall. +increases, casual variables rise while collinear byproducts fall. No solution is perfect, but mtry of 33 yields results that match the intuition about the effect our predictors have on the response. @@ -201,6 +203,6 @@ using `ggvip`. g <- ggvip(m$rf33)$both_vips ``` -We have used the `rfvip` package to tune a strong model for prediction and with +We have used the `rfvip` package to tune a model for prediction and with superior importance values. This was accomplished by assessing variable importance and accuracy metrics across the hyper-parameter mtry.