From 669bb86c7edda4be66e4c84743c67ca5e8274e3c Mon Sep 17 00:00:00 2001 From: Gianluca Bertani Date: Sun, 3 May 2015 19:22:05 +0200 Subject: [PATCH] Split README to separate neural net and bag of words documents; added Bag of Words tutorial --- Bag Of Words Norm Example.png | Bin 0 -> 55325 bytes BagOfWords.md | 213 ++++++++++++++++++++ MAChineLearning.xcodeproj/project.pbxproj | 7 + NeuralNets.md | 231 ++++++++++++++++++++++ README.md | 228 +-------------------- 5 files changed, 458 insertions(+), 221 deletions(-) create mode 100644 Bag Of Words Norm Example.png create mode 100644 BagOfWords.md create mode 100644 NeuralNets.md diff --git a/Bag Of Words Norm Example.png b/Bag Of Words Norm Example.png new file mode 100644 index 0000000000000000000000000000000000000000..a5c7009fd04ce5e0a4c9338d7233ef0d9222c796 GIT binary patch literal 55325 zcmZ^J1CV6f5^ZbRoVMLPZJX1!ZQHhO+qP|c+O}=Gf8Y1sz3;{U@hhSt>a4vpb7fZ5 z$$j=-p)yj!FpwCK0000mq9Ouv002OQ-)}N-&~L=1mURgL05aE5#0R~%8d-+;hoAN`M_rvSMo0j)SXPGxJZ|m4ka5-Hv zfZ1o&J{eqRAqNRc;4cndAB-4~*`AQWNu-v*01%(<#%N%GwAM}L)Y6A1*RO>FHi%RU zfE@UQJ+{935W@n9`;m2NYzT|@6 zteN%(nN%sOQjE6_f)O}fQ~eYc%f9S6KAQ^*!wwgx1v2A^?qM=j;Wp~i5*)ng?)Los zj#7qGFHD2t1-cOICnoPhGWB7{=e(l?$q=mUCw@-yjF0ODP1VwJDQNKE&rwKd@c5$R zUE(u|OoVb;A7UL51q%PMWu~DwiDHoNdq1-09I(y(rLbz1}N9k-U`P(dnl?Q)87;9<*7Lg}C5T}Ms%IGCnk3uOY@&vip&3-i=+KDg#Qil|)$e%eNFN;= zVEHKj_oYWf;Sb(51^^S>BYm~2>Af06AA3)WSXy^xX8URh1jG=+UU+Cv4IO%lV+x-HTM_77#7W=!5VZaf-Bzta1>_?5nUA4f zeyzqiqybYW2F@hPQFwSS@t)Eir0tt6Obe_Aj}-`u@9-Y- zjgSke2W?j1RNv9g(~k5u&9?Q9@J%+jfggLX-?n53@ijy`cwxX^z+HfPz)=8M9+GVE zF)uWUT}XL|;6VO>)2`(1FWWF8$yq|S*gnzGg2)1TIj$4>6ND44REb5RwRqH6*L|3M zdON}wa@UmiR04jw0vDMi8G!`s1T!(P1MUN^M6rd!HaY!L)p_iBwNv6##8Xm;w183& zc|XEhtvtRw)!n24Owqi8sDku;`cbS=xzV{e7GiC2ghGcx&H@#=vjXyh)dJ8$k^3<1v>}8v{GTs1c1g^jv&Ksx+`j$u$WMv33c}#KF<2DBpu0$;X9zon3?^xwT=XmLuEoyBVZ*ecV z->SWe_`mVBbARtt^`Qy|e@)>nATR$F5M~c;rA*Y zgw-DzJN8T8fo_ONfzg9W03(>mjj4$7fsv4r#gy5Edy&9&#gy1A(Zta#(!AUp%&dB* zYR)T%f4+83bathDu9Bg&ycn~vU9nm4DKN_qN1jHwTIwp6TX9Q}hre6wD++M3XA*Bv zr~7v{zJ-Zpih+u*%N#g!9o;Va1b$s(#E zYZMcj5X~C%-%oy5MvF$JM#6t^(#BKXQmyH&S-g#PO?sP3YFhL~8}FMmtXIrl7RN@& zdry59xXjZWa~_n=q|U6Fo0$31-O}mQE!SAr#MXXusJ6qm^R<^hs<=D9Jv z9loyr>7iP=2w1~(W^m4RkAEu&_8(S@hLs+e8)n$5UlJdp=qq183vDi9S!$njUmspI zZj=tDSzOw3PQJpEPaHK`q#ypU_*i^(e35_ee;@zI21Nq}2VMo<;qC6(@QwC!@O%DE z)r*&c-of!~8=@s5G*TUI6V8XCNa==RhSNsUz!8Dc#Ch(rJZ$rHop&9K@QRQZGAS}T zLLA!|8!wwBJ1EOP1C_TVv=t&7wlml;w5w03A3Z2V`KZ=md~^KsjM!BiH0hK|v-N(J zzR<9Ae3B|zV^YnF9TpQ6*_rc_f+9t6FfGJ*NO>r_o@?2CrJA)WF!Q8?m?H z6nVemnljha&E$(KlM$1<;}J$A6Xp_hwWGSb3w8swm*$aM|bFmAJml>skE^cyBIicUmpGx67?Dl<{E#} zplBrLYDsQTZg4K4%&Bn9e(cJ8>q;U`+^V#xHYrgkUDsl3&947i1Y81s08@lR!Fsm~ zaX#ME&F=rB*rr%0Y$IG3&B=k`+VE!Fo^2jYhE2ixSed5HrGTh5r)aLO?yhm*B8#m- zT87C{yDzXMIu&2}K=Kfbc?|O-GDmj)`8K&l8&v8g`;^ntx$QGzGIfkZP2P)PzN5^w zgSgbNgsG^p`%iUc{YGu)iSnFw`IK2kB`c|O)iLLZ$K}fj6dNiDI|z0)jU7#t1ND`M z3)$;zW|g33XYvP)cIU3=fn2ltQ((E2@|2c+%c9f9rQ0}6I_4j)MJ~y1)vv{Q^SL#T zxkn@OY)@_XufET_W(_ONjtkefG3>)kOlSYgM>qlKbxaL&ElOg;NXJOIVrlcem<0gfbYcJg zXldx6i|1l#VP((m!bR|}8SLMm|AJ`<@cuQ$!JLagMN$Tj-`dU)kBOR*nwEeY5)TiL z)6T$%T~0vgKjz<8Tm&W#4mRvGG|tY>)Xogl)^^4;bZl&FG_>?I^z>BUGpOuctsHb+ zsI2S>|Lx>&KLUpK`gW!^4yM*tcz^lT)w6bV;36RSE6_i$fAchSG5v2OEBpVX^_?Ki zUn4Yh)U-7J`2IHK{0n85F?BJtP!TY-G_>otg~zhunnaa4~4~+rZLZBjFa6pShftT0bc8Xe!Pr3JSuWrvM=8 z07>{j&;pXk2!VVm2_4wg{`;~AHfhaa(F@R zI~OIY=?DtW#x-(g*<@oO!1ZHZP|>xkBK(e8dF1T^?Rj5_rZ2H6G+M8fUlgJm$sK{H zEWV=V>*)%$s8LnuK`42SbCltw=Z**aWB(!KlM7Kpr)Sbfnf+JV|DrW5?h}d7>B1WK z4?@cb$SedcnR4)7nr`1_d8ly0^#70yBIy|l$%Ux*_&YfXUYc*S``sOpe~_N(?~>6m z>LyM7RnUJG`V0 zy<=jZhkktcusAxDavawHPm_z?7;Q5>HSZyzE#i4u&!R542S2h}CDP}QgG4~V5*|Nf z`X#@kITXW$tcEiV{LRmmswbdzIcG~8VR2|b>cc92S;&ssCCxZgj~i3jy0eLRCwp9u8wWy1-P#SN|5n(=4KYV0OZle?{Di{@(w#UgXZh$?&% zvq&kT7s})h;_XqTgd90@asDHoKZ>X6mwFLU zmWiQx*e{)~Jd$x2vOBg8J^06sX$}3;K5b)%F4Y5YiPNX9iGx{sc{_kp;kZ8 zA;4GCRU`L|ab%V)1q9?$A_*-{V_JXW!b&THEsN!o@^4EL0?9oUNBAGXgF)M7hJ+$% z7lr}zS(Do`tmE=q41)Usqg2BByTDc?ss_xwf<`~H9R5iU9;9f(W5y4VV4^?CKqXXK zCZ|bsx!%T)(<%^u5LXJ&%k!^vLmET|lx4|{fZ{8s>l*=H&&1yGyf&)xaIGWBr(3oM z50bbO^0{KH)c6A~u4e(Sv2dVR>)6A%<)7fMQN?^3al_xb6jGE8MhGcn8dz#^tf8@0yLn_N5018kY_S@DqwQO2|6s2`9Q{jugH_ zC!t{N`bMkeQG%N3Wmn3j%+YG89XILIMde9kYMhaXOM-7K9C{t^Wp60julUX7#C!GG z>sj|oQFOa=&bI(AvP7nJ@9=pis10kHg=}^UyFZY7Z==>Lb3$M-FM!8gVqiT~p4M;| zRpt1FI4%CUoaKtQtI&AcY~M`+{R}UXv&GFLJieK6YUp*h?(CUd)K}3ycr2e*zj!-& z9Ig=}j{0gyqVXN2KS)nh2LvRjLA_h%S!P1JUMn+M$P%S$I2%Kqp8~(#_%-6;`?qGk zSEJlqYyC%}1r{;?H*89)5erUGE7}Fn5a-24E{fQr(13 zk(2;Zx3y7RD5qG*{C=)ZPf(~sNnTRAF?u_8C7gEZE&th{H9A8e9?aHsMRFYnV2CwO z6Ex2evo05_fi$F1IW-6>*LM(Dg|QftpOs+(tqDgE*Xs0|u*D9ZN0xo9NZ2Uyui``` zTtarM6dGK0kenznAzxD^??1p0iCmk#-crYsgxy}LUm%nlnH3>nEwm^T=S+w6 zT)=RqWyOI(W`rzU*~zJ=T4uvUT5Dv9?`u?DBp$Zw&Y!MtWMf0~`ql}{3eJ2_p9#qo zUtz>B@#da21V8@>A`$H^DM(JV7Uz(`6a2QLz4!|YL`A=JKfvQuWuvNr7kv;rN; zP~QvO466CV%=UC-k9y0FkrGe+wsgoMr6uCT97!d{M-BN%>NAI(%g-0y18@-EsST!z zqZ`Qu4@eV9li74@A}1C%r4!@4ey_peK%tgtSnNN>In2xmVRknq-?|CLSCu&|OhVYb z_MFi`h6}~=zVY02C)RbPy*}N@XSO&gEl6iPsg|<;rtfQ#P%bvDv_qR!JbPL} zEP3UPkB;h35-_HyX021s7L_}g*gD^8SL8^(UrO4NE{8f&8B3R!FQfF3oeeC5s5(tyLAh?L8OOs;0i zDIaq^2Ui8-pjn{Xtv@htl%jeo&&Xu*$cY;t>xL1)47D@K@8JB>cXk~&lxy>)zAT%e zt}2TCUD+#K z=0R>aGK*fEsU~$|-rhY!jF5jG_(M~%bKVvhRq&u}Zt5<`^o(FOTE$E0;mzxXJ;AZ1 zN^Mg+DbgsaObfT{gR4WaICN&}4y*p)33|D50SXs(Ttd2zgWb?B53`J%T_-7{d4+ai zo(xgJU}ENlQZa!@c~QJ<$|<#Zn$#(|Lj4X--H>caw(&?}vx53uyIPVXW>svL&v5i% zygid?$wx#Q^8oqc%O%Gwpd|T2V}Xl{KCb^=@uTn+PBD`+tG?n_Vs@v_oWg>d*qlNm z4X2RfXL%QrQFR!*&`Ce={7des*_Lu9Y#E(ayFzJ!Kc)JH6IxjqWJdYHw6}?oo9s|Z z?xUT?9D13$v2=M8T;iQ0_AAW-^f0WP#uB>k0Dq-2SgRH3(a1|jvrvhY7gNG4lcJ+# z8Cm>&8b*m2^rxDp71%<;RKO?F*6#$we8_D4BVl>>g@jx?HUg~?3VQE23WH<&0}q1$ z=oOWmq!W4VaMGolSMm+_3X4&h`#L;j)nBp7>AX*}nu) zeW7nf6Bh zK^-jZgb}zOeTt{Xy*~R|y)K_xN$aA>Jy!rDsGQdW^%vAvJ=;#PPo#fXpG?Skq%rnZ zTobBb+I{bV;|6*=oRANme8w1twHWn#nl$g3xJcpTORo^{6q>!-MSn;^**Bzr)okAp04CxZg7N>DmPs3$`t7;W_`?u z!&dKg{`y^^R)?uOPCET!Xz`{Uk<~J8Ms~Zi(r8nGZ+&N*e*I7pcztW1OyDFa*H#v} zgIflR_q<#?`@RY zEy5nm1Y}yT8nW7mP|mCoKq@JU?|~Affk?>3bc~DE`A&4<;DI?=Nwt7~8M0gh*`@6! zeq*B+O?8(%TT+_u@EKQxOO+PYH>K`htA7CGvDnOVo)-G+$K}i~Vdi8*k>0!&5mVuV z&aH`Lp*(@MRKTs>4W2tA$Gd%M(Tr)^lZrytPDd@~8$++SydKo3cs$zZ=OOe;ru4j~(Rf(JO)1j=sy+*2=7&>w;oW zY4@pj{9PylCTDn8~1F>x2p1ZMl@3E;zNPp z2upx(?@4>@t>(4H5@2MNxjc0mT0zuhA-8`pbD_|2)So7U>uVTkQJWr4cPzmm*V{q* z3qNN-HPyR99bEU(m@fqG;>(4DB4B9MYr$@$dE)v(pL&BI1R4jsJ~L-MDT(de1`?j` zk15;&ikdp^9M^*LSfPc%%x%;m%Sd9VTlqJCQ9BDfq|F5uIbo>feyb@jp4; zUN@z-L=9@@^K4oZIg|!BdHQIQeatA_d%&#r_`;UnOWUWlI9i}4d39ne72NrG6J?qN=k70};hL#_Gg64JsCk;+mVBAKb((u>2s{g)BO!h5z1Zg>rm~PvyOEFP23NagUOtB31UI1YFo`9B#PMKzKbsk#LN+-XwTk z$zO|mKkRJh{VeM8jm8=|hb|TjSFNfag4xQMCy{fFTgI{GU$0?2l&)6V4gaNa2s0kB zxPiU)$dz=-fiGOjQ~!jxlD8jkoLrkTWgMpkjEddY^5|WBg2`23HfGrMqC`d3-Rh`` z%?E>9aHlRBG}3~T%`ZXXgnbA6`6k-mf$R#<&|n|r4CX!YIvU^Q+P5N}&B#yC%KT>` zC}C)otueO)6^ie(GS}N`u4c}> ztn?B|`9F-pBxFKx<|leZ0*{V(>8@(BBP~cFS$**6=l%hxH1HSkg_aEQhBpiS+oRMD zT>(O}1Bj(C8Sc$vKdpJM#!-9BJ88YE);C0mMUzE9wZw+oMHIF9rs55>=mo$33nZ!1 z60HrCMHU_ND}_d6LzGMF&vz}@DR|Ja+*;u6j{uB`am1dRn++Ubw7sTH2|7w5Xl|Vp zr-o6@=}d8bLslU}bc4w64ThLWmM`dBnCYc2A?l6ia=06cZ_qX-I-#ou38|{{180#I z)4Tk#K`lSzIYeRX;K)vk-;GD3iGIkHNN(_ua)Zx#1&%%Lc&Z-KhRg8rcW-!qjO)jYdFMJ-& z-c;|C-oD&*+lZmTN%9DIYHpOUCqIVgaVxJeo-%O}T;nK3A_R$OpF>3P-r{>>I1?qW z{0XnukaE5H^%|Z02j(jBjJBn#hIV4Q#@3VUBI2rfPwDx$z{fRL)3nJ%kMp0HqRRos ztqs!AC;L$*0iQ<VrDmC%RLFm5gLBhG1^F*Od+Kz?UHeR)(R-Ns6Ggb~o}MtYWCkN7wU6 z1(C#C@Ecx4Gus!5sw4MKuvlZ7Bv$tR^xFPAQCMrro}R7ZKf z)}9qip$=F~+VnRSk4LGVDreNqm1(j?OmJ7J{?oncT0qggk;7}69cIW1}gJkSaavvwk6PnqO&2Uavf{u#?{FU1dP=gT~d{?&{fg9O%rQNp19p+R(7^4Sxt<6lM(Vy z4khIYx{POe-cPA!Rv^>qlLv|emd*(4yZI$!ZQkEN8?#`YjYjZFa_^swSLWGPb_bBlwI;OgY zRU?YalO2u0+Gv`Oq&OyubV(&8?z=Ly5l@~&&% z2Xy}%0=-m5%Q;6I(s@tfXx%YlW%G31}AVLHb3}3jD^!fuRdIuA;!{qBmlZ8TH$rXXTsdvTe zH(m&qt+-=*JF;Yo9&jNc6N`dI3zv}MU2fcx?vEs@>F*u3eNYEWV)S1mk{LE!6TzSp zNGiLRQHimP1>LkD?Zr+&WlOyLv8>_s4d}rskvK>{A)M(2vf4_1D2>OBjC2$gnYC*p zgm$Voiv@*5e&{?0X!OkbUKy1~CYC}>sCz|?<3kaH-Yo=$3=f&KG%yNN4ATwBz0;>0 z!XWO`RKw%lo)}X7gh*bY89QPuQY_x!eGX}@@2sDxT(hCN3(+xgvX(F%%2$RbE_7JHL?72*y;>T<364 zTk31z4*zy!Bz2C3!QN<;6v#YHVL^{q9FM?;gTO14Z{R|qW5$!&OM@PC$x9U)ZWP{B z68e263zT>@Iv?MnwBjJ;>p^xJNVzn^taZqoKAgHjn!%IcHnZp?Or{ZkwqqEiVv7gr zYWkPU8Xd(phasAcC6r{L%UlAVu}c+C_-CZqY2@4(N(Plhjlr9l!vF+j7Bwb8XOV?p zyQilL;~C7vU|8$nHA9ke!2vi;$?K&y}j~|R=s0jHR33- zvihs5qh|~5N^~ff+KhBEs_xAz?KZ#jc}NC%B$9bV@C2A)yv1QasHmi9k`VEWK-WOJ z111?de6ChL-Rn35nuy`F0PgPP7#tx%OKmLvT`#S0 zDdmf|tIn`|u)s??<4YIx0d%R>VuijU@9jB6bc@i0=)u+KaAXtM7_u>ko2k6Kh&$0e z9fP)V5uj=-B9AB*?Ti@~NA%B3HyQkaH;{KsEFs%0MR|Pzv;32RUJ&S;Q@Go(4lWhw zYx@T4JH7EQYQb0i0D}<5z~(I=hoUQQO?L2EzVq`67=3jCg|43f3<;$!v0B(kNiW4zf5sBYz0<6v3m>qG zl9EP(Bm(B7eahuqtk?tfD|d)9#P$kotl~?PFT?FMtmK{EF82s-Nuo4P>ZzXjwR){N0d!n9x1U&l-O29!KYD^D}5L_ zMO^_A%c2FNAVb}Z;O8=SR_c7d!hq$22B{zfHJPZ7r|g!c}k7Xa_+f zWr@%@4k#OQ0FBJ%B%`JVL~I_7%wi{_YaM}>TEAd@hr&MamO%*8I4#N5QN?#mC}*ao zae3vLS*v;rG?Qb76OiNIM>i!l-{&@^Hj-+!>U@A~bF^B2O#~zlnsO(xkurSHCX1`Q|WhM_vr*^@Eal>zYgw>bRDee3@RPkd) zW5JOM*}jn|iwHDT|HQO5|Lz{S)dSh{SIiPGakJ$U3b`BCtHt!+?ie|J>o{k+F5NmLEpZy z&Rwq|;F4K9`#hZ}qPSjxyzj5lr0^(hnyPFHJ~4Nid9q*~UBuneeBLrt@OY)ayIyfp zUQ}>cKzD6Da~aJ_wF0ADDs^`%{_Mpi=F1^xe|T4jYp@+QHC=LkHQG@ah?ZME_OkTZ zXLS*6zn=vk8*p95On#AhdVZCaNOU^8@R34lerR@=A;A&|{sC6-UjKRLt%B|u{jOKA zPwG)5wdOoTj@3!!t)2)WT#)IFYM~Wu+W=)6zOcc?e>MJTa4p{#oilm(JIoqG&b5y-Ht24e=;jJh>4^)=Kz zzd8cTI?C^x`=5|bt>E(}rf|jx7@e@Fg9?8bemsJO`HVI!$&o8{Zy}dI3FK}qk3QqM zaTwiOMvS~Msycnq4n5+8YR)82Po-+L3fSIds_iW!+MgJj^8uA?SZ#>)cIJoEd7N7a z2mm*<({_efvXMA^pplA9mPZf?3|#aF5r@}ir@<{Qu4p9?kM2YhAl8^{!sH8HGGN;N zmRR6y51H0Mpap#VlifG`ssB9dMcaT#%RU?4Ule8He=T6PH2~fMjTw1iOs$`T*jW>~ zd+6~ANFYCPGH7n*d;#*=ksj7QoIEQs>TUB3&D;KSZhXswwrvYQAT|TqY^#5%uf`~l zg|Aw+OKtHqa+Q(qS>-2)+3Z?RG5Bvf4jZLq^r>pSH!B2MNY6B`emyt!2NqpAw40S* z4v}mC%E?-+F(TIF;+($HLSDX+_i>E~{3nBYNJq?FJYJ%7@!Aj60@LElpDl0D&3PZD z1p%9f82s%rY)-sLoiP<(+jOiFcJ%Z(bLP8*wG&IYYt`~uOBHn%rsM%=!F$?sXu=$c zT}ffYHJ~z?Z z7Ikc4<0kaMwyPpB;Z%$2d(byi0G%kF2up47m9z8}lBK~N!t!671sD7|O8aof^T#b*s^@si?w6pKv3f9x?(9y__4SJkOGa}`XwO(EDn9d1CX=b=`0)kEssisNZ7=(&FF4ZKSZ2Ill(z|XaR zyTt?*<-9E__K%OH{-8*B2Ze1msL^x3E;m+jb)jY`O@8N)-AV1wV&$j+wq|PxW-Am~ zMNFOLc~}*}YDu`r&S0nN>;!EzrFy{6=VLC5RyCNb#fykU``5hwX0RGNb)+HYAIuU+gj9O@?SE!J>6I^% z=iZY2YIa7L9P)lEjIMY`s*i)paF}b2mcg>b`G>|HkD4X117cDZe0XZ*+f{CwL{&v~ zP7lOAHp;@;9j{hC9iDO2*!sZ?RYV4?<%p>2r$ewh86dYU?ubh&BQjq-uw4M61!}r` z4X9gVCB#9W8S-2Ac96eRe>fAdf3YweS=P#1NJ5e2$nLQe>ib7;Tzlh=n+^ci*xUx0 z-t*bHbu#3*JL@74YCK~59MFDJq61M~Xadj+HC=7_HEVrk6FCHUV!EZ*uFJ7I11Viu zQC#&vb?EcJYA=t!jwlNzJr-UO<+v;Bv4KUVIG9p+Mu*>HXhgzupAzfLS^nZ38X2nD zaCv*sms_1`XDW7*ig{0F_VR7RWDs1>X$I`i3qqPc-&mckwQZidR%Bs|G~z-3^Rz5s zc64Z>GAto>Efs%g_bt)*xKJDAuu0g))UFR=g{?b za^xOnw2r}!YgIdUvRfHH!U(pEcyLc-gd}ZJHT=q1Qea8+A`&S;cY)db6@g?9e8?<~ zTa_xz>?9N}Ep1OR!wsVA4ixM4R^MQqBii1ZlfQrQPmFDV7K7yF=7)lqHsWZp_dp$2 z^Y%8UVEa>GhkgzH+hyN;^vS9w>le09;qG%jILHKsGEYgOL93zWx67QR>l+iI!Re>*0`?az zgDuyTRTg_?Jx4y<=~{W~B$dY`0KmH-?Cb2J9u`K=+sK$Dho5cC1x{U6R!6zvwhvpG zu{Yb0dDh&*cPLi`AIhzn)}*sft)?)X@0O@t&~H#5!BIe(^l{}R7x9Tjo1#zLSs|43 zlhW}A*(RgrvGbu!?NjB_krzGYLJ^gNV&YrFVh_}@D^Ll}XFmoeN3y!QQkbWu(wY!B z2-Uy3U?nZvXK3lUc37GO&kh@g0@mGOOr;?Lc?QmwgP(_tP*qN*PWyJUZK#Ol*J-ZN9zR9#}5n>lcUNCY1 z@}GlvoFc(mIP^KJ23T9a%QK#=3yXCQS<);0sOsyOGM-UG#N}i_jUK**vG^+c(y=$l z>!6^s-o$$567ud68RN*SAF_}|od5$yEb;&BH{A$ceZal+Y1^+>t!8=+ui0p? zeVVXdZZ!QlH2Dd%cHEn1Na~K@Z}@ctQEi!i?Y);gtrm}aGp`YmGIG%|?v4H)McmsI z&J{SA)4itmY^fgnxko&QlI`pk+(!|luV0|opdY%x*M4U0X`hL?cd}NwRQG`z@_3h{ z-AeU)3Kn~t)0+m=-C`g>>mesmy>dXIlGYEyo0IQ@Xdbzr#|nRGJdX+mYnc~HE`M86 zXk?7bphRLThzys-M^)bVd}UkoB(LW6>}8f6@#sK!I$B7uS+FADmA-up~3$Ue6 zhl&N+RPPGJRi=|cXtg}s_2C@EV$p=C&%-$OlXUy6QAEv{-fhMUIu9X{1UE7Pr8Wz8 z%a7kV!Oa;b>1qGqk*X3ZSCB`bD_1KB5*F>f~Sj@J~yTw&kp-9tGP?&}4)a)wa<=+$aM z?NHiWl6{)G{q5S6U$vn;Um*P08qb%Ju#T9t=&La}kP)~Y2l)58yw@~wC011iDK zp;prV*#1nWj^`EXVUjW>dE~=FuC7;B0h6S>n&JA=kpPi0LqlhZ;6tsv0FBbNE6^es#pznOK%Cq^=Hr8Ic48); z3(=@4X`1T_+bGb2_B?WWE0)d_8T;y*HY9=>wP5tOd{6dUt0yO`7I)VBA6Z$C$X`0Q z0+?a5zcp+nzy5lr`9(GQ@VDqsr{FJj+H8=1W9omzX$nX{77+>wN&9Ypi{ohJz7aIj z{@T^QC3NJFz8>0cO0wi*|2XOa8H@4}D;%r8&Zrg|OO(+*)CgtE5r9azVyQ_oQ4|+M zyQ4&&AXWlzRu0XrfcJZLqrEgjx+tEyVT|Nv#~S{x6fWaiFbFC<@#F9CIzX@AndL>p z{*O8kl=NFc*Nb2e_&1{zAMS6G$NBxX|3!NLO5bKccnsBW5RFW{}(0R+KAYJU)HQHLAwUij1tgc{AC0Hie=EhEy&@|EW^oRvr-Oa zel&H}f>wj0%Cv&7bD;x(x#efOlxrg^TMtlBD1SsYHtm#u%-Gxe)s{`UCDZz*3`(h7 zZ$S}LHr|G~OMK|`Y?bwEzqeBl`g^?gj)$+};|+g2J6#j7@r;6}w1C3dkI9hNgzpmQ z$J_gQI?X2E!VXHd9ic8EkE69BV+W~OcMjxUDdi2FDHVj54C;#(gF5y5zW$-r3E@JL z)8l`q1 zO1D@j0p20e>yz_d@(%`|4+g^$*@hvg9-**v^ zHV@9%*7P+^dLG`%T7EkD*}#JtJ#~c}{J;(eLc^6v9Qd>L5<6A|U$H(wY;#`a{G5}+ z*d%r;+%RIkWDBNi4^e7@AqNw8615;VAww+enC8 zDnt2-;|Mp9um8pZc=t4Y+fz4(VD5gPf%xAbly92j}x*m5^Nd zjzvl(T-k#|W)XLvTEseAXLw;aNdKDbKzNb4Ji7gN;%v+V49xXa|)PJAxTM>|dn z4+s@;_l9b;*u=YjXnw7$Kfs$2DQH2(I%Lrw!ro|e*By*7I3yB_ahZ4vJN-d;5hAE~ z3iiiQtS)Hdqy1QM7~{#Kc`t4eZ8Qct!rk1czuUSd!+x3%`opy7y!#Ex5A3b~@-bV) z9#%*UW#!3_WbfUT2AbVEFs3QJegJ7hy&^U?4|SX^m6r~D`p-m>FvqAGO%T$gS{O*h zC_KWYD_PR0V4^mVl0gV~aVJfZw&HlT(L7w53{S}#z-U#|4|IY)LwDE_k-qqRBs4oP zr=vZ()738-(tskj!*@QRF-@#b1KNsaErB&R#~2|*T#}*%V)T=nUT|C4?fiUs+2$am zfHJ!T54izpW}z_fsQN^FUo#)QBQ}g*2LHM>loWt`e`PsdH6x&zqK+ZLAa@JgCg$>R zN|J4PZ&5QoQ^@PkX_05BzN-sWd$+xKoOoX!HI^+EExz@88f3}9TAI+qCpj{=!h5re zHU$Pq)!2-L8-wwEt*N|ZKTC8KvnyhePQ-s5(kDU`BdCTQ*vtNPAMvdtk@<6Kxvdw* zxQZ_%Jc~GqG)r3>pz2I-qIIfv`wA+tx{R;8V?h*SF*^A?3)8*yg$%R0GK2Kunvwd` zHzHh_2%6=y5rHcB!6)%JCj1y$FTwy5k~~i@BNAd?Feh{mE$lRuxHsy#sT+B5(iT_) zXr>KKJD_81*XFOiC0!NwMM}C28(qr+D?He{6pF7a`d@tIMOERo2jA&+g;hjj% zBU{Q72fXzT;!;z(-Mb4 z3=5NX?5L^c?G4WHnUNW9MjtvkT$G|$r~?%e8A_l{fy zAB_(5_DeGC8IgrZ!0SE{g{Ol{@vptPV266nTTwY%GclNvn@>E6i!r~x*aKT=p7LVV zo%D&0rtHt6ZUu+M`>)OG-JTUg;GtBG?%}&Ar5Nn;^l^q1-f7bpze0!x8HLJ9MhwZA z(v>@8#H1aX8a(7~K~lWAB@#d2kD-Z84z$uuUB^|B5+ zeC_xU2)KUCcR z-ux4i{??A?Tk}=@(`WYnl7-tgwyO)pp7k~F%G%jC{ zEe5@NrhUl&c0NNX-Ys#3J5S8&t_ee}ddMnXFPhNO?FE{fQ@W|2!e%nQ>sp|L<_>)? zZnSMA<3%+lJf2a-ruQ>Dtbfy!=-K`&6ssQfzf}JJmb(joQ}%RVjrya)gn3~MF&J&+SZjq7yIEP zt6@;lQg7@$CG{$|ncI^3!JNq@3pn-7dOZx5zp`0tciNdkcF&RG7b4#dGY8;;;Lgcz zK8C~~esKAw30@QyCipnk4x2Qte9!oOghg-C!&dR#TFIl6`10DXST1IDIquRPkc812 z@D!u)tUKfqMc*@3*|H)JJ6yo67kw84_GV>4(|Nu5J;PfVtc;z2>_yH%Rl0qJ(~419 z@XPZcb`GijhZ$>=7YA;qXt*TnPPU!`d2e*CO6({vN8&ZE7uy!nN6$8pi>aZ#ySHK8 z&Pm{2acRTRN`VS@VInz^1wf#FibV8=nX5dR60e?XMSX)~t_EpHi zb=eIj0O>&Fyk{75)Ye&WA!H0rV9EvYQ4S7aF^eryCw>Qy1N7tn0%(cGW!Lz z2aM|{1|HeO0;ndA1r89_5uh5Hd)BDgF!Y|#9t%^f5Q3el?y!YE&~M%3KU zw87m;F;!HW1^({d<_QGo!sgArt}VG287Lx4kIDa^?mhTO>i{cNZ65~;t7Xa`NQyiY z-#(q$3W9AJkX)RvcN;N|9h(;hdoKe>T~-6-tRRZ|J_3hK@?5G%JzV0UF=D+}ES4KV zM#qhGMEFAU43jkb2Zpn#&4yCOkl$4Uh^AbDMo-~!f6L{?_cSG$tFTr!kf>us)8Yp# z8P*phG@tN791f9aZnEdOBC{49yWBpkURG;rf|AH=f3ciwGE1d=qq=iG@1UJKGI+c! ziQ6rn>MX}juy@%ZHHUT%`}IQ%_8b)t*tagTTg+X#!2NKD9Jb8GHvZ2}R@vKlJ2z~Z z-dcN>k#)ILD$;tlnt0?hnSY22m34uK;^{>O&#(^-f5n|JsA z?&TCNym#c*Mw#udqVaS%x+lO({90L9h30MaZfaM5IJJjG$fS)P*~6 zBs!YHIEe0(zasp694(fQ}`8FjObKt#3Of zv((k{R$(rqEE;mXGnr673dv*aKhGvPv+y}<21Aqfg4yJBrLZ?4I$f(^mZku`WunRs zTXbn8R8`wX(-s=t;PX{Z*vW99JAf}&!Iu4ogD@G`M`Ep5*Qa`{Qec5`JQ&GML4<`V zP^{uM1_GWC)12XpD;-JZv1U>X{OL*;u@)$eUu4*wd5AjB765H4o)B;MOeU(g80F%7 zz<)ow?BV@;X#zQ*-;IVXD$Ob{(m(LiXk#?THT9vYyz~u|)>J<@Ah14`_jg?8O(CXN z^%n(3L%lylB`+HTvEO2tRxJJc^O8^l)*`UdX)jo0LLZj+<0AEGK}&$RbD{}VLrjrb zT{$5P;z^nNqrq&o?SMgK>!AAat010+(c12u_5Y#nEo18l)-~OjnVFf{v14XtW`>xV z8DeI}n3*AFW{8=Y*^ZeV)2!^BWcEJi%+<`j(&!$k4xT0OJ# z>I56m5mCsSWtq2R3*=eoJwLl@CRG(T#UqPLtOkRr)fIjz-~C2UfFx0WIB%KL0x4d6 zjD>>ZhIjSF&_@pR^LuHOcuy5F%9JicQmQ|bO6KZls^;irYx3!A&GsFg8kzb-_Y{bu z{l#qbl6*edj-Y7%!$Y=YK56yaFQQr8)rf|8&)}P^E}#AL7%hwC^_m- ze{y%-)3_TIOi?=ow?pnNy1DP~+O3187TeJ8vnH;Vw>L^J9}O%DE!#1aD&AOjA!im* zQ59;y|G9I><=ua&0{4}4%fGj__OE4o+Mi%(2L9aa*NA|Y>w4D3#NUv>-$L3wDBfa) zt{VQQWuFMpvTtr#MfK+naF773?sHhCtw8gjsLss zLs`TWAHHSYh`B=`7CvZ%-C;1U7e`H%iA@mgCyxYA39L=D(Ro1dl{OROmPBQnVgD;8 zZU|td-#6HXmoFn*PH1ub;HW(tHHDUv+G;9R6g&Fb+f|j};)UFcRP-#U7j2b8H4LAo z-#xA`z7SV(TaLA2W$+KeRk$CbjA3?yO~UD_56}|s8-Dy~uusZAn_&F$vu=pZkb0iSU-qUlY0pv zH>>b^aaV{+b&SQS=pg@@IJQh3Zgsnk>1oNClx$rfbrhuayi=H44}1TX+08#8j_vA@ zWSaj3k5V^L39!wCB2nzMDQu(9L9k_BeWD`Ow;i;2+#tNn^_)PwlSo%#(JiY}7lZ|D z*mUUPp|inv?PNluDbjjf9t?7#&E2EsD7Ro(?Tk(E6t7=~UKJPi!G!KnlOefBr8@BO zxyKGM{-#`ko!H!~J^A=|$>sE&-639VBeN9W>s@fh?k@T3TztjO zZnG{g&7$d@%Xg~}WA?0|!FJAu_00*h&vjZr=~1-(8nkL1BJpr>XQrPAR9Q|x1tTvh zz5P^gnZc{90+v8h@)v8mjNW9DZmb>L&k=sad&bgPMxS0JO(H&D$4{uq3n?nYVsC!p z;ye!H48j5I>j!F32RKe+s+i#X+PSwK0DvXLPS< z=?Xz^jZUo)gFZC zQu7yRed`q{+A-R}XN+BMf@{2*Ai%AsoxXi%-(}U3ypv5aadU3h-?tcHII2}24CSC| zSNBwV#HDF1i@1a8pJ-_G+XOpJGX@FrO+{s5BL#|p=!`J0%+3pV)+ObZ9B7y&a?n=) zfbM(E*(%?}>b377yq*#1e7EBxFFaQ6m8t-Y_h%Uc^XCF$mvj*N zuvI5lhWdtD;Sqa@Ko?HR3x|CtuvZrO2fAuB_ZP(u6X8mk0MTkGUL|u$F}I}VYpr(F z#{OEOWO3dnl5<^Z@XpabLXFrOHXMItTy2(E*f?=FJc;f(S+7dtKBZ_`S}m8Jd_0|c z><%VE{O2YLA|{o!aF+GlPw6~Kq5KV!%8T*UW!!?fOt>s;j#2h`R2MGYYT^Y6t)9D# zW;tkiIzP5w)avZL6$$FgK)A_;NShbEsN6^ML#B$dMB^ESTX*VX81>0>eUFqwu^LeC zdLv#&l0eaS^3^RxuNf#j#cr4vca$7iY}SR5KeFw8smEj@rv>Hei`G|%(S#h0hB_k6 z5iT*sU>3$Fx)g*qxpPx9rG}>^0JX_thi_s4MzT}TolAIAyrEU$y)~js^1cFghN~@_X71?Wt>^E zF7y5!P?{VJ8+S+}ujegAZ`Oj%0mfq?o%5xLL7nr?PkirZX=+hM6p{xzc0lku5_0pG(2O>M9MQOJc; zfWp*2c7rbjx8%_vXoJknm9-*s*-#h^eOnhlHi(LPRxC+8G286!{ZLpdZbaNY%OYZ1 z*-C(rJNDJiwg$!}sye7Y!s&z?@^09y`RpRxJ0W2D`cwz|IF?|+W-KkHgoxE{f7DF! zC@{E^+0LQ4T-E=Pg9P!N5*y3y$G1ikD$A_xZLugsXZ>TEGr}>eEjjiWn}8>5$>542==^lMS(^9p7~D2 z>rt6OIc$-L1f+DyBEvMbcpM<$c^LoUC)qZqW{Ly}W=9u6hV{;D4 z&eafE%*8Pqi;J0OFO0L^T6Z>4+CI(qCHZ)Pgew+6kz{+`YeTtsoTUAXTMrm+_ zaPCLID%UkmnA99eYUR`bCzr=1={;l3VNO%_4CNsWS6_vbH(VbWW)05YujvlX>jp0* zS3?M)e5!bPS(#*X*Yd%L}b!=&GOvu*l2vtsz-#ADw)ZF zh7-17!y`;+vr;%}u`;}Xe?l4#S8aQEi<~3iE=$dp1yL*-E|9EHkvy>18c@RhMjn zs+&dXW!Fyb^w|k&B|B>9@FAwpBNO6FO4dlcV3_(tw0J>mnT8;ltp%-%OIGH``r$(W zxLVBFLah0Qf`fpANW>tDk&wbyu}BJ%!0(hg=P^X3B8xaqFroyQ93#2AgGD z;c0gu6*pGk>u`i-qs0_BvGu(&2|wTdh*}bUa>+G!`teVO3AjHHR*5as_I z7A~@xf9~(*Wc5z2|Azl_Ehc2P;Guq>2R1Wc_D4nm#H-qh{I-P1?H>J9_z1dMKgtRu zgPa)gP2?<^eUEL8{GW2uWy!1 zI0?>#4tdK^e@4LH5_UyUpZ4O-+s5R*uo@}w#OOGYzGu|sPBhpg}MXiWuM1VK) zg<(jj$#Y#^avT!an+(p}*=~bW{#uwA?*O@#rlI1hgkxIVE8_-M97oXU#V(&C&=jeRPXyx-A^zHMLyA@=!)n+ zYlOMJiu%c5EUnH7wfUn^S@A}16a-uYt5$%$X!FgH9^}WNh=+FtJ|40bXr6K-kCTw> zH}h0lj_S=E?yHNbR^CFY&zF*+spiQo6c+nDQHF_P%6QZVxC7PKmKf0$j{MebRP1~3 zrxQa)a}DpgvXC-&V^3h!b{HXiSg=1TF{T@J@d~DmgThMNABlK)F^TA*y_}q=Pe#@m z%1*Q}6h6EP$0M_f267n81_pbQK9FxT0gcg07t~=`XaO|&rHhXKbMvBk_yDS^ZR3|` z-Mw@%^c&4j(T!<2GUWog62wIopffHd^ zR19t5{5iTY66$p(aApOt;bZ}JqS991{*hNGg9Fd|MP?p`^Qsss;_V>ycr_rb%=ERX zBvc)B8nMVQYxj)jEI0cgmFS#t(i6b;f?RJRtOk#UA&NWU_rPA=F;(Vvu-~{5)Bf(5^EF*V>uU; z{ba*WLRekip=h^&!2HG7G|YdN8}aECy5cq+!86;Sq&RFRe;PRy;XbghehjQU`A%!L z1ncsU#AkweMfCFkAQhrS^|w&91;J1%5Oa)U;@OrafD9~U?Wt#;yWwEskJgR* zeer0HP$}mlY(*aRQ_eMpbjedkqxAxf8qBqe>QL>v@2tXAxPf;R7sLToQCpvta&jbk zs$^i8_0209OgjFNaixK7IN=IW+#n%UQ^@GeLZ+DFX-jqF9kl3)j z(PT9#hs!TJcu{6Mz8p^3FQID$6NDI-#_~fhSeP=3*qn}4Ny$FH1R>%?hLLFs1TVw7 zb|LMHmG;)QVL+0nEQzy)=hD991FL>4r}<3`1l0V7!mQM~o%UUI?8{y6F=O$wMP5l$!IsU`7To+_Mc;J)8g} z2Y0M=yPISODb&v93~%9WGP~$-VZW|#170KVqAb;Y)wM-v*zbl!W2V7Sx zg+4nlV7El|;%H<|JG^l>om1*LwX?wf(v5mSp}%A!JRGR8;|`P_`)}XaF#I!DC>gMf zwYQbpu+&mFt$)(-PWUjXl7Aa(M7cl_jFyg$6@@{uGa5HaJqp`)>BhCh0gbul+|C-Om&S% zOxc^%o~%;7&4dG}b$GVp7V9qHU=HDc<(A(hW&^-;d z=qY5%nTFi)ar|GL$qz=z|DYIQ^)S=B_&pIEY{X;k+nA`R!)_j12i{spbIKi)zCOh< z8BNP9Dm^I)2RuYCeO{cm@6sHNi6vUq5wb*PHkk%-kTNmR*laWQ*VWPHmC>n{>pOk3 z-r%Hcz2_tnzzYjuq=JV}lxd_hwNz%Bsm2V0*YcWfkf~~%!(nIdgW{`nhpxNMORPPP!wuS+uqmnAWzi;h z4NjIjG9{A6l|;zXWZGge;ZRJL}e^n~*sXIG-Wk`=>;+vtvKMFo~fcAMWarzm1U} z3DEms$k|u%q=0cl?;Vb4s67cndNbz1jEA0m7*(^+ZB&y^xC;1?>iQX*U5s^&U^}1m z0uXRW_Hz|R0`{lS)&pmii=Q1$`IwVBosNev2B6|x8uch3$#P*L^ z#I63H69y@y3FUxC9`)-Tcw=e`=*-ek2@5|~W|=xfQ$qZ%)Z&;5Qla$xTdCC{IamyY zbw|##f_CI%ArKsDx10sS9Dli20@(r5c5`K>OoAVhem8i|(TLgA3yzcx8+^AAVWw3L zpJ%qyC%pc>Na-DCC$C#9rZK0g^}SLP_|>+y$XuAU{(Jx>BOm34)-|XpeN|ytwtu!J z&IDh|aqhqf;TO9oTj8)}INbJ+Mr=buLlgOu-rK;55x+~8%ad|20W5;3bPjfG?Tz*{ z9B0A_VhrR#x%bpU5q?FwlgI;JK2SDIGZqWZA_H(kxz5mjfPfqS+q1!{orXP#=}?Oz zJw$un+k=xlMTpp_ZoNyQNCxThm@7(v7O!{RdnMTA^wZy*!PdWEHpS zf{nwn=H8)ICz1_Y2;}mVz{!{m#K8G;?upR8nz$W!Obm@VJd^Ke;iMFRs@`rlsP<#= z2*zmyq@L^?TVER7Y#}!NL%0$#aDkueO6;Ai_|`?yy*Tdo5&t<&ExdDuxzE2UDZQ;W z$QMqmWSs%V&r$P4=B38Pn^!mvXI+|d;lj_?@`a@$-+;!#!pwj}ymgd;7OkGu+Lw&lC&N68Q)FSBC z=CjnYBC~<(Y@;x$)>L5NG#HyYQ4(;H!ca&YmhTPMB5dTx?y*Z2c7*I*OOpv_{LVwq zjmloa{T*5DfJJgVngRxJ zo4N433PeUN_6Ef--?M~sa3~1ndQDEty`W8e8^PPMVME-aMFUM4D{Zp2SxTO^A*od_ zdP3QyObhmRTLKP>iC*?_aq-at`{E&w!58|e6vt>;7|7~t5q3#xz>!G6?RX^ElE`#>qAK z!_a%?_IX9612=p5=au3Vo@eLHS(dZekV0`k(BDV*mk*gpFu2@=K*;|*j?96bL_N~k zS4sY~)DZHU_5(v8{O#nC{vk6ii+tk#ZwoEt6jp6_56JH^2>r9j`ArLo!|Hhcv;e^X zy8vp?6w)C``GJ2QQ&7N-8#l%e*6oedu7Vh<#~*+^8N!bJDACbChi4mtfzI|d?;YwJ zaS@`)Pt~8|U#-AwW|#8%xc?3sf|B!b$FAS%e_jqc5GP6F5o65X>)QrBie$%S!uflF zH6nf!p>&*Mf3MGu9nc3}Yz+MG$NOTp4nAu-ACvj!Tx+Cn?fXH8VW1gWtE%>k)+p>4DVbSMQ~NSQjFMwQRx?5zIr zW1xY-!JoX_x6r;qH7Hq1?~?+%fBndK7hus#2n6E4vuJd=()PJvS^_vFoH_1^gwI$! zdx=eF`*8GF^xrr2+AawvcRfjBb#1?559z7<|m4&63!@ZhJmfg3mp-H8`!~B zIerztQz(1LHFwlnj4pxUdFQ1LQXK2?C`}bKXp~;N%Xy_sCd+NW8sZn{4IE6@3h2oN zZ?LBYH{&k%{)#|wGMYYn-GW=d#&*KFcgSX z#RX02geug-VmE~NEUg=JQAE$?hDXI23#8nku-uDiG<2+y7KXxAVB^mBSz*$^Sc9q4 z!>xox==xRyH3r*DKvC!FomPbvl90b2+(lTx-T9Wx=nsu5G<&!M-@l&}i?TkR{RCy1 zXU_q}heH^U^I`tbgCNu2Z{Jh%OhDJ-GpSP#v3NNVpXr8_fAtK-+uZ*H7^A_+wbUCb zJ36W_B{gg)y9I{L)+I%IGXBPU0{tj9a2ONpWuGYZapAc;dYpaMB1teP0Aw%+)gJTf zjg;9`(9q35iM-)%2s&Fc=~PEwxkfbEz1mn+ocs)wCOhF{tj- z$Dchq8Wjl`c0xBw7^*7fE@X5pVr%nXg5M-HT8XOETV?dztF*H_eEG&k?TTeZdu9Gcy05&FB%bAh z#8JWD7gP?uyzjeGhOp|8vd%`Uuaqc-)i(Jur5Wxurd{-vk&xTTxmUKvw&{s;d#G00ad2 z^$9o7F#r!@gwA<4?iq3FQ@DK%q9+Yb&Z2J5+gy-3nfIog5m zvn%L9Xk}LaosB< z9TTAii-@~~33(5Z?#f=-!W}1x1{yrR^Q54UP^QA{a0#jhAUz%1i)yWl#XdgQ309!a{Tie&UhnNeqF(Uw{EV~HMs<5;9ZHMMhY0lg93fX?XCpjS-{82Q9KuI0 z)6HhgJ90~4=uz_sa+7Q@G6O11Ob}Fj6xCdBP|6Jz%l`}&S}ND{!|kw1vi+4%hxpM( zc0gSO5%>#*;q`>6-<(p(cg>&?ly1L`B+AW{AP=PK)n}xUq+)w3`-&2@AT5e(I~E$X zei(ZPPyDK*m1++yGQ8IDjmv7Kyzvl5c^`(su%&LDBsT_f^;1V6Hj5<&=%WTZOO&-v^CsK{6HXvCmbRS;`?%-4nq`j{$Rj1F}2PcXG z4t)04wNHU#XDp-^Kd&KP<2#~_YIqYy++m}E=D#nHw!qX zdj-DZL5$cVd7Mvo>O;9A(+~;p$12}Y106eI(gxfb=J=Sj{AZjt+|2Nk#UdQ+v};K+ z#V(vMiM}Sx8{%DsW=A5g&l^jM?I1UUll|S!gB69xA(wh6AofDEvHY~%YPOgNU0{Qc zRoA4jC<+PX+c;n9zv;NqJJEx85+woUB{9e$vba5?H%-Wo0Du1!&f&+T{?$U#LO_ln(jD70EjJv~ujg-|>-|8N$f;^5O zCSU7;Alg4Zv~IF!Mn^D-$HE7ww&Swe`LEIcONd-_jUtj@q(5F48jnG^G1Y|Fm% z>5tLiN)B3a-6LB+v#2b&Evbw;Rh^gYj~Lb!#)fZa#I|UFXV$PhW3gZb)L&)pS6(~G zmkUQrgKrigm8ljH=?mCuQ)$uTZ>sdk(bsMBNZif)Wy5|m0jgwistk<2cCFlS*3rro zg==W`7B_L!Ub6||wym=Z!aUoDRz2aspGG%sB}b52Q1|?=6(4d-3=|Y zB!r^(aS;-@9MRSpO+fou&mI-zk{p2MV5x7a*w^f6xmk%n%xn@6W69~@6dZjW zb>MDpj5=Gc_RKD_w^p6Y`qOm_9x>;m14K@p#2=9KUj^XYnn@4luN(Q-(TxXy-aUAJ z(}cgc+5|O))iVBt>2Dt{{_gWx7FT|XPZEEKcFP3RszZ4G+*8y_#pR={>>{3Lj37s5 zuWS{fJ7Ro)4tY%g}2@wIU(zf6oC=0YL5T(B*$l0`S+54nX5p z(+2o|Cg#@%n1(&O;itb8tsem(crB+?vA>hB1N6a-PKEn@EgztkhylZ*;~o6^_w8?_Zpc@@8?1HE#dvV;R=d)@*u z(FjbgIa%WmrWDW>!kns=5cMljzT-+CUESK@%2-kKfI8%o*8>{q%m;XI@od)Y{F9OHsL};kMLftgcQ=aZi^wMBdNKf7k#%=Q^&A*nL}O3g7A z`9aN&{D{y|1?U?^Kat&`Lci_{IFZ_-!9%>2nO1nM!nvRgG5p}-Zmn%s-^&a_t|k-Q z^R-0~C>peB$nLGqacGNrgt;$nXnbu~Z`Zt{D%YxYX-MWSJqL+*NFb0i2IFn1SK*)1Lv9CTU^G4aKG$Xfh6zmQrzx?q(sc3K)&vfbJ}~GO+i*G7~=U;$WiShJFx%^~t*aFwy@GhCObc!3imB<%RA#y5D@Ue23}Z zLHXW_zOdh*b@vHoT2dOE!5Kq z9G=z}TFa%M#IOhK)Y~1~r_2pKM*I%78pq-$0+dsb-*dZ+(g&GFu%kwYYCf_*`rCTy zCK3_tsu5^?6q_GVry1|E+xs2`WM3G*Wi}iK2GF{U{Wj-MgITvDR(4G0L&a&b`j$ct z2x%YKj`aF7R(4_S^al9Gk#*#vW(ud8()f2fbCXUeYv(lemPmfS1OqQ5MvV;Tn9QC^ zzxxJDMF?!?$0O!iwJ;V>{17Rd-v=0GY@$irO)Vd^i^0X3h! z@e}ylAaBtN6SeFfP6Sz4l#LeCBh(RFu~k}K=1p?5$O&bE9;m?`dymb}CswkSVvau^ z3*0)ZzG{mw^h|q7s(*zK6~Q=7Zu8!Gxq3zDv&-Bf$uzcqenUb^k#v9l8OqU-c_sUi!(GVh& zJA#~UbT6g8Hbk2n`@B?kUvJl2T+Ir6uDCDCgl?&l^GeD)n;IL>{8pknJ)yZ09z z7PcJzM}prC^ds#A>w9+*>s%SGnpSaPX})%d2tw?!E?Q68vT>5Is`GhK--n#>@3(*- z-LchJ63KtA3hI6FvqwH6^>17t4k~GTNoND*q)M5`70jvbYPNoD9dw$R%c$(J52+W< zt>Vv->;AA6Qf~OM8xyc{;kd-5*RBJk|9&KlrU=KkVP5a++u+#1GWDWZ>dxtQUC|EP zM7a0G)`v#~1T-3YPy5;`_j%zc4|{~~v6n(*9e792U^b zOs;K5YITY;Goja{vJ2NI4OmdVTw?mS^&)vY)TE!2`16$v^hP)-YfVgb8mRj0yqtfdhf6#FGl}=+C zwDR?E+_`+J1fPhV2>4+~s0CH&=C=2To}yP)NQUJmdcm01H5n-SN)EIA$$iLV=Sp%* zM?^MGlrVp0&u;B>{h6t_M;D;6-^`pQKHt z0r{#k?g~aR@;E#wlt3#yv5v-WzuVz)esAZn&y<(|M!{uQh_XU^y%G7O2MX|lCs9&% zJ;Ug5GEzh@W~in?Z;0ABX|m^jC@MamJBj#abHjPe@0E-n@Xng^w9wXgm)Eg!umfLe z`kwVAao{<V*KWWV4^+I z$7ruvuL{o|lMlrwCUHU)0K3_baOJBT!?YISP_D||%VnIty{@l#Fg!_p`FLv(?69zp z^=^MhKm+JXhVqo`X01q{%wy_-&G~ zpVHc+YE3j|?1Zzd`H#OBd}Yf)rueWN^QxJRvoaNMZvhg{MP^>L6Ye%OqUV!x9EPa1 zz6kC0=ImLcyIf{8l%eZ38<*a)LI!_Kwt!M}_6qgDw&-)*po}##`2$B?k%#xiF;NiVC zS96N~`%&Et4X{&HhEykiAj3uy;G(`_C$AI#?xRKoc)F|5ittGMQ-gKO{6Yk!zHeH; zpYB`K0J>=oFD#+>GbR;K+Xa^WtWVvakMINYh<7jGtZQ{DRVHaRQa4eL1=^75Uyw1a zucSgilE^JGGygZYaq>kXL@X`+K$7JOu}NXjtU0PhJ)<<}&-Y{zn=GNE;0Pr0aV^qI z8!)-Kx;t#B3JP_I`)EPZWMTnz@IfoKoxOAJ7y5p;8aEteHk{=}bLA9}+U(bWUng+o z)Q#tjn6X!#MX7An!}&K&XB(7%=YXA9Ovko>!OZ)evb)kmnwpF^d!F=Jtj=D!)MCQl z*}aGuFWTul7Kc(Z#_PFxKL(ECRFm(AdOVxXOf;uW!% zsflvHa6qwV?8J}pe#SdU@qwHVH(yFoqgheTwR2&Y-Asy3jc7={AM&TfdZu~NYTc+K znr!`c@7TH*d5eb(KBvCt6Cc8T)KxnvI*Q`Z9s9kcg6R~@Y#X<+`{-x5NRk;;(q zyljEk#EvWrhX+N)(j~`XD4+d31_6=D71`(h2FLAo&)4Xjb4z}o19V}}@OG{g(x(Cl zCVLZn;J&HW@V)wKDgfFFI%y>>R4x7MZnY`$lnRmg47G3(hLK0CDY0FUt|gA`2N17m zuWwgx-{la4QL1t{ot-}XI})`=AF&^4r8{=-4NaAoS}@dVZLJ1lYVF%VDXpP!E-8=t zK9UgopsbuQ*YYF!`;hlN66-VDT`ys+L7VBdk>{R8vSbsiXta9|u#9vWBq1*)w?O6t z3!{9@7UEW{+2!XjM2GJwdkwL>2n>TnUcd9yuWzmbqu1{Ar)jt&c7amaeDjMH!^r9M z0QZ6z9pJ}{0ou`f1O+pOpKkVkHKaNuwtJIL##D);)MlJ$LCyb^4BG#8))C&5VMkl) zb{_W|neMcK*JmC7A`^(q=z8b!_RvGbDxqvBPOFy(j&SNZ0g{*^m93tAb)b$rdS%md zj?^lPs@AdP>2Tf^2+~wn*JEI~gb-*HJB|D3^o0dc(Q_CbHXk1>d?w8&{ST$VhW5}i zDZmjXkwe{nA7CTjKTs5MIXDfrDPTwO#gl@^P~`OVSQ}!|@dEL&x?kWU3~&I2-YmL` zO4UcuX;!VrRNv913)7+Mb;~C2W5%cY>IKSzmWIjWh<0#*n*SC) z+@?TVWeqohu6$UvHO<;WT#5fV#KmAc%UQkkwYii483G)c5pl{ZBIAsI<`i<)f#Ewl zg@5OpPq%Z_aX~4ER=}$7{6y);1_xY!ULX)Zf~xTG4@PDc^o98=P-_6=6`>OOzoxH3 z|Nlq&LJDCg`KA!3PMH`cw;XVj$Iw7VU~pSdy5w>VU9b6FK|%j?2?B{y$w+U-V=p3c zgLi}dPJmJ{q&na_7&bhcht|eD;fq}#tga?6{Chs|iOHv6ZXb7QAf0t{D-4^{$ste= zYT%S(*GuDLsAk`7SIa&$yrsyqCmH{b}4pRzEO&R?nH{YK+Z`qy0a(NQ8v5t(#2zMhSg)c`3S>?X7LA3 z_?;!O?I97fgq}%>lWZ5$N@{2A6lJ0FkMoong|~CDZjYF%#KH)Bv`U6f!w%_D+-oa| zg2QMUgg2Q@UmMNYLlW<4u*~&AbB4B1?Z)o3Nw#A>+@Hd;6bin1x|U50xCoU<^Jg+M z#hy=}ypxa1)7dvDWzozPex4NF=^enc}zt2O0gC=I5ZGl|9TXoy=k!5bCEo1FWqeDDN-~A)HjLnvxtZL4f@7{880sCwoZz@wg z_<*`96r%EF-!&}@vbK=Z-3B%IQ7ov~V#_GdmC=UNeKTtgd|a8O+Uv9B zNuq~H_C~2h*}sBC_7_hjur6c#c0qOm0BF;mGE+|bt(exr0Di^w8~L-~?>_Y60BD23 zj-pQZr-Je00z`~EvJ1v~zca>euwT$7kgxFh&zKay0@EGY%+G&5;s<&T3#7%MVErW9 zpo+axwpPZ7(wgMgk<#yEX@pN0$*nUwT%TgOJUYDsNPN?4jP|AO>(H8IjQ)?4%>P@V zYL3*z@Y5G#;mUCl1y$hW&noQN%Z=FbG_za#Xc6Ae@dLKlbJl2v>8nU3D=ES9zui*Z zLjGR+#M#fIXuOft4F6IBJF`iu=EOIq-N#BurY!sBB7uConanP z*7W;7`eM^m4Hb^8?mYlXiNtZ)vw2D}STsydSl1_PBrnU!u98w_E;B0KCe^KNFEWE_ zEyCx4WhVlT>S&cOCn=97ju7X%#_WLB0|8#3KoCS$f=A#LzA2Zwdl-v`iLDqG4sL$w zlO5h5!+Kby9;J#J={b{@j-Ja}g zY@BK17XvamHzO#8TJcf!O(!tips_E=dsW$=6H$`}`5>_8?Q>eh8k1lDF?`mpR@1l{2C)yjboyEX7JQ}#oS1UkoQtgXU0)6L644qli!Ny5rpi_ufnUjEW=YjXm_q0OBJ?cj=lkI< zIxP`0!|1)nE@&L(;e^0>>dPv1B>p`)U^nqnOnlsSmxXfW_7Rt=Vu>cK1>a{~D@eTT@JSF4@0CnZs^B zMHq}!xmap-3ve=#unb-I^tbO-Qc*c^fq+1+HVH6<%6S3nvI&1@nxupuerKB2G;&8? z5?(&17uHj6qBo-Iv$i+6L)#|L^h@WsE4`bt;2W*dAKr$G|;y^7(**k|F!u0^7p{=P=*rzp{cFh5PpL!8ib zmju;j-0na+mYtwgvMf}h3mew2=Sp`6)Ga(Ft*nO{VJ3XxJko#2B3J%3i)4hU{+mU* zMR3XgGmDfB)84vuTt#Fyg$8c5`*}8)6bDWkLlARp>jO0$aCO-{4NeB~6TUa!(vQwD zL4h#qeY%vo#=wq}oD4k_9$aJ6G9o6{tai-U%wI1{bc^Ca4pGWFq91OCb?R7oc(on= zNj=mY!DXAFXO{3=7vNi_m(kyf6cK#?k11l7sMx_Pl$$l|`PVL}h|%l*!{$fS?OdkrVWYd$kG|Z!Y+<(`gY9}rMUY`gsrIiz zus?Q=n{d=bgSY2}j#GKf!GRaBJlpMdeO|H=vApz3g-={bn9Lv1jYN&Pp{aFrHD<)4P16msZ7w*)xcMl3NPdE$Iu_%#YM(en9A4AOH!P&%Nu59 zU<$2_wMvWpHOW|R5(E|mIifV_@VjpP+dl`#8q-Wl3fW|@alF~Y49p4q3jN%&7Uk%1 zb(PBQAl)@hcOyCKQ$A|$PaOAO=?8VNBj}PHg>YIYo1kiKsrUj#| zihts#SU(740Au)znx_15R%w9z$)4O&{tr@J@&k~Hd+O+v`A=kp0`OoH#&9QpkP817 zQbAy6>X85TV5YxPuCxjPoj*tg0=48l84AH{@aHUkpsi_sl_@1FfL+fjN7gGS1{w%| zIWD^Jn<;pp>w1vk@88TBC_WD3sfO%s!{g*cjQ_VoC@0P3m3S*-w7Jlq)CFNp^<)>L z1GHCPbCq2+$!S{I{KMCb#cgjBcynvbOJYIRivSN)4{bHLjCj|d;cyu=cD8*U1BWZvjM{bV#QN>cy8^|fO|U2k|;`yQXZJ~3Be2PG~=FYj9k z1)lbws#b%f*7rt ztGn8pSbZcPGJ`>K@F=akhUp(XrSARku5awdY{3)$#nM^zUtp@Lkh8)0!@toKJAbEG zs4^O%rywq{E569-fGwhfGoA~>_u$5;2}>0fN@>Wx9yPh1A=>x<@AYSS$(3Tgw7Y1Q zbIdn1s%Q=gDQQ(EDTQYNd(tY#WW=(HzWZ*pt7#!KKVX(uP_#E0uinli$}-MQ5m&2ifS{Tp`j38U-d+A5m1W9f^+qTJehc;2xxo&vtC+VMXuU&lpk4(1 zl~>=z|Lg^T;;a&?@s08q_e4Y~nFVLh6oD7Ck6BD7xK@DWh8?uT<@gDm z7_9JW_43CMaZW1RIN^^ZnO)9Bl7(wT;HY4l4|>yD>q+2OnK8Z*tRrH^iqHX$NS8x9 zw}+Qsf!<0Ux3mgou7GUh*$yol-Z&9y(1K_5jrjQ}^X9Qu8aNmHfufP)uU zHRarxaSX3d)YG~Lzn*JF(Tc#7&8|Ph6W@h;Y@$>!0~7!KI|x4NS~|90q;#l4c*w0H(6)1e*@X_TBc) z11h{mDlYrxQXPq`T41!=p5er+W6GOE%rLB=i5zjz)HBEtrcXdgZ1)oKPA*j3Gay-9 zEvu(I$811mjjZ|sa%*qsyti!!6N^f_*-QAj@#UwP7KtOitN)j?HmJN9EpS{Yf(%B; z!4YJlgS?)gBj8~~{F}pjSBBhHzBW?fa>6FMoookV&Bc6#7N9pwB3@srqO+c@w^Cj% zQ-nhq=zP9On;413ip3mFMBh zC8ahYdT^i2_<>atEkxv=a)&U(*!+*YlOHY$l2Vz?;^ic_)4ER#9kLgDkwG~L6mSnJ zqkB>Fv@~yETDeLw4=~)p=g=!}jbhBp)(*m4ZFig@jG-`1g=S|GMDaQY(RL(?rk`73 zm6j20uEyU2>B#SThaiPTbO`IXP3Iam!V&_ZFv5xB@!2zzO(ZF4DrR zhp53XZhC_=O&9mC^H>GEp$YfwfQCrvRV>ohk4b)4^3(%6g87Ux3(QohPKNz^PzRH* z_{`)L{gqb`A{&kvRZ-eNG;4v?bkD@icwkgA0H*v71~IAr;qeU?Z#XN8;>{JPQwlBq zX1c@>LnX!}9WO{M$w0lOf4EV0EgrdH*1r1nYBuu?h^2CxM0S639%k)zRH!>Lg#R zG-!@5zI9fv_w%7O-!Kd%*9~Yij&a>JNp~_1>ysUq_vFk607wI9Rkz$;_ttG0zcz|Y zj8lzo*w$XAoGf@f$|iIhMykT2ZV~WvI`IMD$D;V3i$&>A-A0GOMeMCimzW=v@@@m-+GgQHcC)$n}oNN?ml_EXo|%k0tGCp?-Ku z%7kz8)C>FZoB}!2S}l)_z&QlZZ@D%1K^z$T`hPA*pQ+l|_yJb~oHef##(bOBdgRH0 zQTo}0ay~O@B$kv-OZ$@gI>zkm?GSeS$GsXmy!#rj@=MKGVJ=r~nSemIOXt9tF z7N#5BV|ku*W0rn4_oER(8(P>aOxVPft5p(7kkWGx#VZwqJW^msB;H3J|VRdpizX|eO%L2OK@Y{_6ds} zx@vzuEm3c73CfIak=RifVt?O~7Qo&firHDj|C+PU&N82Q(Ny^NoReiS&{F@uNi~8p z6;cZ?Z>WnVy2D64T*g2`)oL6{8yMJ^5zG${qK?f;2082@B%7fkT^@7fVJAPRQC z<1c;hWQEt+FT?XX{<+z`?pGX7%l;jwR)3fs3W*`wMRKs-w$?c)H`|U(nFZm zBI9h7tQU}Gony)0$>%ri9E00$xjI=!sFMTPq$(DK-~>_=1WG1|>ik4#WmyRx-}ovI ze@e&hh4~@apa-{S_#@p=x36zVNQ)NI!!&gekMSTqBkLK83!3V-@69Vj_>lYny+RQG zR7zx^grNB|+w)@$6y?4wbAqq&EJq)G7f+%ep7as!g+mS=x;f#Fb)CWn<@0*TbPqWL zXpndahbadtQc?K5q5Y%d{REJ&veyNzqB&Jj-KQz4IYj0#{$5|OTj7Z|-8{cZiK2^* znG8^JeiO|aKyV4H(^@KyG_MjPI^6N_Zd==9#))no8!zH~Ln|&D^!i)+*TW&0HJdmLEhynzQ0f3W+`45GN<7Qo{ zx1570Y3q{S$6}HyIaO}w%m$Q zu}L8^0`7Ydh#S_%*)rU-;sZX@-9dLeOphDJm6+DIEr}g7rR-i{MvdL7w68$?TM=Mj zdNZS5M9u8&0|?3X0*wu`#wV0rlHC~oM{!j^fj`LVRYLwqXl}}Zd4xNzSMFrG9{hi$ z)eT~ET**f7jB|DRk8gdxI_#O_Y2i^@h;A)}^59L3qF)MStY!sMf93&pHBTSxt*rHA z6#S4<)vS^|3Mm#M5=&v-3b(2fdS~zjK7KG#lPt=25wi)OikV(4!I%heHwJ+K$|m3o zPaeWnlcUR|MUOgFt^?eU$!FicOLy}^T_YfA%~Moju@x7bV*g$8p#@w0FXHDlA)mtj z1Rsc$e*vG!Gc9aEq^#xQ?$!{@7&QgZy-uTX31r8wg7ZW?tq?z$*s;O4v57KEL2x00 z4HWaPB?=D;)j>x2E6}1Loxe9Q)eq3$krN5e{gRIDx&l8zHI)3E7+zrDX*c03XSwK5 z;8~}C4*0^(6Zl7ZT#HM?W=cjK@Qwsm_$snF|x z0xE9b321PW&z}EayO!sYla5c+%?ys=N}YFqoG|)AmgL2c@m#IIAWj_Y4+?7C43V&K z-WUx*x*Z;atU11NOyTlqR%g=pjD#KCn+ZDrwjFta zzP%sU4DMzz?M> zRQbCHPI)1HyHTfW69&yrM<`h--DLaM2-&oZA(7bX|x$vK4pKrM8 zs6`Ttb<^z*GlK^H(bsw|{97*}gmo_CR88DR1(-dW=7)ibY_(d-sOPX6Z7A@LKH;WA z;9)$hOs3$Qry)YHkiz+@J3($U8gzAY+C|8&QTI}n(8A3AQJ1!rdz@W34N?(gU8;s7 zH~;}eY>;!-oc?G1SmsRFP7jQGB+jBu$f44|3?3%q4z!ItKv()kOC|Hyz~BKR zfb2ZrSaE+Ruu#mx{nm=imrMIu{&epE&+q{1-K=YPLP63>fO)g{2WUNu)KiN6a_ zjU3Tg6FRo#^hQQ>o`{&5qtld^p~GWRHGZwrqD956Ka_e7NdwWv7H3nX3`f=!}L zY))d7N8b=h&7r^@Ejg_l?xJX+)o>rlPFOLh%%AoqB{jL5qDZadCbls^72B{Eif_< zr+CBgiVu71V9FBzM7WYALp4l&y#u(_4n}C0JBUs4j?@;K}F6ivePQ0nk*N<#nN7wCe7+ z3TVlgq|PTU~D;CW;Sr)trPU(D3A|= zu?GmZDw+%!9|?S?*-tZ0ZOtcv<3VMpF*^Dy{cY3pc0-EG2Cihs7@o6UP1<0%0s<~y zJj?o3!{Ji-lSmy85uSBZcRfNrJ#p6NGb7pGT1TYO6xth>ZaqDfqf^qZg`^f7Zr{gg z$o3j7xtGOA`q}e{&>3Oj1uE^_IeIkPCPs{;0fP%$;6SmAH zj*Z8e`njLx3ld|KA}vJa(V2y`dM*#CD|}=G?dn$cxj~pk!t9Bp-1fKE$xUS}e_U`y z{bCoXoos%Z|FJ-xGHh~T2V%W$w!0qEpEOfqJOrdFf&tiMHFEi&Ls)4b}|QfuhEw;efad4xSG5_ogOd84DJU5 zFA(6l4y)tkLpw7aXup0TttS+ybOzrI`9xqZ^7k{8owAJ8ysT7AN{oR=tPJ&C?6(Lv z^Le9)6^2w~3}Z-r)Wi!9etRw$?YuJ(W2tsHGBbn0y|^Ov3MwS~D9_o+n?&QeDvx)2 zd$^!)wFDFmNX4*nyI7KeLq%k(hN&|{lq zc*uLb)+x&f;v#W}p-~|!N#lQ39Qd06(CRWPc9NSU4vX-Tqly$i)-f}#d%QKf$s)~2%o`es+rc2hblo8SnJ~Y?otO2K5gW!1B?D*+|=c!=B-6Z1@F^M z2J?&}GriL1K z2EGTOYCb#({tiODr`N-AqAY#~q1phmc-wSIM*P2BJJI(bltQ0^_3t1QCWs4QfG;_! zmhd048Q`yL6u`#ITJo+asTg&wl+OBI3qUKL0HM2^IG(m>V|j@@z>Hb#aedF_CRd|{ zw9@};bOn>}54N%_U?oc-TklL#&9lf{tVBbf5qa75Zh_#$mU3>hsEP4J`Uh3Ttz>?m zBVyLa_{RPynO*!HBN%$elzX1p51!%1-x7i-o zBNPn(-P#jw(D=a$ZKkZ!dk8f!=PY&2d@Qv7xTT(OMHDv0nv>g#gnCQgd%+GnF2!(U z1;&zm9MeJ!>iDYxqtS_tw?uvl$xJzOurt}<;^$pup8OQcp-)9IObjnEf%h7PL3qAGfnJoT;N{RXo95Ra_4H{0h;4hL=(e+G&?K zG)}SzDV46a>@7_y$>ckWW1ex9!20A0ESwz%fkT&f*3WmMs>5=m(SSIX7ok>Jhe2i# z-4aGc7H2p>6}C0%g25DSk?!D3IieZ;Nw~Eb7bmQxaiv)9{HWkt(v}OTJ6NPY7MFzpPIiTZ8SHj>wf-RC=YOxKPuiR z(#pdD=89}0aE>CA=;b4=aMY}?FT_%PO0cxTA_J|BG|An$fWLGvmXeTNaDtVQ;$o*a1fk$f{g~w9r z3Fs!@Dz;2%OQrm4EqKTJgvA_wq{A}#Xq{sDJCv>&3N8)ZMVgCMKH=p{$D2peyj9Gg zhZV{?m*8U3{r%-y>pYo+&elja+a{Zcr8-@}WZ%~d41K>X^@`PSG?6?GXHK*pnD*)h zbv*2V_woltkbmud*!cqJe(>)es&Vz@e6@V!Se^BS>h-1D1JZ_M)VpY=ch6a!{&~FM zQCZgec=5>!nbhnjY_vM{fD;7~5#grobZ;qbrhs_qwBsKHg>$?>eVnDV!KsNt9RbkD zHdhd=0RUBi5gykHq0L80_21S(81(f4yg|>uyunRatT`u0u_~Kp$Gr4RWKwiOLg;Aw zYHY=RUeY_NXg<;!xfZlQ_;f(}rAs33kMGN_#6*Q+|2 zGv^T*X4<2H@50;p9dvuZPsa@@B%kj+jz1|}AZo+l0Pnqx4}M_}(zDfICBWx}qm&lw zuEXfe$ffJz$?$8ox6pXI{DIPfoT%F#p%)xsfM_~?2Dl>u3VGIn1N>3dh$doY8d4e` zj}f@YnXXf5@Bq4jje7^~x*^G@%zBu4p~jvmb^Kso1kQ+DePn~CJTzT?U!YdhA`v>f z#};d-RAzACnF+E?I0Oys*vPb?z~P~A^QWAd5`_!?^E4a1#KI7YP{G=6AG!1r(@d)TaxmlW7r)KO7qiH--83lnrBMj7ajDuqM+ zi%JQkJrjfGDK6p1nGe8fErc5|wjVf(wNzM#e%wF>DcaqeG+BVt&;knPrg{$I^zU3os?hyl(qi=vNYd^79h*AZlm8^A z{Y#2UZdo1uJwizOz>ek*NNwGe;PQ91°1i0Pj9m$1Sg;4Vjaak~&H5YwNN~UYJ;A+Fmo0pT zZEuxKnfbfDpgZ$DDn(&gOaCVhZGL%=qvw1t{vDNSeve9>8Oh%Mb{@6gokvU-9F^Zu zDcAR?RA%`|=Wpi`_C1gFMJ#{x_ci6fPC5anaE<*w1qgZ-8>CSu&*JrHmaU;qrDXy} z=7bI8RtII)8s0sG=*o#!cvh#~hnrH!6{SX-ajIE%N_F)z<%(fVw{*pY#(p z4!MC1Q&a<5JxS%esIiI!s;!yydUCkNHoI`iSsst7D<+8hu`qI?szV!htELgTh#+9| z~7^zibW6y%+CFX zEP7w0mTHXty(0J~KZv{tm7S|!wlP~YPn4gS3Ry)?)}2Z7 z#I`Rq{he>F2|BtVoWz|@e&ITS7o5H zyNgPrMvgSI&+xPC{Tz``%a&HELge$QCxfvXF$^jKvBAEKRgdBcEuB~&#gswM9L=-u z>rT5WpTWaz+yMUcTGGq%<523+!)fG4Vl0o;1C7!Ts z(=(85!T6c*d$y;lDY-!j&qIV^@)Yhxswo8QNBAuM!&o5Q5+>D}Ge~PXrvG=u^g-dO zo}L)z&*ejs&A*EhJk~AkpuARc^vYIg^}{@cAk!@#5Pe1ZZH2TxldIB^PEQ$O>4M2X z$PsV{XaS#EE`F%bj0N;@)m!BMqmC=o9W0uMSG<{6U?z9N;IY|01+z^EZ!cF0KZKJT zgM{A(aWWijo}Uk;Qj*6cD!vJ22yBY{4;!(vDO506y1~@$rBWic4N^Q>S~?s~PEw#8 zccTnz9D3+uq$6Yk$`hkEId4d(YmRx7g`(XxAXrrhYVcjYZ70j4yM zzNeY+%F2r^cWUK0r-A_gdq52?wH!QO;!+p~aE@>mi%pTWk5b(k?n#1q|FwV<$j$kNC$vhKMVKC$I*FsyLPIvRt^x##80 zm1sd-BNBjkd4I+LwaV>$X2GAAi#f-n8KX9>Tw;IK9ObWli)6(tO04Qi$-7c7;Pw1m zAZP;n1(}0M+eF*JzNFB6q5rGqk}wmG0Z0Ar61ekrH$Xqqdi;Tx0K{_@9yu&8DbmS@ z9{shUVP;;{d@$CJ@Nw-&a06mNw_OjM(tv`MRx4tPJz+91&-FHxr+T5!CAd$zC&7L? zX3dW40AOEbEW(3a7$w>!6s*@-cM@Oz^>wW;9*$(|-o#3;K86gVo%}r&^G^{M(}UiP-hce7^$J@W3=TOrB(n162e#|!%qC5W8 zw;Sv02tjOo*Loy1yos`r^?+J!@b0<-7rc9AZbLrzA%4n-ZOi~Pifu02N( zX*HwL@ElhEh$8&Lghkmp0r*YD?v+d9s;*@o^GgK|DMHBxrUs_wZ7Vzv1O zr(w3m!~So>x;uK-jD@qf=JBcA4^0}m$)ag8VoS=bK&?5q6Xmghirsq?{0N6445y!d zE}T*IYKSXiYxC(lZHWj%Q-fj_)>YLSRv~ven6nUB zYP8Y?vev>@_D0RsK_kC9FL+hSd^Zzm%bz0>Z+WOolgyL{_tAJaKo zNLmbNgKksnWPWmfnq@h&rYrbU8UuoM!v%ocx(VZv^q=Y(pBO+ri^fivApXOqpt>kkYU+M~S9mrpH=zEXV|A~ueJBZQY$%`Ww~#kVJ4C`7wbg?>A2_G7T=0DrY|LrG%O7x?xd-JtHQq74IyrBZDgt( zja>Y?D?F}#4--V0{k1Fi#3Z_;H;mV>djQpTYv6pMw<=+3#z{GiSKuSP^491joROC) zP$P9VnU=z_U;d7lNy0Z?YifF?*Y+36PU~$wmQ7nnTw`BT0y&%r*^hQKQ&|Olkbi)u z1LMJ&hgUTxI4C5fYqLXxz_gnY0Z&Nk27Zo)&zxabUu+7)1Wy!d0E0sd0Tfa6OG98_SAE`G)J3K9ZeJ&ZyAI@q; zA-2u#kCV>UJGl`#@utE}TA*DRc4ai>MHpJ#?xBD47cJWGv!9p28!p3u7DF2k?{49q zLG~=cVSb}V6GhGLf1`4Qz`4>?TZ?rB_>}&A6u_*QjN9L_Hm}GUfH_B~bJ_!R9?B)S z{R$tV6jOPjv)e$V^Q_Gs*^PF-5%mK3~6N=BEq<&O5P7RO9N4IG2yMjmO@Fqq0HKb{$Kf^VC-#P0$l0UkmCXW)c zRNj+cgH~2eEf`H*Ax`%Ypw2^zysfnj<5`o%k}%sU{r1ZE>i=}HdV)N?#)(#@`Ub|~ z2~*T%ubF`D6(b-zsns0HnZj?2_r=+e!Sol*<|cvH5{vr1Ui#YiZ&kPF0-$UpRI1Se zYw9wd577eqmK+;lfcFABLn{=57 zcYLv56^2De#ODETjs{f#>_>dAhd*z>X326p#JAG=Hgvh`o>F%UYlM-Ivm3&JRc5+V z!?H3Hwx`Jn_vEtK$r?yWzm907=@&R6wG96(|7w$Tk2b)? zu&2(p1YsibsB2Si5B^eQ^x`c@l;qL_4`wP$)W(7`iH`+EqrhZn@1^&m7k7%3#)wJh zMd?@gdT>y9j1-m>m&vDbxeD z>p@x9OppE=prY>d_DK>iJmLi$i}NR_wwo%Bx-YnTRyZa>I*oaJY%K)>DQ3pk!zD;9 z=rEz4F?qvU0jw@Ybj!YU8_(e42blq1QJ8DW``KR>1U%_s*Ze2l`*-|(`QN25#P_sR z`ji004X`yWk=tJyp@Jf^4R2zU=R!fX8$MT-0li@y@?0{2FxfuyzAzwg5SSzpD2caT zzamBsbrUFci&J4l`OH!@9QxIp<<5G+-=4Ux-;r)cSRGt7Rd}!5MN1%%C1=kpawVE# zF6M$Jn}})7DZ8QB*mhI|t~Qz07*CF3Q(R$$92)AE(*IwU+ z0-eJZ+-ksPiAiGDJb(ym>6bi8)8@+3p@unJ9p6luC?+>ifVX~B|4jJhQ`PYxk{T{L zSUKI~nbjb(hJFvS{>}^n`quBEcBjEMlka1VKNk%~pW_VJbqHUn_ zy==*NM#`=DX>#pqu`Z}#qaE*%I6=1yk${2qLC&9|N77x3+{csJAHrTI|JnoYVa%j| zh(EgDdnRtplF?Lc>lC-Cy&;yb@UB~+MX*J`(KT&<6y|1dU?^>5zdl~zO)qut^|6%( z;)js;gE?2}snH^jR@bIJQ@DMF`EGncV)Tix;pxPpu8e|v`Ak+vrQ+aBwH{a$TrzgD zwsxs`8ZASv>TL{MG*;B@f+gniWB}l)fDG!}^HjS$j8V4geUW5KE$p{N(cRTacl<7I zB8sv|S~LgsE0E;bMh+&%ju4_g>-Ez!Oe7M^zF5m|nh{(LJ&Y%&4e7ul2Xb1!b5w$MitiEej+GJ0ZgVh#4QrNo7c6oHdERB$Nv zlq*2tEg*!I$-r)xOmOH?Nk^#fjbWl1fc;heeaM8mI zNr?4ca8nq)i>a(F_MeIg_)u+4DwU_iyVSxw=dHb>_DZn1ALfUsdK5JFzH3Api4W+p zr5cT{5()}7-QHppi0*KH+nTQ1(PLt*mvE$%V9o_5H^hGq_yQOSp-VJLT+8^O@w!%C zxTq|i#cNfX?#wWnsX3hqdLplAW-j7s{{7OQ<(IQ|JY)rkv+Iv+sP7i&RgjQQtO-@kSWa@{u(n6mCR6qGPOwG03{Dq{0?`BEI} zfEF-!M{t5%knK#`U7+>JxdAq8{zVw*@)-5_oVC=I#{t`%P04WTi{=xTje-y|mxT{8 z%wEICKc<)e(HAIjNnXHN4*Q z$lj!@(lc$4_fd)J&k4*Jzv%b6{T&FCU%$QDS%6o2=>)65|0m~N*aI-nYqi_zcVDR+ z7l2^1Y>1D3x0CY8bsx^M@U8o>{Z-MQg9~K_C`~=|6sw1E4LIv)g)xm5y0PHMG~;o1 z0ntRn$nEez>V#CCIPL+BoB>Wmdol;7n>G8ckMq~UR^B~hZUfbSJW0UcocEG{4E(~s z8c&-u0TO5vsR{UB6s!rL*#8q3Y2J;{=17Y;fam1MYJi}wm10d~{U%Dv(;~qvAB-Yf z@FOb&G#`yh3PC*k0A3w+4RXu;v$}XNnt_Qc@*hCz*v6JXY+k7?zqQpE-M(Zi8r1W7 zUdaX%W!Tf2+vK^V9hOWnfgP37>mvqzr{>9o?192s72cGV>UV@83;n$wowNtRemj#l}|9TQsv${mH*6K39*{&TY?7@0x0LU3^ZR6WMl+*}%Q zHvY&7(cfc-6YpMy=Jn}r@q3qN>2-%@gm^5kGnvpjUnp5JU(b)@&>v6AY;!e@Dweb} z$r$CSU;DZq@Fv8!sNHcd&U$52p6#)-iXV_#GcSkNJ@PcnnZs1YH^1~>TVs?Fq=i+R z9xX4_Wby%`fY=;fD4!gWNeLo>>2yKH(SLNB&?UG|Q)eHLjZj)&r#wHwf7!}Zs}VAx zPpBu8T(ST~pTI4RCN&Qow<+G+C)qIzTqbffPZ}N;h@Um4oA@+EFX1K945H5pA>pBs zik&(ozVkR=!g0SFJzx8L9&eg*DW0(S|R$jbrb{aMVp{!G1j z3SQx_TZpIX*Nx4@qV+*Xtl+nlqiE`O-3xuKrXq8-i^=}Pd+z7Ca|S<2VlJ1S%hdzzn5jS3W5x!`VG)Yp5`~_E0I^^tUe+V{KB;? zy_$WxV03jGUUfwqUajU0D!MRMst)xD%?ttx^J?;wkJU+NXEg-{7@8GHbqj8x;^P>W zTT68dt8;r6o#Sy_#|^#8oN^61mgOf453H7gChgw_G#tar%9BY+u2!hG9vqHhCvyI1RH>&=^Kl-Y-CGfR&-7t7B|PM>p~)w0bjHD>*m(CE7s zai3<6=FE=eJas*mJSeHU(+E&aWzR{wMk&abG-p3lANk?9mX|AHaH*pF;e5k$&Bz&7QN+lsT&Kz3-fy2zPc8qzfe~qu?U*J@dih`AD@tR@=IDU zWRlgS0fsj=lKD#ERmDv1))WMExZu{hy)@1{$h~-G^srI!7 z9glD>_iZZRwHN^v5kS(MDsitPQNV7PNacpEqwZD}D|24U1nZ?xUPyp@6q4<>fR>yL zmu}}s|6_X>Ol;9u^}*O3a;& zXTMU1ZUN(}5de{L;{b>h)wX{`rL>hJ=#g!4NV(_vN&y5oRd67JZE(%So@i!W3KW_zkJmRznK`bAQ3@{$M3u zgu3!@)*!9$&y7%*8dx5iy`Q1SOSQQ zmcobxi|Pd5Il-Y1P%nfS!G-2GdEixTo9x6Wd$^0+eLv&pUor=ssV`Hvm>%WAzHamt zU4Cco68UcHFt7O2rxSLt6$6CJYsPs}LKrlGnD~mq<174p?zR~O#tB4knsOX`xrBDq zi6ZHEtv;)955fWaD-`J_N@|-)YCfe(gF+@eN(}L3bZSTQd57M-#D%zZY(-r;slS>6aMq zyoBhaziCBjJ=2+?hoffNVKlbqaG}9SytWTX-b+ewmFvAe*%iZwS7)%PKINdp-#acE z5kjlZXF!E58W%LnqcwMbM1FSA4EZnt_{bp$C&c+UH4#FHVl1uJ2?>WCN*Nl7=K?z5 zY%8#l6kCYq|E$L2t|& ztknkSC{V=tFH!WhnC;Deyj-LBjF|DC2-;k>^yA0l<*Fw}0rB?i0|%Q4Wz!akZE4@Y;7D|n@c}HsDr|3t@6>WTDHG+nD37b0gfF-baN(?l# zx!XZSNp{WWTv&?=cTgFosU7+~BWi(g0HuDb02MJQ151rm*cSzP)*NV}Jnf*!+&b^5 zIBni1k5_o*fvC5qZJ-V3S=rw%si{vc(kwK+-oDxNYl|q^7)FuwqdT~D_2;0R;Q|+Q zO8^3bwU87MR9+@AwCdm?MsZtdUga|VIhDCoHI^AWSvDG10nNdzbNtM+NG_1I?8QTy zczeYXeqGMjPT-N$Ro1{=x8bT?;kWl!@=u~09l(OxxkI&o5f|M%J~9cx{G}V(y{7}! z4r*xr0=cnwX)z)Oog|vkAUM)^CCoO><8@;03#i>qTB0 z%?djCKxuoe#P^-LF~9eO_w;ZgZNH-tpZQgJJsr9FbOH*k6D=H5m(~WOuP+Lz*?|WX zdjs3=bkuZ2b&l&0Qu>`8Rdh8!dY5SG7#Wq{^U^gY>9cK4#U<$)waYfxMiogexfe66 zCCzO?0rb6J;4N5AuDpCdGQRi+1aND+i^D{CoVaqJpcZNviR^;t-DB}^vBLzF-Mp{z z0c?Db$*3H9C_)4jf>o)B%NDd3^8-m;HO!&~ETj35RK#lOX)zo1WFG;MUZ-Y2#KcMs z(Z)<~5^oMn*)5F|og3^vj>6|c_ z7gj>PmHE8F)j0bTpwWT`%Y#_Z8T98fcJE13CSu1OkTFSgadXeU5bj3NxJlI0n?U;#~DLZ=brcBBHVua8Y2L{#y1ekONB6?7n{M$5YLLq`R$~{CC&kb8vAII~Mvaj}*w@`Y5hx{&jJF`96YZT{w z(a?GKM0!Wj-sh#6e=Ae^uFwP`2M5b^o0s?Z-|+htXy{rB(!nOdd78;yNFcsF zymVou-!C5;$*h5jOR+tz8TE$-FC*~#tQ=l1mo?1a>xd=tt*^cjC2ZEPTq-rv6Q&&e zsi}CV+riRCKbMtRkp3rbMT26so^%49xCS$&9E(@5INuoJP>FW^w*_1XI!WD55o1*)U#| z@!9r7x=bm*g4-CcA#iL#SOIv!=}>z1X}IbtRI+h_1-hIoXE3w0v>_|Tix6%3(XhbE zxiH~-k!D%At7Mj$r8}-Oo2tQ^y$L@+E-TBnGqkK<& zq$%5d^#OX^yakoG^p9Pa>c8mu1??^*o;M;LY}CX1(BUn(y1at@=tPRx_QigYvuY2v zc!mB@3w;?-DdzPA8@sC*IJkk_rStDmf<6cW13Zc9f+8yEj#W7#j{e4rXFfOSg~ES;0lL~+j%UA zuEUZrgChiqJy+21y5;1mx~zD2Z7?L)XH{C~WZ1p8cn?4?@m* z-UX>vkni^$GZX-8;@)N)Ju8M@7mU&hbz7FCPP@la#GbUY@cAKf*iPF zkU`ddq8?&xvCZqZLm;HrA}PqbJWb`v7$1$@0r6_(MqJcx$E|mVNuxdUGJvHomyF2f z4;r(7GifuEnb(z)!Rt+cj>&n0n>%a>qa(@H^X)5IlY=?8JZ4pU#I z-}LRxmQSS5zifP!NU+rocZjWc#6DKtuG<=7Fi`v!(e{p*!P9p8j*{S$FJ72~li5*v zm_b`~nb-;q4p%CK3<%*n{taWgV&B10W#)%sQMx3(Vm;?&nYJT0YZ{;dj=Ddw1v&xN z@{BGHI-{D2oT|Z3e??#;p>@kKWX>JpRAr0QsZbo5TTKwb5DzLj@~Nyoe~s71Djuy=z~pbh#fK)!TG1<02~>59}BTOQcznDl+0?4fApDPpoSoSzI^ zX}>}0Un|v^g}k9pO6Xd|c3$`QF8X6qY_5y0RF`we^A+tn;_+tyMZ7j+{KAwK>}Lz* zHa-jJzqRY*EcG(6RzOlr5!!bVu|RvfW4H#*x|t$ z$Sy5iHABn-{i&i~44&6|T9$i{oNYd>h>fdvX>-d-or^+^#iU@ZPC!Qz1D!Ey?D)Ud zrMuU&MFq0*V=p*N7HE`*6n-sId}_5J6syqw+Li{PI`aJ;9>38?-aGL{MPi2;hJ|qA zGM+bf)|+FrMml5p?)97qZodr6*Lk2R;KO?-R8&U6e2o5S@qwFa;moaWW}~Kid7I{sZcG&=so z{wK#BfCJF)8icei^tW?80yyAS>s*ZcCmmjcypP@b$1aKdRvV1~2Y8ma=6+k#9>Bmw zfCmPL`| + +// Guess the language with linguistic tagger +NSString *lang1= [BagOfWords guessLanguageCodeWithLinguisticTagger:@"If you're not failing every now and again, it's a sign you're not doing anything very innovative."]; + +// Guess the language with alternative algorith, +NSString *lang2= [BagOfWords guessLanguageCodeWithStopWords:@"If you're not failing every now and again, it's a sign you're not doing anything very innovative."]; +``` + +The language is expresses as a ISO-639-1 code, such as "en" for English, "fr" for French, etc. + +### Text tokenization + +To tokenize a text into a bag of words you need a dictionary: a map that tells for each possible word in the text the position that represents it in the bag of words vector. + +With MAChineLearning, the dictionary is built progressively as texts are tokenized, you just need to fix its maximum size from the beginning. + +```obj-c +NSMutableDictionary *dictionary= [NSMutableDictionary dictionary]; +NSUInteger dictionaryMaxSize= 50000; + +// Load texts into an array +NSArray *movieReviews= // ... + +for (NSString *movieReview in movieReviews) { + + // Extract the bag of words for the current text + BagOfWords *bag= [BagOfWords bagOfWordsForSentimentAnalysisWithText:movieReview + textID:nil + dictionary:dictionary + dictionarySize:dictionaryMaxSize + language:@"en" + featureNormalization:FeatureNormalizationTypeNone]; + + // Dump the extracted tokens + NSLog(@"Tokens: %@", bag.tokens); + + // The actual bag of words vector is accessible in bag.outputBuffer + for (NSString *token in bag.tokens) { + NSNumber *pos= [dictionary objectForKey:[token lowerCaseString]]; + REAL occurrencies= net.outputBuffer[[pos intValue]]; + + NSLog(@"Occurrencies for token '%@': %.0f", token, occurrencies); + } + + // Fill the input buffer of the neural network + // ... +} +``` + +Each tokenization loop adds words to the dictionary. When a new word is encountered, it is assigned a position at the end of the dictionary. Once the loop is completed, it may look something like this: + +- "think": 0, +- "majority": 1, +- "people": 2, +- "seem": 3, +- etc. + + +### Use in combination with a neural network + +In most of use cases, bag of words are submitted as input to a neural network. With MAChineLearning, you may specify that the output buffer of the bag of words is the input buffer of the neural network. This reduces memory and time consumption. + + +```obj-c +for (NSString *movieReview in movieReviews) { + + // Extract the bag of words for the current text + BagOfWords *bag= [BagOfWords bagOfWordsForSentimentAnalysisWithText:movieReview + textID:nil + dictionary:dictionary + dictionarySize:net.inputSize // Use network input size + language:@"en" + featureNormalization:FeatureNormalizationTypeNone + outputBuffer:net.inputBuffer]; // Use network input buffer + + // You may run the network immediately + [net feedForward]; + + // Evaluate the result + // ... +} +``` + +### Choosing tokenization options + +The BagOfWords class provides two factory methods preconfigured for sentiment analysis and topic classification, but you may want to fine tune the tokenizer to your needs. + +There are two kinds of tokenizer: + +- the **simple tokenizer** simply splits the text searching for white spaces and new lines; +- the **linguistic tagger** again uses the iOS/OS X integrated linguistic tagger, which provides more in-depth knowledge on each token found. + +The latter is of course better, but it is much slower, and in some applications is "a bazooka to kill a fly". E.g. in sentiment analysis you usually have better results with wider dictionaries, rather than with carefully picked word combinations. + +With the **simple tokenizer**, tokenization options are: + +- omit stop words; +- keep all bigrams; +- keep all trigrams; +- keep emoticons and emoji. + +With the **linguistic tagger**, tokenization options are: + +- omit stop words; +- omit adjectives; +- omit adverbs; +- omit nouns; +- omit names; +- omit numbers; +- omit others (conjunctions, prepositions, etc.); +- keep verb+adjective combos (e.g. "is nice"); +- keep adjective+noun combos (e.g. "nice movie"); +- keep adverb+noun combos (e.g. "earlier reviews"); +- keep noun+noun combos (e.g. "human thought"); +- keep noun+verb combos (e.g. "movies are"); +- keep 2-word names (e.g. "Alan Turing"); +- keep 3-word names (e.g. "Arthur Conan Doyle"); +- keep all bigrams; +- keep all trigrams; +- keep emoticons and emoji. + +Tokenizers and options are specified using enums `WordExtractorType` and `WordExtractorOption`, respectively. + +The most extended version of the BagOfWords factory methods includes parameters to specify both the tokenizer and its tokenization options: + +```obj-c +BagOfWords *bag= [BagOfWords bagOfWordsWithText:text + textID:textID + dictionary:dictionary + dictionarySize:net.inputSize + language:@"en" + wordExtractor:WordExtractorTypeLinguisticTagger + extractorOptions:WordExtractorOptionOmitStopWords | WordExtractorOptionKeepNounNounCombos | WordExtractorOptionKeep2WordNames | WordExtractorOptionKeep3WordNames + featureNormalization:FeatureNormalizationTypeNone + outputBuffer:net.inputBuffer]; +``` + +Default configurations are the following: + +- for **sentiment analysis**: + - simple tokenizer; + - omit stop words; + - keep emoticons and emoji; + - keep all bigrams. +- for **topic classification**: + - linguistic tagger; + - omit stop words; + - omit verbs, adjectives, adverbs, nouns, others; + - keep adjective+noun combos; + - keep adverb+noun combos; + - keep noun+noun combos; + - keep 2-word names; + - keep 3-word names. + +You may need to experiment a bit to find the correct configuration for you. + +### Choosing normalization options + +When no normalization is applied, ach position on the bag of words vector specifies the number of occurences of the corresponding word in the text. Normalizing the vector means keeping its values limited, to improve the neural network convergence. + +The following normalizations may be applied: + +- **boolean normalization**: occurrencies are not counted and each position may hold just 1 or 0; +- **L2 normalization**: each vector position is divided by the vector's Euclidean length. + +Another common used normalization method, called *TF-iDF*, is missing for now, it will be added in the future. + +Take for example the following sentence: + +- *"I think the majority of the people seem not the get the right idea about the movie"* + +Its bag of words vector could be the following (keeping stop words and adding some more words to fill up): + +![Bag Of Words Norm Example](Bag%20Of%20Words%20Norm%20Example.png) + +Normalization options is specified using enum `FeatureNormalizationType`. + + +## Examples + +The library contains some unit tests that show how to use it, see [BagOfWordsTests.m](MAChineLearningTests/BagOfWordsTests.m). + + +## References + +The following two resources provide a clear introduction to bag of words and their implementation: + +* [Bag of Words Meets Bags of Popcorn - Part 1: For Beginners - Bag of Words](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words) +* [The Vector Space Model of text](http://stanford.edu/~rjweiss/public_html/IRiSS2013/text2/notebooks/tfidf.html) + diff --git a/MAChineLearning.xcodeproj/project.pbxproj b/MAChineLearning.xcodeproj/project.pbxproj index 2e4e747..5ea4f25 100644 --- a/MAChineLearning.xcodeproj/project.pbxproj +++ b/MAChineLearning.xcodeproj/project.pbxproj @@ -109,6 +109,9 @@ 8CC9E4E41AE98DAE002659EA /* BagOfWordsException.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BagOfWordsException.h; sourceTree = ""; }; 8CC9E4E51AE98DAE002659EA /* BagOfWordsException.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = BagOfWordsException.m; sourceTree = ""; }; 8CC9E4EA1AE990AA002659EA /* Constants.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = Constants.h; sourceTree = ""; }; + 8CF961F91AF66C240071A995 /* NeuralNets.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = NeuralNets.md; sourceTree = ""; }; + 8CF961FA1AF66FEE0071A995 /* BagOfWords.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = BagOfWords.md; sourceTree = ""; }; + 8CF961FC1AF690420071A995 /* Bag Of Words Norm Example.png */ = {isa = PBXFileReference; lastKnownFileType = image.png; path = "Bag Of Words Norm Example.png"; sourceTree = SOURCE_ROOT; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -183,8 +186,11 @@ 8C4CEF1A1ADAC51200F1E139 /* MAChineLearning */, 8C4CEF581ADACE4500F1E139 /* LICENSE */, 8C4CEF591ADACE4500F1E139 /* README.md */, + 8CF961F91AF66C240071A995 /* NeuralNets.md */, + 8CF961FA1AF66FEE0071A995 /* BagOfWords.md */, 8C4CEF5E1ADAED9300F1E139 /* 3-Input Perceptron.png */, 8C4CEF5F1ADAED9300F1E139 /* Network States.png */, + 8CF961FC1AF690420071A995 /* Bag Of Words Norm Example.png */, 8C58A3151AECDAA3006AB74D /* StopWordsGen */, 8C4CEF271ADAC51200F1E139 /* MAChineLearningTests */, 8C4CEF191ADAC51200F1E139 /* Products */, @@ -657,6 +663,7 @@ 8C58A31A1AECDAA3006AB74D /* Release */, ); defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; }; /* End XCConfigurationList section */ }; diff --git a/NeuralNets.md b/NeuralNets.md new file mode 100644 index 0000000..a728990 --- /dev/null +++ b/NeuralNets.md @@ -0,0 +1,231 @@ +# Neural Networks with MAChineLearning + +Neural networks in MAChineLearning currently support: + +- multilayer perceptrons of any depth (limited only by memory); +- 4 kinds of activation function: + - linear, + - step (0 if output is less than 0.5, 1 if greater), + - logistic, + - hyperbolic; +- training through backpropagation with standard gradient descent; +- training both by sample and by epoch; +- load/save of the network status; +- single/double precision (needs recompilation, default is single precision). + +Internal code makes heavy use of vDSP and vecLib functions. Compared to the well-known Java library [Neuroph](http://neuroph.sourceforge.net), it is around 20 times faster. It is as fast as it can get on a CPU, but not as fast as it could get with OpenCL on a GPU. An OpenCL version may come in the future. + +Following there's a quick tutorial on how to setup and train your neural network. + + +## Tutorial + +### Set up the network + +Setting up a network is a matter of one line: + +```obj-c +#import +#import + +// Create a perceptron with 3 input lines and 1 output neuron +NeuralNetwork *net= [NeuralNetwork createNetworkWithLayerSizes:@[@3, @1] +outputFunctionType:ActivationFunctionTypeStep]; +``` + +This line creates a single layer perceptron with 3 inputs and 1 output, with step activation function. See the following diagram: + +![3-Input Perceptron](3-Input%20Perceptron.png) + +When creating a multilayer perceptron, hidden layers always use the logistic activation function, the function of your choice applies only to the output layer. + +The import of Accelerate framework is optional and is explained a few paragraphs below. + +The network object exposes all you need to control it, namely: + +- the input buffer, +- the output buffer, +- the expected output buffer, +- methods to feed forward, back propagate and update weights; +- methods to save the status and create a new network from a saved state. + +Buffers are exposed as C arrays for performance reason. They are of type `nnREAL`, which by default is a typedef of `float`. You can redefine this type to `double` in the `NeuralNetworkReal.h` file and then recompile. Just follow the comments. + +**Be careful when reading and writing these arrays** since there's no bounds checking, you may easily cause an `EXC_BAD_ACCESS`. + +### Loading input + +This is how you load your input data: + +```obj-c +// Clear the input buffer using vDSP +nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); + +// Fill appropriate buffer elements +net.inputBuffer[0]= 1.0; +net.inputBuffer[2]= 0.5; +``` + +To clear the buffer with vDSP you need the import of Accelerate, shown above. Here we use the `nnVDSP_VCLR` macro, which is appropriately defined for type `nnREAL` in the `NeuralNetworkReal.h` file. Using Accelerate to clear the buffer is advised as it is quicker than a simple loop. + +### Computing the output + +Once the input buffer is filled, computing the output is simple: + +```obj-c +// Compute the output +[net feedForward]; + +// Log the output +NSLog(@"Output: %.2f", net.outputBuffer[0]); +``` + +### Training + +If the output is not satisfactory (as it always happen on first iterations), you can set the expected output in its specific buffer and ask the network to backpropagate the error. + +```obj-c +// Set the expected output +net.expectedOutputBuffer[0]= 0.5; + +// Backpropagate the error +[net backPropagateWithLearningRate:0.1]; +``` + +The network automatically computes the error and applies the gradient descent algorithm to obtain new weights. The *learning rate* parameter makes learning faster (and more uncertain) for greater values, or slower (but more certain) for lower values. + +**New weights are not applied immediately**: they are stored inside the network, so that you may run multiple feed forwards and backpropagations before applying them (i.e. train by epochs). + +Once your training is done, update the weights in the following way: + +```obj-c +// Update the weights +[net updateWeights]; +``` + +### Training loop example + +You may update the weights after each backpropagation, and this is called *training by sample*, or wait until an entire training set has been performed, and this is called *training by epochs* (a training set is an "epoch", don't ask me why). + +A typical training-by-sample loop is the following: + +```obj-c +for (int i= 0; i < numberOfSets; i++) { + + // Load the i-th set of samples + // ... + + for (int i= 0; i < numberOfSamples; i++) { + nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); + + // Load the j-th training sample + net.inputBuffer[0]= 1.0; + net.inputBuffer[2]= 0.5; + // ... + + [net feedForward]; + + // Check if output is correct + if (net.outputBuffer[0] == 0.5) { + matches++; + + } else { + + // Set the expected output + net.expectedOutputBuffer[0]= 0.5; + // ... + + [net backPropagateWithLearningRate:0.1]; + + // Update weights for just this sample + [net updateWeights]; + } + } +} +``` + +While a typical training-by-epoch loop is the following: + +```obj-c +for (int i= 0; i < numberOfSets; i++) { + + // Load the i-th set of samples + // ... + + for (int j= 0; j < numberOfSamples; j++) { + nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); + + // Load the j-th training sample + net.inputBuffer[0]= 1.0; + net.inputBuffer[2]= 0.5; + // ... + + [net feedForward]; + + // Check if output is correct + if (net.outputBuffer[0] == 0.5) { + matches++; + + } else { + + // Set the expected output + net.expectedOutputBuffer[0]= 0.5; + // ... + + [net backPropagateWithLearningRate:0.1]; + } + } + + // Update weights for all samples + [net updateWeights]; +} +``` + +At the end of the loop you usually check the number of matches and decide if the confidence level is high enough or not. If it is not, the training restarts from the beginning. + +The network tries to enforce the correct calling sequence by using a simple state machine. Check the following state diagram: + +![Network States](Network%20States.png) + +If you try a call that does not correspond to state transition in the above diagram, the network will throw an exception. + + +## Choosing the appropriate model + +This is the hardest part. Your neural network depends on a number of high-level parameters: + +* dimension of the input layer (i.e. how you format the input data); +* number and dimensions of the hidden layers; +* type of activation function; +* training model (by sample or by epoch); +* learning rate. + +Choosing the right values is quite difficult and is exactly why frameworks like this exist: so you can try locally before spending lots of money running the same network on a datacenter. + +Some simple advices: + +* the network may easily diverge if input data contains higher values (logistic and hyperbolic functions makes use of exponentials), **normalize your input data** in some way; +* **keep your model simple**: don't add hidden layers if they are not needed, they will slow down the network consistently and may not improve its prediction abilities; +* if your output is in range \[0..1\] (i.e. for **classification**), use the step or the logistic function, if you need an output in the range \[-1..1\] then move to the hyperbolic function, if you need a more wide-ranged output (i.e. for **regression**) use the linear function; +* start with **training by sample**, switch to training by epoch only if you know what you are doing; +* finally, **keep the learning rate small**: 0.1, or even lower. + + +## Examples + +The library contains 3 unit tests that show how to use it, see [NeuralNetTests.m](MAChineLearningTests/NeuralNetTests.m) or [NeuralNetSwiftTests.swift](MAChineLearningTests/NeuralNetSwiftTests.m). + +The first of them is the NAND logic port discussed on [Wikipedia](http://en.wikipedia.org/wiki/Perceptron#Example). The tests includes a few commented lines that, if uncommented, dump the network status after each training. Note that if you compare it with the Wikipedia example number will differ. This is due to the use of bias, which the Wikipedia example does not include. + + +## References + +There are a lot articles out there explaining how neural networks work, but I have found these two in particular well written and clear enough to base my coding on them: + +* [Machine Learning: Multi Layer Perceptrons](http://ml.informatik.uni-freiburg.de/_media/teaching/ss10/05_mlps.printer.pdf) [PDF] +* [Designing And Implementing A Neural Network Library For Handwriting Detection, Image Analysis etc.](http://www.codeproject.com/Articles/14342/Designing-And-Implementing-A-Neural-Network-Librar) + +I am grateful to these people for taking the time to share their knowledge. + + + diff --git a/README.md b/README.md index 6c51bf7..6680509 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,16 @@ # MAChineLearning + Machine Learning for the Mac ## Intro -This framework provides a quick and easy way to experiment with machine learning with native code on the Mac. It is written in Objective-C and it is usable by Swift (unit tests included for both languages). - -The framework supports: +This framework provides a quick and easy way to experiment with machine learning with native code on the Mac. It is written in Objective-C and it is usable by Swift. -- single layer perceptrons; -- multilayer perceptrons of any depth (limited by memory); -- 4 kinds of activation function: - - linear, - - step (0 if output is less than 0.5, 1 if greater), - - logistic, - - hyperbolic; -- training through backpropagation with standard gradient descent, both by sample and by epoch; -- load/save of the network status; -- single/double precision (needs recompilation, default is single precision). +Currenlty the framework supports: -Internal code makes heavy use of vDSP and vecLib functions. Compared to the well-known Java library [Neuroph](http://neuroph.sourceforge.net), it is around 20 times faster. It is as fast as it can get on a CPU, but not as fast as it could get with OpenCL on a GPU. An OpenCL version may come in the future. +- [Neural Networks](NeuralNets.md) +- [Bag of Words](BagOfWords.md) ### Use on iOS @@ -27,215 +18,10 @@ Internal code makes heavy use of vDSP and vecLib functions. Compared to the well The library should work well on iOS too. Ideally, one could write a program on the Mac to train the network, save its status and then load it on an iPhone or iPad and use it pre-trained. Haven't tested this scenario, but I see no reason it should not work. -## Tutorial - -The library contains 3 unit/integration tests that show how to use it, including the [NAND](http://en.wikipedia.org/wiki/Perceptron#Example) use case discussed on Wikipedia. Anyway here it is a quick tutorial. - -### Set up the network - -Setting up a network is a matter of one line: - -```obj-c -#import -#import - -// Create a perceptron with 3 input lines and 1 output neuron -NeuralNetwork *net= [NeuralNetwork createNetworkWithLayerSizes:@[@3, @1] - outputFunctionType:ActivationFunctionTypeStep]; -``` - -This line creates a single layer perceptron with 3 inputs and 1 output, with step activation function. See the following diagram: - -![3-Input Perceptron](3-Input%20Perceptron.png) - -When creating a multilayer perceptron, hidden layers always use the logistic activation function, the function of your choice applies only to the output layer. - -The import of Accelerate framework is optional and is explained a few paragraphs below. - -The network object exposes all you need to control it, namely: - -- the input buffer, -- the output buffer, -- the expected output buffer, -- methods to feed forward, back propagate and update weights; -- methods to save the status and create a new network from a saved state. - -Buffers are exposed as C arrays for performance reason. They are of type `nnREAL`, which by default is a typedef of `float`. You can redefine this type to `double` in the `NeuralNetworkReal.h` file and then recompile. Just follow the comments. - -**Be careful when reading and writing these arrays** since there's no bounds checking, you may easily cause an `EXC_BAD_ACCESS`. - -### Loading input - -This is how you load your input data: - -```obj-c -// Clear the input buffer using vDSP -nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); - -// Fill appropriate buffer elements -net.inputBuffer[0]= 1.0; -net.inputBuffer[2]= 0.5; -``` - -To clear the buffer with vDSP you need the import of Accelerate, shown above. Here we use the `nnVDSP_VCLR` macro, which is appropriately defined for type `nnREAL` in the `NeuralNetworkReal.h` file. Using Accelerate to clear the buffer is advised as it is quicker than a simple loop. - -### Computing the output - -Once the input buffer is filled, computing the output is simple: - -```obj-c -// Compute the output -[net feedForward]; - -// Log the output -NSLog(@"Output: %.2f", net.outputBuffer[0]); -``` - -### Training - -If the output is not satisfactory (as it always happen on first iterations), you can set the expected output in its specific buffer and ask the network to backpropagate the error. - -```obj-c -// Set the expected output -net.expectedOutputBuffer[0]= 0.5; - -// Backpropagate the error -[net backPropagateWithLearningRate:0.1]; -``` - -The network automatically computes the error and applies the gradient descent algorithm to obtain new weights. The *learning rate* parameter makes learning faster (and more uncertain) for greater values, or slower (but more certain) for lower values. - -**New weights are not applied immediately**: they are stored inside the network, so that you may run multiple feed forwards and backpropagations before applying them (i.e. train by epochs). - -Once your training is done, update the weights in the following way: - -```obj-c -// Update the weights -[net updateWeights]; -``` - -### Training loop example - -You may update the weights after each backpropagation, and this is called *training by sample*, or wait until an entire training set has been performed, and this is called *training by epochs* (a training set is an "epoch", don't ask me why). - -A typical training-by-sample loop is the following: - -```obj-c -for (int i= 0; i < numberOfSets; i++) { - - // Load the i-th set of samples - // ... - - for (int i= 0; i < numberOfSamples; i++) { - nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); - - // Load the j-th training sample - net.inputBuffer[0]= 1.0; - net.inputBuffer[2]= 0.5; - // ... - - [net feedForward]; - - // Check if output is correct - if (net.outputBuffer[0] == 0.5) { - matches++; - - } else { - - // Set the expected output - net.expectedOutputBuffer[0]= 0.5; - // ... - - [net backPropagateWithLearningRate:0.1]; - - // Update weights for just this sample - [net updateWeights]; - } - } -} -``` - -While a typical training-by-epoch loop is the following: - -```obj-c -for (int i= 0; i < numberOfSets; i++) { - - // Load the i-th set of samples - // ... - - for (int j= 0; j < numberOfSamples; j++) { - nnVDSP_VCLR(net.inputBuffer, 1, net.inputSize); - - // Load the j-th training sample - net.inputBuffer[0]= 1.0; - net.inputBuffer[2]= 0.5; - // ... - - [net feedForward]; - - // Check if output is correct - if (net.outputBuffer[0] == 0.5) { - matches++; - - } else { - - // Set the expected output - net.expectedOutputBuffer[0]= 0.5; - // ... - - [net backPropagateWithLearningRate:0.1]; - } - } - - // Update weights for all samples - [net updateWeights]; -} -``` - -At the end of the loop you usually check the number of matches and decide if the confidence level is high enough or not. If it is not, the training restarts from the beginning. - -The network tries to enforce the correct calling sequence by using a simple state machine. Check the following state diagram: - -![Network States](Network%20States.png) - -If you try a call that does not correspond to state transition in the above diagram, the network will throw an exception. - - -## Choosing the appropriate model - -This is the hardest part. Your neural network depends on a number of high-level parameters: - -* dimension of the input layer (i.e. how you format the input data); -* number and dimensions of the hidden layers; -* type of activation function; -* training model (by sample or by epoch); -* learning rate. - -Choosing the right values is quite difficult and is exactly why frameworks like this exist: so you can try locally before spending lots of money running the same network on a datacenter. - -Some simple advices from a beginner like me: - -* the network may easily diverge if input data contains higher values (logistic and hyperbolic functions makes use of exponentials), **normalize your input data** in some way; -* **keep your model simple**: don't add hidden layers if they are not needed, they will slow down the network consistently and may not improve its prediction abilities; -* if your output is in range \[0..1\] (i.e. for **classification**), use the step or the logistic function, if you need an output in the range \[-1..1\] then move to the hyperbolic function, if you need a more wide-ranged output (i.e. for **regression**) use the linear function; -* start with **training by sample**, switch to training by epoch only if you know what you are doing; -* finally, **keep the learning rate small**: 0.1, or even lower. - - -## References - -There are a lot articles out there explaining how neural networks work, but I have found these two in particular well written and clear enough to base my coding on them: - -* [Machine Learning: Multi Layer Perceptrons](http://ml.informatik.uni-freiburg.de/_media/teaching/ss10/05_mlps.printer.pdf) [PDF] -* [Designing And Implementing A Neural Network Library For Handwriting Detection, Image Analysis etc.](http://www.codeproject.com/Articles/14342/Designing-And-Implementing-A-Neural-Network-Librar) - -I am grateful to these people for taking the time to share their knowledge. - - ## About -I am a professional developer but not a data scientist. I wrote this library because, you know, they say you haven't really understood something until you code it. So, here it is. Use it to experiment and have fun, and if you find it useful I will be happy to hear it at [@foolish_dev](http://www.twitter.com/foolish_dev). +I am a professional developer but not a data scientist. I wrote this library because, you know, they say you haven't really understood something until you can code it. So, here it is. Use it to experiment and have fun, and if you find it useful I will be happy to hear it at [@self_vs_this](http://www.twitter.com/self_vs_this). -Remember, anyway, that it may contain bugs and/or conceptual mistakes. If you find any of them, please report. So that I can fix them and, most importantly, learn something new. +Every effort has been taken to guarantee the library is error-free, including a side-by-side weights/results comparision with other open source libraries. If you find bugs or conceptual mistakes please report them. So I can fix them and, most importantly, learn something new. Enjoy.