Skip to content
Olanto Foundation edited this page May 31, 2018 · 1 revision

Examine the training and test log

open indexer

 START [2018/05/31 12:33:54] : global time
 running OS:Windows 8.1
 ...
  HOSTS
     EXPERIMENT_NOTFORPROD: false
     CHECK_CONSISTENT: false
 ...
 init mode:QUERY
 start loading
 ...

load catalog

 maxbottomgroup:22

load BOW

 START [2018/05/31 12:34:07] : global time MAINGROUP --------------------------
 in memory :true
 in memory load docbag ...
 START [2018/05/31 12:34:07] : avgLength()
 lastdoc:21537
 lastword:401448

statistics on BOW

 #doc:21536, avg:20, min:1, max:104
 STOP [2018/05/31 12:34:07]: avgLength() - 62 ms

filtering words and compute test set

 GLOBALMINOCC: 2 , MAX features:33573
 start mem: 121454680
 2.
 lasttraindoc:21536
 lasttestdoc:21536
 Train 0..17228 Test ..21536
 maxgroup:22
 after localgroup: 124413040
 Active group:22

start training

 START [2018/05/31 12:34:07] : TrainWinnow
 after init: 125684344
 filter  used:0, open:33573, discarded:0, filtred:0
 Start loop 0 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 - 4 - 5 - 7 - 3 - 6 - 0 - 1 - 2 End loop 0
 Start loop 1 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 - 5 - 6 - 1 - 4 - 7 - 2 - 3 - 0 End loop 1
 Start loop 2 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 - 4 - 0 - 1 - 3 - 5 - 6 - 7 - 2 End loop 2
 Start loop 3 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 - 1 - 0 - 2 - 3 - 4 - 6 - 7 - 5 End loop 3
 Start loop 4 + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 - 1 - 2 - 0 - 5 - 3 - 7 - 4 - 6 End loop 4
 # features: 33573
 # maxgroup: 22
 # maxtrain: 17228
 # avg doc : 20
 # repeatK: 5
 size of NN: 738 [Kn]
 estimate #eval (if no discarded feature): 37884 [Kev]
 estimate power (if no discarded feature): 185 [Mev/sec]
 STOP [2018/05/31 12:34:08]: TrainWinnow - 219 ms

start testing mono-class

 Mainclass1000.0,1.06,2,300.0,300.0,9979,20,9955,18,4
 detail in: C:/MYCLASS_MODEL/experiment/langdetect/detailworddetect-MainDetail-Class.txt

start testing multi-class

 Manyclass1000.0,1.06,2,300.0,300.0,9979,20,9955,18,4
 detail in: C:/MYCLASS_MODEL/experiment/langdetect/detailworddetect-ManyDetail-Class.txt

build confusion matrix

 START [2018/05/31 12:34:08] : ConfusionMatrix
 1000.0,1.06,2,300.0,300.0,995,995,4,0
 confusion matrix: (line=real category; colums= prediction)
 >>predict,HU,ET,PL,LT,EL,LV,MT,DE,SL,BG,EN,SV,NL,SK,FR,CS,FI,PT,IT,ES,DA,RO,
 HU,203,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,
 ET,0,204,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,
 PL,0,0,202,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 LT,0,0,0,194,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 EL,0,0,0,0,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 LV,0,0,0,0,0,193,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 MT,0,0,0,0,0,1,186,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 DE,0,0,0,0,0,0,0,190,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
 SL,0,0,0,0,0,0,0,0,203,0,0,0,0,1,0,0,0,0,0,0,0,0,
 BG,0,0,0,0,0,0,0,1,0,216,0,0,0,0,0,0,0,0,0,0,0,0,
 EN,0,0,0,0,0,0,0,0,0,0,215,0,0,0,0,0,0,0,0,1,1,0,
 SV,0,0,0,0,0,0,0,0,0,0,0,221,0,0,0,0,0,0,0,0,1,0,
 NL,0,0,0,0,0,0,0,0,0,0,0,0,201,0,0,0,0,0,0,0,0,0,
 SK,0,0,0,0,0,0,0,0,1,0,0,0,0,195,0,1,0,0,0,1,0,0,
 FR,0,0,0,0,0,0,0,0,0,0,0,0,0,0,211,0,0,0,0,0,0,0,
 CS,0,0,0,0,0,0,0,0,1,0,0,0,0,2,0,199,0,0,0,0,0,0,
 FI,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,205,0,0,0,0,0,
 PT,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,177,0,0,0,0,
 IT,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,195,0,0,0,
 ES,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,197,0,0,
 DA,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,200,0,
 RO,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,98,
 STOP [2018/05/31 12:34:08]: ConfusionMatrix - 47 ms

build top features

 groupe, nbdoc, kw1, kw2, kw3, ...
 ---- 0 HU, nbdoc:1000,az,és,bizottság,hl,tanács,nem,következo,csatlakozás,kell,ig,szóló,egészül,vagy,vonatkozó,rendelete,hoeromu,mint,támogatás,nemzeti,melléklet
 ---- 1 ET, nbdoc:1000,bulgaaria,või,ning,kui,lk,rumeenia,euroopa,eü,lisa,artikli,aasta,mis,nõukogu,kohta,kuni,bulgaarias,alusel,suhtes,määruse,punkti
 ---- 2 PL, nbdoc:1000,w,dnia,dz,sie,dla,oraz,we,panstwa,przez,bulgarii,rumunii,które,czlonkowskie,przy,jest,sa,zgodnie,rocznie,lub,przystapienia
 ---- 3 LT, nbdoc:1000,del,eb,iš,iki,istojimo,kaip,pagal,bulgarijos,punkte,ol,saugyklai,bulgarija,gali,i,nuo,yra,dalyje,arba,bulgarijoje,tarybos
 ---- 4 EL, nbdoc:1000,?a?,st?,t??,ap?,t??,??a,t??,p??,µe,t?,de?aµe??,t??,st??t??,t?,ß????a??a,??,ta,s?µe??,st?,?
 ---- 5 LV, nbdoc:1000,gada,uz,punkta,eiropas,pievienošanas,kas,lpp,vai,attieciba,ka,pec,panta,ov,atbilstigi,padomes,lai,lidz,dalas,bulgarija,ša
 ---- 6 MT, nbdoc:1000,li,apos,ankara,jew,ghandu,doganali,fuq,ghandha,ghandhom,dawn,u,minn,ghal,ta,ikunu,inkluzi,fl-appendici,ma,kif,dak
 ---- 7 DE, nbdoc:1000,und,aschebecken,vom,für,von,verordnung,werden,nach,wird,aus,über,mit,nummer,oder,absatz,abl,zur,nicht,dem,sind
 ---- 8 SL, nbdoc:1000,iz,pristopa,bolgarija,ul,ali,glede,lahko,št,bolgariji,prilogi,odstavka,sveta,tem,pogodbe,kot,skladu,sklep,države,podlagi,odbora
 ---- 9 BG, nbdoc:1000,??,?,?,?,???,??,??,??,?????,?????????,????,???,son,????????,altesse,royale,grand-duc,??,????????????,???????????
 ---- 10 EN, nbdoc:1000,shall,and,decision,following,oj,or,accession,regulation,european,council,committee,treaty,republic,member,may,with,amended,by,areas,executive
 ---- 11 SV, nbdoc:1000,och,av,skall,för,från,till,att,inte,enligt,får,förordning,vid,följande,är,genom,senast,anslutningen,tillämpas,republiken,som
 ---- 12 NL, nbdoc:1000,van,het,met,voor,door,bulgarije,zijn,aan,wordt,worden,asvijver,dat,tot,lid,volgende,bijlage,op,een,roemenië,bij
 ---- 13 SK, nbdoc:1000,ú,alebo,pre,ktoré,ako,rozhodnutie,pristúpenia,týchto,sú,môže,komisia,popol,súlade,sa,vo,ods,nariadenia,opatrenia,štátov,odseku
 ---- 14 FR, nbdoc:1000,dans,bulgarie,est,les,adhésion,une,du,république,paragraphe,règlement,aux,qui,sur,pour,cette,état,au,roumanie,sont,peut
 ---- 15 CS, nbdoc:999,nebo,pristoupení,pro,spolecenství,narízení,ve,techto,opatrení,být,cl,pokud,oblast,príloze,odst,smernice,státy,komise,souladu,prosince,které
 ---- 16 FI, nbdoc:1000,ey,kuin,päivänä,sekä,euroopan,jotka,mukaisesti,tuhka-allas,kohdassa,artiklan,neuvoston,liitteessä,sovelletaan,osalta,komissio,eyvl,bulgariassa,vuoden,annettu,että
 ---- 17 PT, nbdoc:1000,em,ao,conselho,roménia,artigo,com,não,regulamento,uma,adesão,membros,os,aos,bacia,cinzas,decisão,nas,comissão,é,até
 ---- 18 IT, nbdoc:1000,di,della,dell,dal,che,il,è,consiglio,regolamento,adesione,nel,stati,sono,dei,dicembre,allegato,commissione,gu,pag,stagno
 ---- 19 ES, nbdoc:1000,las,consejo,los,comisión,el,decisión,y,adhesión,ejecutivo,miembros,podrá,unión,artículo,declaración,hasta,con,diciembre,rumanía,miembro,común
 ---- 20 DA, nbdoc:1000,og,til,af,stk,skal,fra,ikke,ef,disse,inden,afsnit,følgende,omhandlet,bilag,forordning,rumænien,før,nye,tiltrædelsesdatoen,fastsættes
 ---- 21 RO, nbdoc:537,?i,în,acord,cu,sa,care,pe,prin,sau,prezentul,pentru,catre,poate,nu,sunt,acest,consiliul,fiecare,acordul,este

(total time: 17 seconds)