-
Notifications
You must be signed in to change notification settings - Fork 0
CRFsuite
byeong il, ko edited this page Mar 2, 2015
·
7 revisions
- Source : http://www.chokkan.org/software/crfsuite/ by Naoki Okazaki
- 텍스트 청킹은 텍스트를 구문적으로 상관성 있는 단어 부분으로 나누는 것.
- 예)
- "He reckons the current account deficit will narrow to only # 1.8 billion in September"
- [NP He] [VP reckons ] [NP the current account deficit ] [VP will narrow ] [PP to ] [NP only # 1.8 billion ] [PP in ] [NP September ] .
- NP : Noun Phrase(명사구)
- VP : Verb Phrase(동사구)
- PP : Prepostional Phrase(전치사구)
- 이 테스크는 "Sequential labeling" 업무
- 텍스트의 토큰순서들에 대해서 "라벨"순서로 채우는 것.
- chunk(토큰의 Span)을 라벨로서 나타내기 위해 IOB2 표기법을 사용함
- I : Inside of a chunk
- B : Begin of a chunk
- O : 청크에 해당되지 않는 것들(out of a chunk)
B-NP He
B-VP reckons
B-NP the
I-NP current
I-NP account
I-NP deficit
B-VP will
I-VP narrow
B-PP to
B-NP only
I-NP #
I-NP 1.8
I-NP billion
B-PP in
B-NP September
O .
- 본 튜토리얼의 목적은 주어진 문장(토큰의 순서열)에 청크라벨을 예상하는 모델을 CRFsuite를 이용하 빌드하는 것
- CoNLL 2000 shared task에서 배포하는 training & testing data를 사용함
- example 디렉토리 밑에 스크립트로 가능하다.
$ cd example
$ wget http://www.cnts.ua.ac.be/conll2000/chunking/train.txt.gz
$ wget http://www.cnts.ua.ac.be/conll2000/chunking/test.txt.gz
$ less train.txt.gz
... (snip) ...
London JJ B-NP
shares NNS I-NP
closed VBD B-VP
moderately RB B-ADVP
lower JJR I-ADVP
in IN B-PP
thin JJ B-NP
trading NN I-NP
. . O
At IN B-PP
Tokyo NNP B-NP
, , O
the DT B-NP
Nikkei NNP I-NP
index NN I-NP
of IN B-PP
225 CD B-NP
selected VBN I-NP
issues NNS I-NP
was VBD B-VP
up IN B-ADVP
112.16 CD B-NP
points NNS I-NP
to TO B-PP
35486.38 CD B-NP
. . O
... (snip) ...
- 문장세트의 데이터 구성은 다음과 같다.
- 단어 열 : London, shares
- PoS : JJ, NNS
- Chunk Label : B-NP, I-NP
- delimiter : space
- 학습과 테스트의 전처리 과정
- 데이터에서 단어(아이템)의 특성을 설명하는(표현하는) 속성들을 추출함
- CRFsuite는 내부적으로 데이터 셋안에 속성들로 부터 자질들을 생성함
- 기계학습과정에서 가장 중요함 : 자질 설계(Feature Design)이 라벨링 정확도에 가장 큰 역할을 하기 때문
- 여기서는 19가지 종류의 속성을 이용함
- w[t-2], w[t-1], w[t], w[t+1], w[t+2] : 5 unigram
- w[t-1]w[t], w[t]w[t+1] : 2 bigram
- pos[t-2], pos[t-1], pos[t], pos[t+1], pos[t+2], : 5 unigram Pos-Tag
- pos[t-2]|pos[t-1], pos[t-1]|pos[t], pos[t]|pos[t+1], pos[t+1]|pos[t+2], : 4 bigram Pos-Tag
- pos[t-2]|pos[t-1]|pos[t], pos[t-1]|pos[t]|pos[t+1], pos[t]|pos[t+1]|pos[t+2] : 3 trigram PoS Tag
- 설명
- w[t], pos[t] : 현재 위치의 단어와 part-of-speech tag.
He PRP B-NP
reckons VBZ B-VP
t --> the DT B-NP
current JJ I-NP
account NN I-NP
-
속성들
-
w[-2]=He, w[-1]=reckons, w[0]=the, w[1]=current, w[2]=account
-
w[-1]|w[0]=reckons|the , w[0]|w[1]=the|current
-
pos[-2]=PRP, pos[-1]=VBZ, pos[0]=DT, pos[1]=JJ, pos[2]=NN
-
pos[-2]|pos[-1]=PRP|VBZ, pos[-1]|pos[0]=VBZ|DT, pos[0]|pos[1]=DT|JJ, pos[1]|pos[2]=JJ|NN
-
pos[-2]|pos[-1]|pos[0]=PRP|VBZ|DT, pos[-1]|pos[0]|pos[1]=VBZ|DT|JJ, pos[0]|pos[1]|pos[2]=DT|JJ|NN
-
"name=value" 같은 컨벤션은 따르지않음. -> 속성이름과 그것의 가중치롤 사용되기 때문
-
CRFsuite는 속성을 TAB('\t') 로 구분함
-
구현이 어렵지 않음
$ zcat train.txt.gz | ./chunking.py > train.crfsuite.txt
$ zcat test.txt.gz | ./chunking.py > test.crfsuite.txt
$ less train.crfsuite.txt
... (snip) ...
B-NP w[0]=He w[1]=reckons w[2]=the w[0]|w[1]=He|reckons pos[0]=P
RP pos[1]=VBZ pos[2]=DT pos[0]|pos[1]=PRP|VBZ pos[1]|pos[2]=VB
Z|DT pos[0]|pos[1]|pos[2]=PRP|VBZ|DT __BOS__
B-VP w[-1]=He w[0]=reckons w[1]=the w[2]=current w[-1]|w[
0]=He|reckons w[0]|w[1]=reckons|the pos[-1]=PRP pos[0]=VBZ pos[1]=D
T pos[2]=JJ pos[-1]|pos[0]=PRP|VBZ pos[0]|pos[1]=VBZ|DT pos[1]|p
os[2]=DT|JJ pos[-1]|pos[0]|pos[1]=PRP|VBZ|DT pos[0]|pos[1]|pos[2]=VBZ
|DT|JJ
B-NP w[-2]=He w[-1]=reckons w[0]=the w[1]=current w[2]=acc
ount w[-1]|w[0]=reckons|the w[0]|w[1]=the|current pos[-2]=PRP pos[-1]=
VBZ pos[0]=DT pos[1]=JJ pos[2]=NN pos[-2]|pos[-1]=PRP|VBZ
pos[-1]|pos[0]=VBZ|DT pos[0]|pos[1]=DT|JJ pos[1]|pos[2]=JJ|NN pos[-2]|
pos[-1]|pos[0]=PRP|VBZ|DT pos[-1]|pos[0]|pos[1]=VBZ|DT|JJ pos[0]|pos[1]|po
s[2]=DT|JJ|NN
I-NP w[-2]=reckons w[-1]=the w[0]=current w[1]=account w[2]=def
icit w[-1]|w[0]=the|current w[0]|w[1]=current|account pos[-2]=VBZ
pos[-1]=DT pos[0]=JJ pos[1]=NN pos[2]=NN pos[-2]|pos
[-1]=VBZ|DT pos[-1]|pos[0]=DT|JJ pos[0]|pos[1]=JJ|NN pos[1]|pos[2]=NN|NN
pos[-2]|pos[-1]|pos[0]=VBZ|DT|JJ pos[-1]|pos[0]|pos[1]=DT|JJ|NN pos
[0]|pos[1]|pos[2]=JJ|NN|NN
I-NP w[-2]=the w[-1]=current w[0]=account w[1]=deficit w[2]=wil
l w[-1]|w[0]=current|account w[0]|w[1]=account|deficit pos[-2]=
DT pos[-1]=JJ pos[0]=NN pos[1]=NN pos[2]=MD pos[-2]|
pos[-1]=DT|JJ pos[-1]|pos[0]=JJ|NN pos[0]|pos[1]=NN|NN pos[1]|pos[2]=NN
|MD pos[-2]|pos[-1]|pos[0]=DT|JJ|NN pos[-1]|pos[0]|pos[1]=JJ|NN|NN pos[0]|p
os[1]|pos[2]=NN|NN|MD
... (snip) ...- token size
$ zcat train.txt.gz | wc -l
220663
$ wc -l train.crfsuite.txt
220663 train.crfsuite.txt
- CRFsuite 는 state(속성-라벨)과 전이(Transition , 라벨 바이그램) 자질을 생성함
- 데이터의 조건부확률 분포의 log-likelihood를 최대화한 자질 값이고 그것을 CoNLL2000.model에 저장함
$ crfsuite learn -m CoNLL2000.model train.crfsuite.txt
...
$ ls -al CoNLL2000.model
-rw-r--r-- 1 user staff 30095264 3 2 10:48 CoNLL2000.model- 성능 측정 (accuracy, precision, recall, f1 score)
- 모델 성능 확상을 확인 할 수 있다.(!!!!)
$ crfsuite learn -e2 train.crfsuite.txt test.crfsuite.txt- CRF 모델과 tag chunk라벨을 테스트 데이터에 적용할 수 있음
$ head ./test.crfsuite.txt
B-NP w[0]=Rockwell w[1]=International w[2]=Corp. w[0]|w[1]=Rockwell|International pos[0]=NNP pos[1]=NNP pos[2]=NNP pos[0]|pos[1]=NNP|NNP pos[1]|pos[2]=NNP|NNP pos[0]|pos[1]|pos[2]=NNP|NNP|NNP __BOS__
I-NP w[-1]=Rockwell w[0]=International w[1]=Corp. w[2]='s w[-1]|w[0]=Rockwell|International w[0]|w[1]=International|Corp. pos[-1]=NNP pos[0]=NNP pos[1]=NNP pos[2]=POS pos[-1]|pos[0]=NNP|NNP pos[0]|pos[1]=NNP|NNP pos[1]|pos[2]=NNP|POS pos[-1]|pos[0]|pos[1]=NNP|NNP|NNP pos[0]|pos[1]|pos[2]=NNP|NNP|POS
I-NP w[-2]=Rockwell w[-1]=International w[0]=Corp. w[1]='s w[2]=Tulsa w[-1]|w[0]=International|Corp. w[0]|w[1]=Corp.|'s pos[-2]=NNP pos[-1]=NNP pos[0]=NNP pos[1]=POS pos[2]=NNP pos[-2]|pos[-1]=NNP|NNP pos[-1]|pos[0]=NNP|NNP pos[0]|pos[1]=NNP|POS pos[1]|pos[2]=POS|NNP pos[-2]|pos[-1]|pos[0]=NNP|NNP|NNP pos[-1]|pos[0]|pos[1]=NNP|NNP|POS pos[0]|pos[1]|pos[2]=NNP|POS|NNP
B-NP w[-2]=International w[-1]=Corp. w[0]='s w[1]=Tulsa w[2]=unit w[-1]|w[0]=Corp.|'s w[0]|w[1]='s|Tulsa pos[-2]=NNP pos[-1]=NNP pos[0]=POS pos[1]=NNP pos[2]=NN pos[-2]|pos[-1]=NNP|NNP pos[-1]|pos[0]=NNP|POS pos[0]|pos[1]=POS|NNP pos[1]|pos[2]=NNP|NN pos[-2]|pos[-1]|pos[0]=NNP|NNP|POS pos[-1]|pos[0]|pos[1]=NNP|POS|NNP pos[0]|pos[1]|pos[2]=POS|NNP|NN
I-NP w[-2]=Corp. w[-1]='s w[0]=Tulsa w[1]=unit w[2]=said w[-1]|w[0]='s|Tulsa w[0]|w[1]=Tulsa|unit pos[-2]=NNP pos[-1]=POS pos[0]=NNP pos[1]=NN pos[2]=VBD pos[-2]|pos[-1]=NNP|POS pos[-1]|pos[0]=POS|NNP pos[0]|pos[1]=NNP|NN pos[1]|pos[2]=NN|VBD pos[-2]|pos[-1]|pos[0]=NNP|POS|NNP pos[-1]|pos[0]|pos[1]=POS|NNP|NN pos[0]|pos[1]|pos[2]=NNP|NN|VBD
...
$ crfsuite tag -m CoNLL2000.model ./test.crfsuite.txt
B-NP
I-NP
I-NP
B-NP
I-NP
I-NP
B-VP
B-NP
B-VP
B-NP
...- 참조 라벨과 예측한 라벨을 출력함
$ crfsuite tag -r -m CoNLL2000.model ./test.crfsuite.txt
B-NP B-NP
I-NP I-NP
I-NP I-NP
B-NP B-NP
I-NP I-NP
I-NP I-NP
B-VP B-VP
B-NP B-NP
B-VP B-VP
B-NP B-NP
...- 평가
$ crfsuite tag -qt -m CoNLL2000.model ./test.crfsuite.txt
Performance by label (#match, #model, #ref) (precision, recall, F1):
B-NP: (12000, 12358, 12407) (0.9710, 0.9672, 0.9691)
B-PP: (4707, 4872, 4805) (0.9661, 0.9796, 0.9728)
I-NP: (13983, 14483, 14359) (0.9655, 0.9738, 0.9696)
B-VP: (4466, 4663, 4653) (0.9578, 0.9598, 0.9588)
I-VP: (2549, 2698, 2643) (0.9448, 0.9644, 0.9545)
B-SBAR: (448, 498, 534) (0.8996, 0.8390, 0.8682)
O: (5939, 6113, 6174) (0.9715, 0.9619, 0.9667)
B-ADJP: (322, 403, 438) (0.7990, 0.7352, 0.7658)
B-ADVP: (711, 835, 866) (0.8515, 0.8210, 0.8360)
I-ADVP: (54, 82, 89) (0.6585, 0.6067, 0.6316)
I-ADJP: (110, 137, 167) (0.8029, 0.6587, 0.7237)
I-SBAR: (2, 15, 4) (0.1333, 0.5000, 0.2105)
I-PP: (34, 42, 48) (0.8095, 0.7083, 0.7556)
B-PRT: (80, 102, 106) (0.7843, 0.7547, 0.7692)
B-LST: (0, 0, 4) (0.0000, 0.0000, 0.0000)
B-INTJ: (1, 1, 2) (1.0000, 0.5000, 0.6667)
I-INTJ: (0, 0, 0) (******, ******, ******)
B-CONJP: (5, 7, 9) (0.7143, 0.5556, 0.6250)
I-CONJP: (10, 12, 13) (0.8333, 0.7692, 0.8000)
I-PRT: (0, 0, 0) (******, ******, ******)
B-UCP: (0, 0, 0) (******, ******, ******)
I-UCP: (0, 0, 0) (******, ******, ******)
Macro-average precision, recall, F1: (0.639230, 0.602508, 0.611080)
Item accuracy: 45421 / 47321 (0.9598)
Instance accuracy: 1176 / 2011 (0.5848)
Elapsed time: 0.637321 [sec] (3157.0 [instance/sec])$ crfsuite dump CoNLL2000.model > CoNLL2000.model.dump
$ head CoNLL2000.model.dump
FILEHEADER = {
magic: lCRF
size: 30095264
type: FOMC
version: 100
num_features: 0
num_labels: 22
num_attrs: 335674
off_features: 0x30
off_labels: 0x8A2BB8