The Oslo-Bergen Tagger OBT+stat - a short presentation André Lynum, Kristin Hagen, Janne Bondi Johannessen and Anders Nøklestad.

Presentasjon om: "The Oslo-Bergen Tagger OBT+stat - a short presentation André Lynum, Kristin Hagen, Janne Bondi Johannessen and Anders Nøklestad."— Utskrift av presentasjonen:

1 The Oslo-Bergen Tagger OBT+stat - a short presentation André Lynum, Kristin Hagen, Janne Bondi Johannessen and Anders Nøklestad

2 Morphosyntactic tagger and lemmatizer Bokmål and Nynorsk Based on lexicon and linguistic rules Statistical disambiguation for completely unambiguous output (Currently Bokmål only)

3 Purpose Annotation for linguistic research (e.g. The Oslo Corpus) Large scale corpora annotation (e.g. NoWaC in progress)

4 Applications Grammar checker in Microsoft Word and others Open source and commercial translation systems (Apertium, NyNo, Kaldera) Commercial Content Management Systems (TextUrgy)

5 Resources Lexicon based on Norsk ordbank Bokmål: 151 229 entries Nynorsk: 126 323 entries

6 Resources Hand-made Constraint Grammar rules Bokmål: 2214 morphological rules Nynorsk: 3849 morphological rules

7 Resources Development and test corpora Training/development corpus approx. 120,000 words each for Bokmål and Nynorsk Test/evaluation corpus approx. 30,000 words each for Bokmål and Nynorsk

8 Resources Dependency syntax for both Bokmål and Nynorsk

9 Technology Multitagger Common Lisp CG Disambiguator VislCG3 (C++) Statistical Disambiguator Ruby, HunPos

10 Pipeline

11 Results Competitive results on varied domains

12 Multitagger Sophisticated tokenizer, morphological analyzer and compound word analyzer (guesser) Enumerates all possible tags and lemmas Tags composed of detailed morphosyntactic information

13 Multitagger output Dette " "dette" verb inf i2 pa4 "dette" pron nøyt ent pers 3 "dette" det dem nøyt ent er " "være" verb pres a5 pr1 pr2 en " "en" det mask ent kvant "en" pron ent pers hum "en" adv "ene" verb imp tr1 testsetning " "testsetning" subst appell fem ub ent samset "testsetning" subst appell mask ub ent samset. " "$." clb

14 Multitagger output en " "en" det mask ent kvant "en" pron ent pers hum "en" adv "ene" verb imp tr1

15 CG Disambiguator Based on detailed Constraint Grammar rulesets for Bokmål and Nynorsk Rules compatible with the state of the art VislCG3 disambiguator Efficiently disambiguates multitagger cohorts with high precision Leaves some ambiguity by design

16 CG Rules #:2553 SELECT:2553 (subst mask ent) IF (NOT 0 farlige-mask-subst) (NOT 0 fv) (NOT 0 adj) (NOT -1 komma/konj) (**-1C mask-det LINK NOT 0 nr2-det LINK NOT *1 ikke-adv-adj) ; # "en vidunderlig vakker sommerfugl"

17 Example output Dette " "dette" pron nøyt ent pers 3 SELECT:2607 ; "dette" verb inf i2 pa4 SELECT:2607 ; "dette" det dem nøyt ent SELECT:2607 er " "være" verb pres a5 pr1 pr2 en " "en" det mask ent kvant SELECT:2762 ; "en" adv REMOVE:3689 ; "en" pron ent pers hum SELECT:2762 ; "ene" verb imp tr1 SELECT:2762 testsetning " "testsetning" subst appell mask ub ent samset SELECT:2553 ; "testsetning" subst appell fem ub ent samset SELECT:2553. " "$." clb

18 Example of ambiguity left unresolved Setninger " "setning" subst appell fem ub fl "setning" subst appell mask ub fl kan " "kunne" verb pres tr1 tr3 være " "være" verb inf tr5 "være" verb inf a5 pr1 pr2 ; "være" subst appell nøyt ubøy REMOVE:3123 vanskelige " "vanskelig" adj fl pos ; "vanskelig" adj be ent pos REMOVE:2318. " "$." clb

19 Example of ambiguity left unresolved Setninger " "setning" subst appell fem ub fl "setning" subst appell mask ub fl

20 Example of unresolved ambiguity Det " "det" pron nøyt ent pers 3 SELECT:2607 ; "det" det dem nøyt ent SELECT:2607 dreier " "dreie" verb pres tr1 i2 tr11 SELECT:2467 ; "drei" subst appell mask ub fl SELECT:2467 ; "dreier" subst appell mask ub ent SELECT:2467 seg " "seg" pron akk refl SELECT:3333 ; "sige" verb pret i2 a3 pa4 SELECT:3333 om " "om" prep SELECT:2653 ; "om" sbu SELECT:2653 åndsverk " "åndsverk" subst appell nøyt ub fl "åndsverk" subst appell nøyt ub ent. " "$." clb

21 Example of unresolved ambiguity åndsverk " "åndsverk" subst appell nøyt ub fl "åndsverk" subst appell nøyt ub ent

22 Example of lemma ambiguity Det " " "Det" subst prop gamle " " "gammel" adj be ent pos SELECT:3064 "gammal" adj be ent pos SELECT:3064 ; "gammel" adj fl pos SELECT:3064 ; "gammal" adj fl pos SELECT:3064 testamentet " " "testament" subst appell nøyt be ent "testamente" subst appell nøyt be ent. " "

23 Example of lemma ambiguity gamle " " "gammel" adj be ent pos SELECT:3064 "gammal" adj be ent pos SELECT:3064

24 Example of lemma ambiguity Oslo " " "Oslo" subst prop er " " "være" verb pres a5 pr1 pr2 byen " " "bye" subst appell mask be ent "by" subst appell mask be ent vår " " "vår" det mask ent poss SELECT:2689 ; "vår" det fem ent poss SELECT:2689 ; "vår" subst appell mask ub ent SELECT:2689. " " "$." clb

25 Example of lemma ambiguity byen " " "bye" subst appell mask be ent "by" subst appell mask be ent

26 Example of unwanted ambiguity Livet på jorden har tilpasset seg og tildels utnyttet de skiftende forhold.

27 Example of unwanted ambiguity og " " "og" konj "og" konj clb ; "og" adv REMOVE:2227 til dels " " "til dels" adv prep+subst @adv utnyttet " " "utnytte" verb pret tr1 "utnytte" verb perf-part tr1 ; "utnytte" adj nøyt ub ent tr1 REMOVE:2274 ; "utnytte" adj ub m/f ent tr1 REMOVE:2274 de " " "de" det dem fl SELECT:2780 ; "de" pron fl pers 3 nom SELECT:2780 skiftende " " "skifte" adj tr1 i1 i2 tr11 pa1 pa2 pa5 tr13 forhold

28 Example of unwanted ambiguity utnyttet " " "utnytte" verb pret tr1 "utnytte" verb perf-part tr1

29 Statistical disambiguator Uses a statistical model to fully disambiguate Simple model based on existing resources Must discriminate between the ambiguities left by the CG disambiguator

30 Earlier ambiguities - now resolved Setninger " " "setning" subst appell fem ub fl "setning" subst appell mask ub fl

31 Earlier ambiguities - now resolved om " " "om" prep "om" sbu åndsverk " " "åndsverk" subst appell nøyt ub fl "åndsverk" subst appell nøyt ub ent

32 Earlier ambiguities - now resolved gamle " " "gammel" adj be ent pos "gammal" adj be ent pos "gammel" adj fl pos "gammal" adj fl pos

33 Earlier ambiguities - now resolved byen " " "bye" subst appell mask be ent "by" subst appell mask be ent

34 Statistical disambiguation process Statistical tagger is run independently of the CG disambiguator The output is aligned Statistical tagger result used to select among ambiguous results Simple lemma disambiguation

35 HMM modelling Robust performance on smaller amounts of training data Good unknown word handling Cheap and mature

36 Our HMM model Trained on 122 523 words in 8178 sentences Variety of domains More than 350 distinct tags Not very good accuracy really

37 HMM model integration Ambiguities in ca. 4.5% of tokens Coverage ca. 80%

38 Lemma disambiguation Mainly resolved by tag disambiguation But some are still disambiguous

39 Using word form frequencies Idea: lemmas occur as word forms in large corpora Use word frequencies from NoWaC to disambiguate among lemmas

40 Remaining ambiguities Randomly selected

41 Expectations Cheap and cheerful modeling Facing a variety of hard disambiguation decisions On a large morphosyntactic tagset Evaluated on a slightly eclectic corpus

42 Results: CG Disambiguation Precision 96.03% Recall 99.02% F-score 97.2%

43 Results: Full disambiguation Accuracy 96.56%

44 Results: Full disambiguation Overall accuracy 96.56% Tagging accuracy 96.74% Lemma accuracy 98.33%

45 Details Tagger coverage 79.39% Tagger accuracy 81.70% Lemma coverage 54.23% Lemma accuracy 86.71%

46 Forthcoming (technical) Optimizing for very large corpora (> billion words) More sophisticated modeling Discriminative modeling or MBT modeling Constrained decoding Better lemma disambiguation

47 Forthcoming (theoretical) Finding the best division of labor between data driven and rule driven approaches Pivoting on specific errors and ambiguities Working more with syntax (CG3 dependency trees)

48 Links http://tekstlab.uio.no/obt-ny/index.html http://github.com/andrely/OBT-Stat