Manually Annotated Sub-Corpus
MASC is a balanced subset of 500K words of written texts and transcribed speech drawn primarily from the Open American National Corpus. The OANC is a 15 million word corpus of American English produced since 1990, all of which is in the public domain or otherwise free of usage and redistribution restrictions.
All of MASC includes manually validated annotations for logical structure, sentence boundaries, three different tokenizations with associated part of speech tags, shallow parse, named entities, and syntax. Additional manually produced or validated annotations have been produced by the MASC project for portions of the sub-corpus, including full-text annotation for FrameNet frame elements and a 100K+ sentence corpus with WordNet 3.1 sense tags, of which one-tenth are also annotated for FrameNet frame elements. Annotations of all or portions of the sub-corpus for a wide variety of other linguistic phenomena have been contributed by other projects, including PropBank, TimeBank, , and several others. Co-reference annotations and clause boundaries of the entire MASC corpus are scheduled to be released by the end of 2016.
WordNet sense annotations for all occurrences of 114 words are also included in the MASC distribution, as well as FrameNet annotations for 50-100 occurrences of each of the 114 words. The sentences with WordNet and FrameNet annotations are also distributed as a part of the .
Genres
Unlike most freely available corpora including a wide variety of linguistic annotations, MASC contains a balanced selection of texts from a broad range of genres:| Genre | No. files | No. words | Pct corpus |
| Court transcript | 2 | 30052 | 6% |
| Debate transcript | 2 | 32325 | 6% |
| 78 | 27642 | 6% | |
| Essay | 7 | 25590 | 5% |
| Fiction | 5 | 31518 | 6% |
| Gov’t documents | 5 | 24578 | 5% |
| Journal | 10 | 25635 | 5% |
| Letters | 40 | 23325 | 5% |
| Newspaper | 41 | 23545 | 5% |
| Non-fiction | 4 | 25182 | 5% |
| Spoken | 11 | 25783 | 5% |
| Technical | 8 | 27895 | 6% |
| Travel guides | 7 | 26708 | 5% |
| 2 | 24180 | 5% | |
| Blog | 21 | 28199 | 6% |
| Ficlets | 5 | 26299 | 5% |
| Movie script | 2 | 28240 | 6% |
| Spam | 110 | 23490 | 5% |
| Jokes | 16 | 26582 | 5% |
| TOTAL | 376 | 506768 |
Annotations
At present, MASC includes seventeen different types of linguistic annotation :| Annotation type | No. words |
| Logical | 506768 |
| Token | 506768 |
| Sentence | 506768 |
| POS/lemma | 506768 |
| POS | 506768 |
| POS | 506768 |
| Noun chunks | 506768 |
| Verb chunks | 506768 |
| Named Entities | 506768 |
| Penn Treebank syntax | 506768 |
| Coreference | *506768 |
| Clause boundaries, nucleus/satellite distinctions, discourse markers | *506768 |
| FrameNet frames/frame elements | 39160 |
| PropBank | **88530 |
| Opinion | 51243 |
| TimeBank | *55599 |
| Committed Belief | 4614 |
| Event | 4614 |
| Dependency treebank | **5434 |
| Lexical substitution | **35,547 |
All MASC annotations, whether contributed or produced in-house, are transduced to the Graph Annotation Format defined by ISO TC37 SC4’s Linguistic Annotation Framework.
The online tool can transduce annotations over all or parts of MASC to any of several other formats, including CONLL IOB format and formats for use in UIMA and General Architecture for Text Engineering.
Distribution
MASC is an open data resource that can be used by anyone for any purpose. At the same time, it is a collaborative community resource that is sustained by community contributions of annotations and derived data. It is freely downloadable from the or through the Linguistic Data Consortium.MASC is also distributed in part-of-speech-tagged form with the Natural Language Toolkit.