Moby Project


The Moby Project is a collection of public-domain lexical resources. It was created by Grady Ward. The resources were dedicated to the public domain, and are now mirrored at Project Gutenberg., it contains the largest free phonetic database, with 177,267 words and corresponding pronunciations.

Hyphenator

The Moby Hyphenator II contains hyphenations of 187,175 words and phrases. The character encoding appears to be MacRoman, and hyphenation is indicated by a bullet. Some entries, however, have a combination of actual hyphens and character 165, such as "".
There is little to no documentation of the hyphenation choices made; the following examples might give some flavour of the style of hyphenation used:.

Language

Moby Language II contains wordlists of five languages: French, German, Italian, Japanese, and Spanish:
LanguageWordsSize
French138,2571,524,757
German159,8092,055,986
Italian60,453561,981
Japanese115,523934,783
Spanish86,059850,523
Total560,1015,928,030

However, some of the lists are contaminated, for example the Japanese list contains English words such as abnormal and non-words such as ' and '. There are also unusual peculiarities in the sorting of these lists, as the French list contains a straight alphabetical listing, while the German list contains the alphabetical listing of traditionally capitalized words and then the alphabetical listing of traditionally lower-cased words. The list of Italian words, however, contains no capitalized words whatsoever.
The foreign languages list does not use accented characters, so "" is how a user would look up the French word être''.

Part-of-Speech

Moby Part-of-Speech contains 233,356 words fully described by part of speech, listed in priority order. The format of the file is word\parts-of-speech, with the following parts of speech being identified:
Part-of-speechCode
NounN
Pluralp
Noun phraseh
Verb V
Transitive verbt
Intransitive verbi
AdjectiveA
Adverbv
ConjunctionC
PrepositionP
Interjection!
Pronounr
Definite articleD
Indefinite articleI
Nominativeo

Pronunciator

The Moby Pronunciator II contains 177,267 entries with corresponding pronunciations. Most of the entries describe a single word, but approximately 79,000 contain hyphenated or multiple word phrases, names, or lexemes. The Project Gutenberg distribution also contains a copy of the cmudict v0.3. The file contains lines of the format word pronunciation. Each line is ended with the ASCII Carriage Return character.
The word field can include apostrophes, hyphens, and multiple words separated by underscores. Non-English words are generally rendered, as stated in the documentation, without accents or other diacritical marks. However, in 36 entries, some non-ASCII accented characters remain, represented using Mac OS Roman encoding.
The part-of-speech field is used to disambiguate 770 of the words which have differing pronunciations depending on their part-of-speech. For example, for the words spelled close, the verb has the pronunciation, whereas the adjective is. The parts-of-speech have been assigned the following codes:
Part-of-speechCode
Nounn
Verbv
Adjectiveaj
Adverbav
Interjectioninterj

Following this is the pronunciation. Several special symbols are present:
SymbolMeaning
_Used to separate words
'Primary stress on the following syllable
,Secondary stress on the following syllable

The rest of the symbols are used to represent IPA characters. The pronunciations are generally consistent with a General American dialect of English, that exhibits father-bother merger, hurry-furry merger and lot-cloth split, but does not exhibit cot-caught merger or wine-whine merger. Each phoneme is represented by a sequence of one or more characters. Some of the sequences are delimited with a slash character "/", as shown in the following table, but note that the sequence for is delimited by two slash characters at either end:
Symbol
/&/æ
/-/ə
/@/ʌ, ə
//rɜr, ər
/A/ɑ, ɑː
/aI/
/AU/
bb
dd
/D/ð
/dZ/
/E/ɛ
/eI/
ff
gɡ
hh
hwhw
/i/
/I/ɪ
/j/j
/ju/juː
kk
ll
mm
nn
/N/ŋ
/O/ɔ, ɔː
//Oi//ɔɪ
/oU/
pp
rr
ss
/S/ʃ
tt
/T/θ
/tS/
/u/
/U/ʊ
vv
ww
zz
/Z/ʒ

To this collection are added a number of extra sequences representing phonemes found in several other languages. These are used to encode the non-English words, phrases and names that are included in the database. The following table contains these extra phonemes, but note that the extent to which some of these may exist due to encoding errors is not clear.
Symbol
Aa
ee, ɛ
ii, ɪ
NNasalisation of preceding vowel
oo
O
Rʁ
Ss
uu
Vv, β, ʋ
Ww
/x/x
/y/ø
Yy
/z/ts
Zz

Shakespeare

Moby Shakespeare contains the complete unabridged works of Shakespeare. This specific resource is not available from Project Gutenberg.

Thesaurus

The Moby Thesaurus II contains 30,260 root words, with 2,520,264 synonyms and related terms – an average of 83.3 per root word. Each line consists of a list of comma-separated values, with the first term being the root word, and all following words being related terms.
Grady Ward placed this thesaurus in the public domain in 1996. It is also available as a Debian package.

Words

Moby Words II is the largest wordlist in the world. The distribution consists of the following 16 files:
FilenameWordsDescription
ACRONYMS.TXT6,213Common acronyms and abbreviations
COMMON.TXT74,550Common words present in two or more published dictionaries
COMPOUND.TXT256,772Phrases, proper nouns, and acronyms not included in the common words file
CROSSWD.TXT113,809Words included in the first edition of the Official Scrabble Players Dictionary
CRSWD-D.TXT4,160Additions to the Official Scrabble Players Dictionary in the second edition
FICTION.TXT467A list of the most commonly occurring substrings in the book The Joy Luck Club
FREQ.TXT1,000Most frequently occurring words in the English language, listed in descending order
FREQ-INT.TXT1,000Most frequently occurring words on Usenet in 1992, listed with corresponding percentage in decreasing order
KJVFREQ.TXT1,185Most frequently occurring substrings in the King James Version of the Bible, listed in descending order
NAMES.TXT21,986Most common names used in the United States and Great Britain
NAMES-F.TXT4,946Common English female names
NAMES-M.TXT3,897Common English male names
OFTENMIS.TXT366Most common misspelled English words
PLACES.TXT10,196Place names in the United States
SINGLE.TXT354,984Single words excluding proper nouns, acronyms, compound words and phrases, but including archaic words and significant variant spellings
USACONST.TXT7,618United States Constitution including all amendments current to 1993
Total863,149Not the total of unique words.
Total Uniq639,995Total of single, proper nouns, acronyms, and compound words and phrases.