How the Spanish-English dictionary is compiled (ctd)

Corpus-based tools

In addition to the "manual" approach mentioned, various corpus-based techniques are also used in compiling the dictionary. The "corpus" in this case consists of a body of Spanish and English texts taken from the Internet, from a variety of sources. Sources are generally chosen from sites that offer RSS feeds (and the texts taken via those feeds), and where those feeds are broadly categorised thematically¹. Note that in practice, this does of course give a bias to the material: sites offering regular high-quality feeds tend to be on-line newspapers and magazines.

The raw HTML files downloaded from the Internet first require quite an aggressive "cleanup" to remove unwanted tags, and to differentiate between useful text and unuseful text such as copyright messages, menus etc. Some HTML tags which make some linguistic sense (e.g. title tags and header) are actually used in parsing and categorising the texts. A fairly "shallow" parsing process is applied to the cleaned-up text with the purpose of reducing declined forms (e.g. plurals, conjugated verbs) to their base form. From this body of material, we can then perform various tasks. (Custom-written software is used for this process.)

• words are highlighted that are commonly occurring in the corpus (either in terms of raw number of instances, or number and variety of texts they occur in), but which have sparse or no treatment in the dictionary;
• words are listed alongside sample sentences containing them from the corpus, and these sample sentences are used as templates for examples in the dictionary;
• common colloquators are also listed with a particular word, and thus compounds are also found for adding to the dictionary;
• based on common patterns of English-Spanish cognates, the program also displays side by side lists of example sentences containing a given Spanish word with sample sentences containing the believed English cognate (for example, the program can list instances of the English word presidential alongside instances of the Spanish word presidencial).

Based on this project's sister French-English dictionary, a similar process to the last point is also performed but taking the English translation of the French cognate of a given Spanish word². This helps to find translations for words where the most appropriate English translation isn't always the cognate. In some cases where the relationship between a French and Spanish cognate pair is no longer very obvious from the orthography, such pairings have actually been marked up "by hand" in the Spanish dictionary data.

Dictionary reversal

The dictionary is generally compiled in the direction Spanish-English. Most of the English-Spanish entries are actually generated by a computer program that "reverses" the Spanish-English entries. This obviously more or less halves the amount of effort required to compile the dictionary, and also avoids certain inconsistencies that can creep in when both halves of a bilingual dictionary are hand-compiled (e.g. a particular expression is included in only one side of the dictionary). Special markers are added in some cases to the Spanish-English entries to help the reverser. For example, "index" markers are added to the English translations of Spanish examples where it is otherwise difficult for the reverser to guess which entries to include the given examples in.

Of course, this automatic reversing process still works best for certain entries. It doesn't work very well for certain key, often functional, words such as articles which require more of a textual explanation in their entry than example phrases. So some entries of the English-Spanish side are actually hand-compiled.

Notes:
1. For example, a newspaper may offer a "sports" feed and an "international news" feed.)
2. Of course, strictly speaking, English "cognates" of a given Spanish/French word are generally borrowings from French!

 Dictionary methodology (part 1)
 Spanish-English dictionary
 Site home