enTenTen: Corpus of the English Web
The English Web Corpus (enTenTen) is an English corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family. Sketch Engine currently provides access to TenTen corpora in more than 40 languages. The corpora are built using technology specialized in collecting only linguistically valuable web content.
For detailed information about TenTen corpora, see Common TenTen corpora attributes.
Part-of-speech tagset
The enTenTen English corpora were tagged by TreeTagger (version 2) using Penn Treebank tagset with Sketch Engine modifications.
Overview of English TenTen corpora
These web corpora were crawled and processed repeatedly during the last ten years:
- English Web corpus 2015 (enTenTen15) – 15 billion words (sophisticated spam removal)
- English Web corpus 2013 (enTenTen13) – 19 billion words
- English Web corpus 2012 (enTenTen12) – 11 billion words
- English Web corpus 2008 (enTenTen08) – 2.7 billion words
enTenTen corpus in detail
The chart shows the distribution of the parts of speech in the English Web corpus 2013.
Further information about texts in the corpus
Basic information
| Frequency | |
| Tokens | 22 729 000 |
| Words | 19 686 000 |
| Sentences | 1 120 000 |
| Web pages | 37 000 |
Distribution of top-level domains
Tools to work with the English Web corpora
A complete set of Sketch Engine tools is available to work with these English corpora to generate:
- word sketch – English collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of English nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
Changelog
English Web 2015 (enTenTen15)
- initial size 28 billion words
v2 (spring 2017)
- 15 billion words
- genre classification
- depth analysis of spam and its removal including too short documents
English Web 2013 (enTenTen13)
- 19 billion words
English Web 2012 (enTenTen12)
version 1 (14 June 2012)
- sample of corpus – 3.7 billion words
- crawled by SpiderLing in May 2012
- encoded in UTF-8
version 2 (2012)
- full corpus – 11 billion words
English Web 2008 (enTenTen08)
version 1 (15 November 2010)
- initial version – 3.3 billion tokens
- crawled by Heritrix in 2008
- encoded in Latin1
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Search the English corpus
Sketch Engine offers a range of tools to work with this English corpus.
or
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.
