daTenTen: Corpus of the Danish Web
The Danish Web Corpus (daTenTen) is a Danish corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 30 languages.
The data for the last version of the Danish Web corpus was crawled by web spider SpiderLing between June and August 2020. It is comprised of more than 3.4 billion words.
The corpus possesses common TenTen corpora attributes.
The old version of Danish Web 2014 was tagged by CST’s TaggerXML with the following tagset.
A complete set of tools is available to work with this Danish corpus to generate:
- word sketch – Danish collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Danish nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
daTenTen20 (May 2021)
- crawled new data by SpiderLing between June and August 2020 – 4.1+ billion token in total
daTenTen17 (December 2017)
- crawled new data – 2.1 billion words in total
- Norwegian (Bokmal or Nynorsk), Swedish, English and German paragraphs were filtered out
datenten17_tt2_2 (May 2020)
- Documents from manually identified bad web domains were removed.
- Documents with the character “Ã” were removed.
datenten14 (12 March 2014)
- daTenTen14 – 2.4 billion tokens
- crawled by SpiderLing in January 2014
version 1 (1 July 2014)
- lempos added
- better cleaning of messy data and tagger mistake
version 2 (4 September 2014)
- Norwegian filtered out
version 3 (March 2015)
- POS tags with slashes corrected, unwanted HTML tags filtered out
version 4 (April 2015)
- Tagging of word “Big” corrected to tag “EGEN”
version 5 (October 2017)
- new lemmatizing and tagging with ePos tagger
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43)
Search the Danish corpus
Sketch Engine offers a range of tools to work with this Danish corpus.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.