csTenTen: Corpus of the Czech Web
The Czech Web Corpus (csTenTen) is a Czech corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of web corpora built using the same method with a target size of 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.
The data was crawled by web crawler SpiderLing in May, October and November 2017, October and November 2016, October and November 2015. The Czech Wikipedia part was downloaded in November 2017. The size of the corpus is 10 billion words.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Before 2018, the corpus was called czTenTen.
Part-of-speech tagset
The csTenTen corpus was POS annotated by the Majka tool using the following POS tagset.
Overview of Czech TenTen corpora
Czech web corpora csTenTen were crawled and processed repeatedly during the last years:
- Czech Web corpus 2017 (csTenTen17) – 10.5 billion words (subcorpora: 2017, 2016, 2015 years and Wikipedia texts)
- Czech Web corpus 2012 (csTenTen12) – 4 billion words
Tools to work with the Czech Web corpus
A complete set of Sketch Engine tools is available to work with this Czech corpus to generate:
- word sketch – Czech collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Czech nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
new version of the csTenTen corpus from 2017 (November 2018)
- crawled by SpiderLing in May, October and November 2017, October and November 2016, October and November 2015 + Czech Wikipedia from November 2017
- Encoded in UTF-8, cleaned, deduplicated, tagged using majka+desamb pipeline v2
version 9 (2018-10-19)
- renamed from czTenTen to csTenTen
version 9 (2017-05-20)
- added lempos (a combination of lemma and one-letter abbreviation of the part of speech, e.g. dům-n)
version 8 (2014-09-17)
- M ? j removed
Thanks to Marek Grác for spotting much errors and contributing to a cleaner corpus.
version 7 (2014-08-04)
- Paragraphs without accents removed.
version 6 (June 2014)
- Machine translated documents from domains infostar.cz and navajo.cz removed.
version 5 (May 2014)
- Malformed vertical lines corrected (MacLeodovy MacL eodůvk2eAgFnPc1d1 –> MacLeodovy MacLeodův k2eAgFnPc1d1).
version 4 “clean 2” (March 2014)
- Documents containing a certain wrong character caused by wrong encoding detection were removed.
version 3 “clean” (2013)
- Paragraphs containing more than 20 % of words not recognized by morphological analyzer Majka were removed.
version 2 (December 2012)
- retagged, corrected ‒ the updated tagset can be found in Miloš Jakubíček, Vojtěch Kovář and Pavel Šmerk. Czech Morphological Tagset Revisited. In 5th Workshop on Recent Advances in Slavonic Natural Language Processing. Brno, 2011, pp. 29–42.
- word sketches
version 1 (September 2012)
- tagged by Majka + Desamb
version 1 untagged (April 2012)
- initial version – 4.8 G words
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
csTenTen corpus
Suchomel, Vít (2018). csTenTen17, a Recent Czech Web Corpus. In Twelveth Workshop on Recent Advances in Slavonic Natural Language Processing. Brno: Tribun EU, 2018. pp. 111–123.
Suchomel, Vít (2012). Recent Czech Web Corpora. In 6th Workshop on Recent Advances in Slavonic Natural Language Processing. Brno, pp. 77–83.
Search the Czech corpus
Sketch Engine offers a range of tools to work with this Czech corpus.
or
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract. Use our Quick Start Guide to learn it in minutes.