csTenTen: Corpus of the Czech Web
The Czech Web Corpus (csTenTen) is a Czech corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of web corpora built using the same method with a target size of 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.
The most recent version of the csTenTen corpus consists of 5.7 billion words. The texts were downloaded in September 2023 and October 2023, the Wikipedia part was downloaded in October 2023. The sample texts of the largest web domains which account for 82% of all corpus texts were checked semi-manually and content with poor quality text and spam was removed.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Before 2018, the corpus was called czTenTen.
Part-of-speech tagset and lemmatization
The csTenTen corpus was POS annotated by the Majka tool using the following POS tagset. The corpus texts also contain lemmatization when each word form from the corpus is assigned to its base form (lemma).
Genre annotation and topic classification
A part of the Czech Web 2023 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, fiction, legal, news, reference/encyclopedia) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org) and includes the following topics: beauty & fashion, cars & bikes, culture & entertainment, economy finance & business, education, games, health, history, hobbies, home family & children, nature & environment, pets & animals, politics & government, religion, science, sex, sports, technology & IT, and travel & tourism.
- genres cover 18.9% of the corpus, i.e. 1.089 billion tokens
- topic classification covers 18.8% of the corpus, i.e. 1.085 billion tokens
Hover over the chart to display a number of tokens of the particular topic.
Further information about texts in the csTenTen23 corpus
Basic information
Frequency | |
Tokens | 5,762,868,630 |
Words | 4,736,886,621 |
Sentences | 314,253,131 |
Web pages | 15,663,989 |
Preloaded subcorpora
A list of preloaded subcorpora (based on the top-level domains or genre annotation and topic classification) that are available in the Czech Web 2019 corpus.
Subcorpus | Tokens | % of the whole corpus |
Czech Wikipedia 2023 | 188,205,904 | 3.266 |
Czech domain .cz | 4,395,131,286 | 76.266 |
Genre Blog | 15,423,312 | 0.268 |
Genre Discussion | 229,427,215 | 3.981 |
Genre Fiction | 105,152,395 | 1.825 |
Genre Legal | 193,689,324 | 3.361 |
Genre News | 329,946,820 | 5.725 |
Genre Reference/Encyclopedia | 215,652,280 | 3.742 |
Topic Beauty & Fashion | 3,266,557 | 0.057 |
Topic Cars & Bikes | 56,314,282 | 0.977 |
Topic Culture & Entertainment | 185,352,857 | 3.216 |
Topic Economy, Finance & Business | 43,065,497 | 0.747 |
Topic Education | 27,055,630 | 0.469 |
Topic Games | 40,496,599 | 0.703 |
Topic Health | 36,583,850 | 0.635 |
Topic History | 5,454,921 | 0.095 |
Topic Hobbies | 31,121,909 | 0.54 |
Topic Home, Family & Children | 35,067,958 | 0.609 |
Nature & Environment | 28,708,670 | 0.498 |
Topic Pets & Animals | 10,694,688 | 0.186 |
Topic Politics & Government | 79,613,402 | 1.381 |
Topic Religion | 21,346,014 | 0.37 |
Topic Science | 27,009,776 | 0.469 |
Topic Sex | 1,843,665 | 0.032 |
Topic Sports | 101,962,216 | 1.769 |
Topic Technology & IT | 152,213,016 | 2.641 |
Topic Travel & Tourism | 10,252,441 | 0.178 |
Overview of Czech TenTen corpora
Czech web corpora csTenTen were crawled and processed repeatedly during the last years:
- Czech Web corpus 2023 (csTenTen23) – 5.7 billion words, topic classification and genre annotation, semi-manual check of corpus texts
- Czech Web corpus 2019 (csTenTen19) – 6.2 billion words, topic classification and genre annotation, semi-manual check of corpus texts
- Czech Web corpus 2017 (csTenTen17) – 10.5 billion words (subcorpora: 2017, 2016, 2015 years and Wikipedia texts)
- Czech Web corpus 2012 (csTenTen12) – 4 billion words
Tools to work with the Czech corpora from the web
A complete set of Sketch Engine tools is available to work with this Czech corpus to generate:
- word sketch – Czech collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Czech nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
csTenten23 (January 2024)
- 5.7 billion tokens
- topic and genre classification
csTenTen19 (May 2023)
- 7.6 billion tokens
- genre and topic classification
new version of the csTenTen corpus from 2017 (November 2018)
- crawled by SpiderLing in May, October and November 2017, October and November 2016, October and November 2015 + Czech Wikipedia from November 2017
- Encoded in UTF-8, cleaned, deduplicated, tagged using majka+desamb pipeline v2
version 9 (2018-10-19)
- renamed from czTenTen to csTenTen
version 9 (2017-05-20)
- added lempos (a combination of lemma and one-letter abbreviation of the part of speech, e.g. dům-n)
version 8 (2014-09-17)
- M ? j removed
Thanks to Marek Grác for spotting much errors and contributing to a cleaner corpus.
version 7 (2014-08-04)
- Paragraphs without accents removed.
version 6 (June 2014)
- Machine translated documents from domains infostar.cz and navajo.cz removed.
version 5 (May 2014)
- Malformed vertical lines corrected (MacLeodovy MacL eodůvk2eAgFnPc1d1 –> MacLeodovy MacLeodův k2eAgFnPc1d1).
version 4 “clean 2” (March 2014)
- Documents containing a certain wrong character caused by wrong encoding detection were removed.
version 3 “clean” (2013)
- Paragraphs containing more than 20 % of words not recognized by morphological analyzer Majka were removed.
version 2 (December 2012)
- retagged, corrected ‒ the updated tagset can be found in Miloš Jakubíček, Vojtěch Kovář and Pavel Šmerk. Czech Morphological Tagset Revisited. In 5th Workshop on Recent Advances in Slavonic Natural Language Processing. Brno, 2011, pp. 29–42.
- word sketches
version 1 (September 2012)
- tagged by Majka + Desamb
version 1 untagged (April 2012)
- initial version – 4.8 G words
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
csTenTen corpus
Suchomel, Vít (2018). csTenTen17, a Recent Czech Web Corpus. In Twelveth Workshop on Recent Advances in Slavonic Natural Language Processing. Brno: Tribun EU, 2018. pp. 111–123.
Suchomel, Vít (2012). Recent Czech Web Corpora. In 6th Workshop on Recent Advances in Slavonic Natural Language Processing. Brno, pp. 77–83.
Search the Czech corpus csTenTen
Sketch Engine offers a range of tools to work with this Czech corpus.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract. Use our Quick Start Guide to learn it in minutes.