urTenTen: Corpus of the Urdu Web
The Urdu Web Corpus (urTenTen) is an Urdu corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.
Data was crawled by the WebBootCat toolkit.
For detailed information about TenTen corpora, see Common TenTen corpora attributes.
The most recent version of the urTenTen corpus consists of 329 million words. The texts were downloaded in March–April 2021. The sample texts of the biggest web domains which account for 88% of all corpus texts were checked semi-manually and content with poor quality text and spam was removed.
Part-of-speech tagset
The urTenTen Urdu 2018 corpus was tagged by Unified Parts of Speech (POS) Standard in Indian Languages (accessible via the Wayback Machine). The urTenTen Urdu 2021 corpus uses Universal Dependencies tagset.
Overview of Urdu TenTen corpora
This is a list of Urdu Web corpora available in Sketch Engine:
- Urdu Web corpus 2021 (urTenTen21) – 329 million words, topic and genre classification
- Urdu Web corpus 2018 (urTenTen18) – 245 million words
Urdu Web 2021 corpus sizes
| Number of words | 328+ million |
| Number of tokens | 356+ million |
| Number of sentences | 13+ million |
| Number of web pages | 936+ thousand |
Tools to work with the Urdu corpus
A complete set of tools is available to work with this Urdu corpus to generate:
- word sketch – Urdu collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word units
- word lists – lists of Urdu nouns, verbs, adjectives, etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Search the Urdu corpus
Sketch Engine offers a range of tools to work with this Urdu corpus from the web.
Use Sketch Engine in minutes
Generating collocations, frequency lists, examples in contexts, n-grams or extracting terms. Use our Quick Start Guide to learn it in minutes.
