bnWaC: Bengali corpus from the web

The Bangla web Corpus (bnWaC) is a Bengali corpus made up of texts collected from the Internet. The corpus was prepared by the Corpus factory method, a method for developing large general language corpora which can be applied to many languages (A. Kilgarriff et al 2010). The Bengali Web corpus contains 11.5 million words. The corpus texts are available with lemmatization and POS tagging.

Part-of-speech tagset

This Bangla corpus was processed by an annotation tool developed by Microsoft Research Lab – India using this Bengali part-of-speech tagset.

Tools work with the Bengali corpus

A complete set of Sketch Engine tools is available to work with this Bengali web corpus to generate:

  • word sketch – Bengali collocations categorized by grammatical relations
  • thesaurus – synonyms and similar words for every word
  • keywords – terminology extraction of one-word units
  • word lists – lists of Bengali nouns, verbs, adjectives etc. organized by frequency
  • n-grams – frequency list of multi-word units
  • concordance – examples in context
  • text type analysis – statistics of metadata in the corpus

version 2 (2017 April)

  • applied new tokenizer and lemmatizer
  • corpus tagged using an Annotation Tool developed by Microsoft Research Lab in India

(2012)

  • shallow tagging (no part-of-speech classification)
  • universal word sketch grammar

version 1 2010

  • initial version

Indian Language Part-of-Speech Tagset: Bengali

Bali, Kalika, Monojit Choudhury, and Priyanka Biswas. Indian Language Part-of-Speech Tagset: Bengali LDC2010T16. Web Download. Philadelphia: Linguistic Data Consortium, 2010.

Web corpora

BARONI, Marco; KILGARRIFF, Adam. Large linguistically-processed web corpora for multiple languages. In: Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations. Association for Computational Linguistics, 2006, pp. 87–90.

Kilgarriff, A., Reddy, S., Pomikálek, J., & Avinesh, P. V. S. A corpus factory for many languages. In LREC, May 2010.

Search the Bengali corpus

Sketch Engine offers a range of tools to work with this Bengali corpus of web texts.

Concordance from Bengali corpus.

Other text corpora

Sketch Engine offers 800+ language corpora.

Use Sketch Engine in minutes

Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.