soWaC: Somali corpus from the web

The Somali web corpus (soWac) is a Somali corpus made up of texts collected from the Internet. The corpus was prepared according to standards described in the document A Corpus Factory for Many Languages (Kilgarriff et al. at LREC 2010).

Data was crawled by the SpiderLing web spider in January 2016  and comprised of 71 million words.

Document count – the most frequent web domains and domain size distribution:

Top level domains Web domains Second level domain size distribution
net 295,358 22,823 At least 1000 documents 73
org 75,860 22,544 At least 500 documents 96
com 7,397 21,904 At least 100 documents 150
info 4,577 17,079 At least 50 documents 181
so 1,930 15,453 At least 10 documents 352 13,923 At least 5 documents 487 13,693 At least 1 document 1,083 13,203 12,189 10,853 10,196 8,378 8,218 7,045 6,755 6,154 6,097

The content of news/politics and religious sites has a significant presence in the corpus sources.

The corpus was created in the framework of the HaBiT project (Harvesting big text data for under-resourced languages), see more at

Part-of-speech tagset

The soWaC corpus contains POS annotation based on Universal dependencies, a multilingual parser development.

Tools to work with the Somali corpus

A complete set of Sketch Engine tools is available to work with this Somali corpus from the web to generate:

  • word sketch – Somali collocations categorized by grammatical relations
  • thesaurus – synonyms and similar words for every word
  • word lists – lists of Somali nouns, verbs, adjectives etc. organized by frequency
  • n-grams – frequency list of multi-word units
  • concordance – examples in context
  • keywords– terminology extraction of one-word
  • text type analysis – statistics of metadata in the corpus

Corpus factory method

Kilgarriff, A., Reddy, S., Pomikálek, J., & Avinesh, P. V. S. (2010, May). A corpus factory for many languages. In LREC.

Search the Somali corpus

Sketch Engine offers a range of tools to work with this Somali corpus.

Other text corpora

Sketch Engine offers 800+ language corpora.

Use Sketch Engine in minutes

Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.