Balakrishnan, Vimala and Lloyd-Yemoh, Ethel (2014) Stemming and lemmatization: A comparison of retrieval performances. In: Proceedings of SCEI Seoul Conferences, 10-11 Apr 2014, Seoul, Korea.
![[img]](http://eprints.um.edu.my/style/images/fileicons/application_pdf.png)
|
PDF
rp030_I3007.pdf - Published Version Download (1MB) |
Abstract
The current study proposes to compare document retrieval precision performances based on language modeling techniques, particularly stemming and lemmatization. Stemming is a procedure to reduce all words with the same stem to a common form whereas lemmatization removes inflectional endings and returns the base or dictionary form of a word. Comparisons were also made between these two techniques with a baseline ranking algorithm (i.e. with no language processing). A search engine was developed and the algorithms were tested based on a test collection. Both mean average precisions and histograms indicate stemming and lemmatization to outperform the baseline algorithm. As for the language modeling techniques, lemmatization produced better precision compared to stemming, however the differences are insignificant. Overall the findings suggest that language modeling techniques improves document retrieval, with lemmatization technique producing the best result.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Funders: | UNSPECIFIED |
| Uncontrolled Keywords: | Document retrieval, language models, lemmatization, stemming |
| Subjects: | T Technology > T Technology (General) |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Mr. Mohd Samsul Ismail |
| Date Deposited: | 30 Apr 2015 01:03 |
| Last Modified: | 08 Jan 2020 07:55 |
| URI: | http://eprints.um.edu.my/id/eprint/13423 |
Actions (login required)
 |
View Item |
