3.10 Problem with off-the-shelf stopwords

  • Most premade stop word lists assume that all the words are spelled correctly.

  • Handling misspellings when using premade lists can be done by manually adding common misspellings

  • Manually and automatically (Chinese has both and non has been accepted as a standard).