An Evaluation of Sentence Similarity using The Adaptive N-Gram Method
This study proposes the method of evaluating the similarity of sentences using the adaptive n-gram method.
Under the suggested method, a given document is separated into sentences and then the morpheme in each sentence is
analyzed. Among the sentences having morpheme, only the morpheme best expressing the meaning of the sentence is
extracted and then, from the extracted morpheme, the core words and the n related words are automatically selected to make
the gram. If there is any sentence using the same core words as in the comparative documents, the grams are extracted from
that sentence and compared with each other to get the degree of similarity. The similarity result from the sentence is printed
out in the form of MDM for the intuitive evaluation of similarity between sentences. In addition, it shows the evaluation
made by context and meaning of word unlike word2vec, which cannot know why two of sentences are similar.
Keywords – Adaptive N-gram, MDM, Sentence Similarity, Word2Vec