A Comparative Analysis of Two Algorithms for TF-IDF-Based Document Similarity

Abstract

By using Term Frequency–Inverse Document Frequency, this study set out to analyze how different algorithms carry out the task of identifying research articles that are considered to be of similar subjects. Two algorithms were implemented in our study, one of which was a Python algorithm based on cosine similarity and the other in C# using standard statistical measures. The goal was to examine how the ranking of documents to the most similar target article can be influenced by factors such as preprocessing, similarity techniques, and programming languages. The dataset contains a total of 50 abstracts, with two versions. One with common words removed and one left unchanged. The 50th abstract was selected as the target document, and since it included 29 references, we evaluated the top 29 results produced by each algorithm. The Python implementation relied on cosine similarity only, whereas the C# implementation used mean, standard deviation, Z-score, and weighted similarity. Performance was assessed using Spearman’s rank correlation and hit ratio. The results showed that removing common words generally improved performance. The best result (75.86%) came from the C# algorithm using statistical measures with common words included. Overall, the findings highlight that algorithm choice and preprocessing decisions significantly affect document similarity outcomes.

Received: 4 September 2025 | Revised: 8 December 2025 | Accepted: 13 July 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

The data that support the findings of this study are openly available in figshare at https://figshare.com/articles/dataset/Article_Abstracts/32834597?file=66102518.

Author Contribution Statement

Glory Tendaishe Muindisi: Methodology, Software, Validation, Formal analysis, Investigation, Writing – review & editing, Visualization. Yusra Nuri: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Edip Senyurek: Conceptualization, Resources, Data curation, Supervision, Project administration.