CALCULATION OF A COEFFICIENT OF UNIQUENESS FOR TEXT DOCUMENT USING JACCARD SIMILARITY

Abstract

The rapid development of the Internet, along with the growing computer literacy, is contributing to the penetration of plagiarism in various areas of human activity: plagiarism is an acute problem in education, industry and the scientific community. According to [1], plagiarism is understood as the illegal use or disposal of the protected results of another’s creative work, which is accompanied by bringing to others wrong information about himself as a real author. Plagiarism can be a violation of copyright and patent law and as such can lead to legal liability. On the other hand, plagiarism is possible in areas that are not covered by any type of intellectual property, such as mathematics and other basic scientific disciplines. Plagiarism with the advent of the Internet has become a serious problem. Once on the Internet, knowledge becomes the property of all, it becomes increasingly difficult and sometimes impossible to enforce copyright. Therefore, checking the uniqueness of documents is the important task. The article analyzes modern methods and means of checking textual information for uniqueness. For each of them there are example of work, advantages and disadvantages provided. It is noted that the important task is to increase the accuracy when verifying texts for uniqueness. The shingles method has been identified as the most common and effective method of plagiarizing textual information. Based on the shingles method, an improved algorithm for checking texts for uniqueness using the Jaccard coefficient is proposed. The complexity of the proposed algorithm in terms of memory and processing power was considered. It is noted that with the introduction of additional improvements, the performance of the algorithm has not deteriorated.