WCSE

Integrating BERT Model for Document Classification in Document Management System

Abstract— The Document Management Systems (DMS) has become important in various organizations today. This study analyzes the application of Bidirectional Encoder Representations from Transformers (BERT) technology into Document Management Systems (DMS) for more effective and precise classification of memorandum circulars. By leveraging the power of BERT's understanding of natural language to accomplish this task. This system comprises a pre-training model, fine-tuning, and seamless integration into a DMS which simplifies document classification. The used model enables effective document management by classifying huge numbers of documents with the use of data augmentation and stratified kfold sampling. The evaluation results of the study showed marked improvements in accuracy and scalability as opposed to customary techniques. This study presents a methodology for implementing BERT models to automate and improve classification of memorandum circulars in the document management system. The BERT-based document classification model does well in categorizing the memorandum circulars varied in nature and does really have promising results based on their F1-scores of 0.91 indicating high result for some classification. Overall, this study focuses on the model’s effectiveness in automating document classification and contributing to an improved accessibility and efficiency in the document management.

Index Terms— Document Management System, BERT Model, Machine Learning, Document Classification, Natural Language Processing (NLP)

Janelle Kyra Sagum, Jallane Roncales
Polytechnic University of the Philippines, PHILIPPINES

[Download]

Cite: Janelle Kyra Sagum, Jallane Roncales, "Integrating BERT Model for Document Classification in Document Management System", 2025 the 15th International Workshop on Computer Science and Engineering (WCSE 2025), pp.64-69, Jeju Island, South Korea, June 28-30, 2025.