Main Article Content


The major challenge in the digital era is the management of big data. A substantial share of digitization is taken from the e-learning. In this respect, the current article deals with generation of questions for testing the acquired levels of students’ knowledge. For this purpose, an algorithm for generation of questions for tests with different level of complexity is proposed. The main stage of this algorithm is using of mathematical combinatorial optimization model. Using this model makes it possible to formulate different tasks, whose solutions determine a subset of questions that correspond to different degree of test complexity. Essential part of this model is the use of binary integer variables to determine whether a question will be a part of the test or not. The advantage of the proposed approach is the flexibility to decrease or increase the number of questions used to compose the test preserving the required score in accordance to the particular level of test complexity. The conducted investigations over a year show, that the effect of testing can improve retention of knowledge and lead to improved end results. The applicability of the proposed algorithm along with the formulated mathematical model is demonstrated in a case study on the excerpt of questions from the web programming course. The proposed algorithm could be used for generation of tests with different degree of complexity for other learning contents.


e-Learning Test Generation Combinatorial Optimization Mathematical Model Questions Difficulties

Article Details

Author Biography

Daniela Ivanova Borissova, Institute of Information and Communication Technologies at the Bulgarian Academy of Sciences

Daniela BORISSOVA is associated professor on “Application of the principles and methods of the cybernetics in different scientific areas (engineering)” in the Institute of Information and Communication Technologies at Bulgarian Academy of Sciences, department of Information Processes and Decision Support Systems. She holds academic research degree doctor of science on “Informatics”. She is also a part-time professor at University of Library Studies and Information Technologies. She major research interests are in the e-learning systems and tools, distance education software, computer science, algorithm design, engineering systems modeling and design.

How to Cite
Borissova, D. I., & Keremedchiev, D. (2020). Generation of e-Learning tests with different degree of complexity by combinatorial optimization. Journal of E-Learning and Knowledge Society, 16(2), 17-24.


  1. Aldiab A., Chowdhury H., Kootsookos A., Alam F., & Allhibi H. (2019), Utilization of Learning Management Systems (LMSs) in higher education system: A case review for Saudi Arabia. Energy Procedia, 160, 731-737.
  2. Arif M., Illahi M., Karim A., Shamshirband S., Alam K.A., Farid S., Iqbal S., Buang Z., & Balas V.E. (2015), An architecture of agent-based multi-layer interactive e-learning and e-testing platform. Quality & Quantity, 49(6), 2435-2458.
  3. Arunoprayoch N., Lai Ch.-H. Tho Ph.-D., Liang J.-S., & Yang J.-Ch. (2018), Effects of question types on engagement and performance of programming learning for non-computer science majors. In 7th International Congress on Advanced Applied Informatics, IEEE, Yonago, Japan.
  4. Borissova D. & Keremedchiev D. (2019), Group decision making in evaluation and ranking of students by extended simple multi-attribute rating technique. Cybernetics and Information Technologies, 18(3), 45-56.
  5. Borissova D., & Mustakerov I. (2015), E-learning tool for visualization of shortest paths algorithms. Trends Journal of Sciences Research, 2(3), 84-89.
  6. Borissova D., I. Mustakerov (2009). A framework of multimedia e-learning design for engineering training. In 8th International Conference Advances in Web Based Learning – ICWL’2009, LNCS, Vol. 5686, pp. 88-97. Aachen, Germany.
  7. Bursalioglu O., Luy M., Ates V., & Erguzen A. (2016), Mobile device supported online examination system appropriate to distance learning. International E-Journal of Advances in Education, 2(4), 95-104.
  8. Dipace A., Fazlagic B., & Minerva T. (2019), The Design of a Learning Analytics Dashboard: EduOpen Mooc platform redefinition procedures. Journal of E-Learning and Knowledge Society, 15(3), 29-47.
  9. Garcia R., Falkner K., & Vivian R. (2018), Systematic literature review: Self-regulated learning strategies using e-learning tools for Computer Science. Computers & Education, 123, 150-163.
  10. Hong J.-Ch., Tsai Ch.-R., Hsiao H.-Sh., Chen P.-H., Chu K.-Ch., Gu J., & Sitthiworachart J. (2019), The effect of the “Prediction-observation-quiz-explanation” inquiry-based e-learning model on flow experience in green energy learning. Computers & Education, 133, 127-138.
  11. Kerzic D., Aristovnik A., Tomazevic N. & Umek L. (2018), Evaluating the impact of e-learning on students’ perception of acquired competencies in a university blended learning environment. Journal of e-Learning and Knowledge Society, 14(3), 65-76.
  12. Manca S., Caviglione L., & Raffaghelli J. E. (2016) Big data for social media learning analytics: potentials and challenges. Journal of e-Learning and Knowledge Society, 12(2), 27-39.
  13. Mustakerov I., & Borissova D. (2011), A conceptual approach for development of educational Web-based e-testing system. Expert Systems with Applications, 38(11), 14060-14064.
  14. Mustakerov I., & Borissova D. (2017), A framework for development of e-learning system for computer programming: Application in the C programming language. Journal of e-Learning and Knowledge Society, 13(2), 89-101.
  15. Mustakerov I., Borissova D., Stoyanov B. (2004) Software system for distant education self-testing. Cybernetics and Information Technologies, 4(2), 128-133.
  16. Omar N.D., Hassan H., & Atan H. (2012) Student engagement in online learning: Learners attitude toward e-mentoring. Procedia - Social and Behavioral Sciences, 67, 464-475.
  17. Pavel A.-P., Fruth A., & Neacsu M.-N. (2015), ICT and e-Learning – catalysts for innovation and quality in higher education. Procedia Economics and Finance, 23, 704-711.
  18. Salas-Rueda R. (2018), Use of the Deming cycle to ensure quality in the educational process on mathematics. Ciencia UNEMI, 11(27), 8-19.
  19. Tasci T., Parlak Z., Kibar A., Tasbasi N., Cebeci H.I. (2014), A novel agent-supported academic online examination system. Educational Technology & Society, 17(1), 154-168.
  20. Terzieva V., Paunova-Hubenova, E., Dimitrov S., Boneva Y. (2020), ICT in STEM Education in Bulgaria. In Auer M., Tsiatsos T. (eds) The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing, vol. 916, pp. 801-812,
  21. Tuparov ., Keremedchiev D., Tuparova D., & Stoyanov M. (2018), Gamification and educational computer games in open source learning management systems as a part of assessment. In 17th International Conference Information Technology Based Higher Education and Training (ITHET), pp. 1-5. IEEE, Olhao, Portugal.
  22. Wolf D., Bixby J., Glenn J., & Gardner H. (1991), Chapter 2: To use their minds well: Investigating new forms of student assessment. Review of Research in Education, 17(1), 31-74.