INFORMATION-ANALYTICAL SYSTEM FOR EVALUATING THE SCIENTIFIC PERFORMANCE OF STRUCTURAL UNITS OF UNIVERSITIES AND RESEARCH INSTITUTES BASED ON THE APPROACH OF CONSTRUCTING COMPLEX INTEGRAL EVALUATION
DOI:
https://doi.org/10.37943/XTPK5061Keywords:
: information-analytical systems, microservices, ratings of scientists, rating of scientific departments, integrated performance assessment, site parserAbstract
The article discusses the creation of an information-analytical system for evaluating the scientific performance of structural units of universities and research institutes based on the approach of constructing complex integral evaluation. A model of information technology for evaluating the results of scientific activity is proposed, consisting of four modules: an information collection module, an information storage module, an analytical module, and a module for user interaction and data visualization. The modular structure of the technology will allow expanding and modifying the capabilities of each of the modules independently of the others, as well as increasing the stability and flexibility of the technology. The implementation of this system is performed using microservices technology. A conceptual model of the information system and a structural model of the functioning of the information collection module, as well as a structural model of the information system database, are proposed.
It is shown that most of the well-known indices for evaluating the performance of subjects of scientific activity, for example, h-index, g-index, e-index, I-10 index, etc., do not fully take into account information about citation. Therefore, the method for calculating the evaluation of scientific research activities of scientists was proposed, which does not lose information about any citation of the author and publication. This method determines the scalar evaluation of the results of scientific activity, and it is based on determining a few coefficients. The coefficients define one scientist's citation in the publications of other scientists. As a result, assessment is obtained by solving a system of linear algebraic equations that are constructed based on calculated coefficients. Most of the known evaluation approaches have their own calculation features and disadvantages, which are associated with the loss of some information. Therefore, it is not recommended to give preference to one of them. For the purposes of a comprehensive assessment of the productivity of research activities of scientists, the authors proposed a method of vector evaluation of the results and the construction of the integral assessment. This method is based on the construction of vectors and scalar estimates for each scientist in a multidimensional metric space. The dimensionality of the space is determined by the number of calculated scalar estimates. The method is also based on the construction of an ideal point, which consists of scalar estimates that are the best in terms of achieving maximum performance. The assessment of each subject of scientific activity is calculated as the metric distance from the ideal point to the vector of scalar estimates of this subject of scientific activity.
References
Biloshchytskyi, A., Kuchansky, A., Paliy, S., Biloshchytska, S., Bronin, S., Andrashko, Yu., Shabala, Ye., & Vatskel, V. (2018). Development of technical component of the methodology for project-vector management of educational environments. Eastern-European Journal of Enterprise Technologies, 2/2 (92), 4-13. https://doi.org/10.15587/1729-4061.2018.126301
Otradskaya, T., Gogunskii, V., Antoschuk, S., & Kolesnikov, O. (2016). Development of parametric model of prediction and evaluation of the quality level of educational institutions. Eastern European Journal of Enterprise Technologies, 5, 3 (83), 12–21. https://doi.org/10.15587/1729-4061.2016.80790
Otradskaya, T., & Gogunskii, V. (2016). Development process models for evaluation of performance of the educational establishments. Eastern-European Journal of Enterprise Technologies, 3 (81), 12 – 21. https://doi.org/10.15587/1729-4061.2016.66562
Biloshchytskyi, A., Andrashko, Yu., Kuchansky, A., Biloshchytska, S., Kuzka, O., & Terentyev, O. (2017). Evaluation methods of the results of scientific research activity of scientists based on the analysis of publication citations. Eastern-European Journal of Enterprise Technologies, 3/2 (87), 4 – 10. https://doi.org/10.15587/1729-4061.2017.103651
Biloshchytskyi, A., Myronov, V., Reznik, R., Kuchansky, A., Andrashko, Yu., Paliy, S., & Biloshchytska, S. A. (2017). Method to evaluate the scientific activity quality of HEIs based on a scientometric subjects presentation model. Eastern-European Journal of Enterprise Technologies, 6/2 (90), 16 – 22. https://doi.org/10.15587/1729-4061.2017.118377
Biloshchytskyi, A., Kuchansky, A., Andrashko, Y., Mukhatayev, A., Toxanov, S., & Faizullin, A. (2020, November). Methods of Assessing the Scientific Activity of Scientists and Higher Education Institutions. In 2020 IEEE 2nd International Conference on Advanced Trends in Information Theory (ATIT) (pp. 162-167). IEEE. https://doi.org/10.1109/atit50783.2020.9349348
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. PNAS, 102 (46), 16569 – 16572. https://doi.org/10.1073/pnas.0507655102
Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69 (1), 131-152. https://doi.org/10.1007/s11192-006-0144-7
Hirsch, J. (2019). hα: An index to quantify an individual’s scientific leadership. Scientometrics. https://doi.org/10.1007/s11192-018-2994-1.
Page, L., Brin, S., Motwani, R., & Winograd, T. (1998). The PageRank Citation Ranking: Bringing Order to the Web. Proceedings of the 7th International World Wide Web Conference, Brisbane, Australia, 161 – 172.
Avrachenkov, K., Litvak, N., Nemirovsky, D., & Osipova, N. (2007). Monte Carlo methods in PageRank computation: When one iteration is sufficient. SIAM Journal on Numerical Analysis, 45(2), 890-904. https://doi.org/10.1137/050643799
Liao, Q., Jiang, S., Yu, M., Yang, Y., & Li, T. (2017, May). Monte Carlo based incremental PageRank on evolving graphs. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 356-367). Springer, Cham. https://doi.org/10.1007/978-3-319-57454-7_28
Kuchansky, A., Biloshchytskyi, A., Andrashko, Yu., Biloshchytska, S., Shabala, Ye., & Myronov, O. (2018). Development of adaptive combined models for predicting time series based on similarity identification. Eastern-European Journal of Enterprise Technologies, 1/4 (91), 25 – 28. https://doi.org/10.15587/1729-4061.2018.121620
QS World Ratings. (2017). Retrieved January 26, 2018, from https://www.topuniversities.com/qs-world-university-rankings
Academic Ranking of World Universities. Retrieved January 26, 2018, from http://www.shanghairanking.com/ARWU2017.html
Ranking Web of Universities. (2017). Retrieved January 26, 2018, from http://www.webometrics.info/en/Europe/Ukraine
Sreeramana, A. P., & Kumar, P. M. (2016). ABC Model of Research Productivity and Higher Educational Institutional Ranking. International Journal of Education and Management Engineering, 6, 74–84. https://doi.org/10.5815/ijeme.2016.06.08
Michelucci, D., & Foufou, S. (2004). Using Cayley-Menger determinants for geometric constraint solving. Proceedings of the 9nth ACM symposium on Solid modeling and applications. Eurographics Association, 285–290.
Lizunov, P. P., Biloshchytskyi, A. O., Chala, L. E., Biloshchytska, S. V., & Kuchanskyi, O. Yu. (2016). Avtomaticheskij analiz shodstva shem i diagramm v jelektronnyh tekstovyh dokumentah. Upravlenie razvitiem slozhnyh system [Automatic analysis of similarities of schemes and diagrams in electronic text documents. Management of complex systems development], (28), 160-169.
Watkins, D. S. (2004). Fundamentals of matrix computations. John Wiley & Sons.
Thurau, C., Kersting, K., & Bauckhage, C. (2010, October). Yes we can: simplex volume maximization for descriptive web-scale matrix factorization. In Proceedings of the 19th ACM international conference on information and knowledge management (pp. 1785-1788).
Morozov, V., Kalnichenko, O., & Liubyma, I. (2017, July). Managing projects configuration in development distributed information systems. In 2017 2nd International Conference on Advanced Information and Communication Technologies (AICT) (pp. 154-157). IEEE.. https://doi.org/10.1109/aiact.2017.8020088
Kuchansky, A., Andrashko, Yu., Biloshchytskyi, A., Danchenko, O., Ilarionov, O., Vatskel, I., & Honcharenko, T. (2018). The method for evaluation of educational environment subjects’ performance based on the calculation of volumes of m-simplexes. Eastern-European Journal of Enterprise Technologies, 2/4 (92), 15–25. https://doi.org/10.15587/1729-4061.2018.126287
Kuchansky, A. (2020). Zadacha Identifikacii Predmetnyh Nauchnyh Prostranstv. Upravlenie razvitiem slozhnyh system [The problem of identification of subject scientific spaces. Management of complex systems development], (42), 70 – 78.
Biloshchitsky, A. A., Lizunov, P. P., Kuchinsky, A. Y., Andrushko, Y. V., & Beloshitskaya, S. (2017). Metodologicheskie osnovy sozdanija informacionnoj sredy upravlenija nauchnymi issledovanijami: monografija. [Methodological bases of creation of information environment of scientific research management: a monograph].
Andrashko, Y. V., Biloshchytskyi, A. O., & Kuchanskyi, O. Yu. (2017). Obzor metodov ocenki rezul'tatov dejatel'nosti nauchno-pedagogicheskih rabotnikov vysshih uchebnyh zavedenij. Upravlenie razvitiem slozhnyh system. [Review of methods for evaluating the performance of research and teaching staff and higher education institutions. Management of complex systems development], (29), 151-159.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Articles are open access under the Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish a manuscript in this journal agree to the following terms:
- The authors reserve the right to authorship of their work and transfer to the journal the right of first publication under the terms of the Creative Commons Attribution License, which allows others to freely distribute the published work with a mandatory link to the the original work and the first publication of the work in this journal.
- Authors have the right to conclude independent additional agreements that relate to the non-exclusive distribution of the work in the form in which it was published by this journal (for example, to post the work in the electronic repository of the institution or publish as part of a monograph), providing the link to the first publication of the work in this journal.
- Other terms stated in the Copyright Agreement.