IMPLEMENTATION OF THE ALGEBRA OF HYPERDUAL NUMBERS IN NEURAL NETWORKS
DOI:
https://doi.org/10.37943/20ERZJ2964Keywords:
dual numbers, hyperdual numbers, automatic differentiation, Taylor series expansionAbstract
For the numerical solution of problems arising in various fields of mathematics and mechanics, it is often necessary to determine the values of derivatives included in the model. Currently, numerical values of derivatives can be obtained using automatic differentiation libraries in many programming languages. This paper discusses the use of the Python programming language, which is widely used in the scientific community. It should be noted that the principles of automatic differentiation are not related to numerical or symbolic differentiation methods. The work consists of three parts. The introduction reviews the historical development of the general theory of complex numbers and the use of simple complex, double and dual numbers, which are a subset of the set of general complex numbers, in various fields of mathematics. The second part is devoted to the algebra of dual and hyperdual numbers and their properties. This section presents tables of the basis element of elementary functions with dual and hyperdual arguments, based on multiplication rules. Two important formulas for finding the numerical values of a complex function's first and second derivatives by expanding functions with dual and hyperdual arguments in the Taylor series are also obtained. A simple test function was used to verify the correctness of these formulas, the results of which were checked analytically as well as through implementation in a programming language. The third part of the paper focuses on practical applications and the implementation of these methods in Python. It includes detailed examples of case studies demonstrating the effectiveness of using hyperdual numbers in automatic differentiation. The results highlight the accuracy and computational efficiency of these methods, making them valuable tools for researchers and engineers. This comprehensive approach not only validates the theoretical aspects but also showcases the practical utility of dual and hyperdual numbers in solving complex mathematical and mechanical problems.
References
Clifford, W. K. (1873). Preliminary sketch of biquaternions. Proceedings of the London Mathematical Society, 4(64), 381-395.
Cayley, A. (1859). A sixth memoir upon quantis. Philosophical Transactions of the Royal Society of London, 149, 61-90.
Klein, F. (1985). Ueber die sogenannte Nicht-Euklidische. Springer-Verlag Wien, Teubner-Archiv zur Mathematik, 4, 224-238.
Fike, J. A. (2009). Numerically exact derivative calculations using hyper-dual numbers. 3rd Annual Student Joint Engineering and Design.
Fike, J. A., & Alonso, J. J. (2011). The development of hyper-dual numbers for exact second-derivative calculations. Meeting Including the New Horizons Forum and Aerospace Exposition, 4-7.
Fike, J. A., & Alonso, J. J. (2011). Automatic differentiation through the use of hyper-dual numbers for second derivatives. Computational Science and Engineering, 87(201), 163-173.
Fike, J. A., Jongsma, S., Alonso, J. J., & van der Weida, E. (2011). Optimization with gradient and Hessian information using hyper-dual numbers. 29th AIAA Applied Aerodynamics Conference.
Neuenhofen, M. P. (2018). Review of theory and implementation of hyper-dual numbers for first and second order automatic differentiation. arXiv. https://doi.org/10.48550/arXiv.1801.03614
Mamonov, S. & Peterson, R. (2021). The role of IT in organizational innovation – A systematic literature review. Journal of Strategic Information Systems, 30(4), 101696.
Mamonov, S. & Peterson, R. (2019). The role of IT in innovation at the individual and group level – a literature review. Journal of Small Business and Enterprise Development, 26(6-7), 797-810.
Boutet, A., Haile, S. S., Yang, A. Z., Son, H. J., Malik, M., Pai, V., Nasralla, M., Germann, J., Vetkas, A., & Ertl-Wagner, B. B. (2024). Assessing the Emergence and Evolution of Artificial Intelligence and Machine Learning Research in Neuroradiology. American Journal of Neuroradiology, 45(9), 1269-1275.
Chubb, J., Crowling, P., & Reed, D. (2021). Speeding up to keep up: exploring the use of AI in the research process. AI & SOCIETY, 37, 1439-1457.
Kantor, I., & Solodovnikov, A. (1989). Hypercomplex numbers. Springer-Verlag, New York.
Sagyndykov, B.Zh., & Bimurat, Zh. (2024). Zhalpy kompleks sandardyn eki olshemdi algebrasy [General Two-Dimensional Algebra of Complex Numbers]. Torajgyrov Universitetinin habarshysy, No1, 89–101. Pavlodar. ISSN 2959-068X.
Bimurat, Zh., & Sagyndykov, B.Zh. (2024). Dual sandar negizinde avtomatty differentsialdau: adisteme, mysaldar zhane iske asyru [Automatic Differentiation Based on Dual Numbers: Methodology, Examples, and Implementation]. Radiojelektronika zhane bajlanys askeri-inzhenerlik institutynyn gylymi enbekteri. Askeri gylymi-tekhnichalyk zhurnal, No2(56), 155–164.
Das, I., & Dennis, J. E. (1997). A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems. Structural Optimization, 14(1), 63-69.
Fike, J. A. (2013). Multi-objective optimization using hyper-dual numbers (Doctoral dissertation, Stanford University). Stanford Digital Repository. https://purl.stanford.edu/jw107zn5044
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Articles are open access under the Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish a manuscript in this journal agree to the following terms:
- The authors reserve the right to authorship of their work and transfer to the journal the right of first publication under the terms of the Creative Commons Attribution License, which allows others to freely distribute the published work with a mandatory link to the the original work and the first publication of the work in this journal.
- Authors have the right to conclude independent additional agreements that relate to the non-exclusive distribution of the work in the form in which it was published by this journal (for example, to post the work in the electronic repository of the institution or publish as part of a monograph), providing the link to the first publication of the work in this journal.
- Other terms stated in the Copyright Agreement.