Please use this identifier to cite or link to this item:
|Title:||A Performance Study Depending on Execution Times of Various Frameworks in Machine Learning Inference||Authors:||Sever M.
|Issue Date:||2021||Publisher:||Institute of Electrical and Electronics Engineers Inc.||Source:||Sever, M., & Öğüt, S. (2021, November). A Performance Study Depending on Execution Times of Various Frameworks in Machine Learning Inference. In 2021 15th Turkish National Software Engineering Symposium (UYMS) (pp. 1-5). IEEE.||Abstract:||This work is intended to compare the latency of various frameworks in machine learning inference through an average power calculation model. This model is created in terms of a 2-layer neural network with PyTorch, in Python. Then, it is converted to a traced Torch Script module and also to ONNX file format. Afterwards, the C++ front-end is used for the inference process. The traced model is run with Libtorch on CPU and GPU, the ONNX file is run with ONNX Runtime on both CPU and GPU and it is also run with TensorRT on GPU. The inference execution times for 100 trials are averaged for all cases and it is realized that TensorRT with ONNX file format significantly outperforms its counterparts as expected. Hence, this work highlights the performance of TensorRT in machine learning inference and sheds light into the future by proposing several extensions. © 2021 IEEE.||Description:||15th Turkish National Software Engineering Symposium, UYMS 2021 -- 17 November 2021 through 19 November 2021 -- -- 176220||URI:||https://doi.org/10.1109/UYMS54260.2021.9659677
|Appears in Collections:||Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering|
Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection
Show full item record
checked on Dec 26, 2022