IEEE

Curvature-Based Piecewise Linear Approximation Method of GELU Activation Function in Neural Networks

Abstract: Artificial neural networks (ANNs) rely significantly on activation functions for optimal performance. Traditional activation functions such as ReLU and Sigmoid are commonly used. However, ...
IEEE

Hardware Implementation of Softmax Function Based on Piecewise LUT

Abstract: Deep neural networks (DNN) achieve great results in many fields. While softmax function is widely used in DNN, how to implement it on hardware considering the accuracy, speed, area, and ...