All signature languages can be an important means of communication for people living with disabilities. The disabled peoples interact through visual gestures. The current research focuses specifically on Tamil Sign Language (TSL), in particular Uyir Ezhuthukal (vowel letters) which dominate a substantial portion of communication in everyday life. It includes image processing techniques for gesture recognition using image processing tools. The image quality is set better and the noise effects are also reduced using the key preprocessing techniques like hand skin masking, Gaussian filtering, gamma correction and grayscale conversion. The origin of the system connects a bidirectional long short -term Memory (B-LSTM) model with fully connected network used toget spatial and cosmic features for accurate classification. Data augmentation strategies are focused on improving the generalization of the system and hyperparameter optimization to improve its performance. The model was evaluated on an in-house dataset (TLFS23) on Tamil alphabet gestures resulting in an accuracy of 95%.