Rainy weather conditions are challenging issues for many computer vision applications. Rain streaks and rain patterns are two crucial environmental factors that degrade the visual appearance of high-definition images. A deep attention network-based single-image deraining algorithm is more famous for handling the image with the statistical rain pattern. However, the existing deraining network suffers from the false detection of rain patterns under heavy rain conditions and ineffective detection of directional rain streaks. In this paper, we have addressed the above issues with the following contributions. We propose a multilevel shearlet transform-based image decomposition approach to identify the rain pattern on different scales. The rain streaks in various dimensions are enhanced using a residual recurrent rain feature enhancement module. We adopt the Rain Pattern Absorption Attention Network (RaPaat-Net) to capture and eliminate the rain pattern through the four-dilation factor network. Experiments on synthetic and real-time images demonstrate that the proposed single-image attention network performs better than existing deraining approaches.