Anomaly Detection System are essential in capturing complex and evolving anomalies thereby securing computer networks against attacks. Traditional systems have limitations in detecting and mitigating modern-day sophisticated attacks. In recent years, Convolutional Neural Networks (CNNs) and Deep Reinforcement Learning (DRL) are the promising approach by providing better decision-making capabilities. This paper presents a novel approach that leverages CNNs and DRL techniques to improve the accuracy and adaptability of anomaly detection. The proposed system employs a CNN-based architecture to extract meaningful features from network traffic. CNNs can capture spatial and temporal patterns, which are crucial for anomaly detection in various applications. To enhance adaptability and adapt to evolving anomalies, Deep Q-Network (DQN) component is added into the system. DQN agents are trained to make decisions based on the CNN-extracted features. It is inferred by the findings that CNN-DQN based anomaly detection system have the potential to significantly improve the detection and mitigation of network attacks with accuracy of 98.01