Breast cancer, having its origin from the breast tissue is usually detected by mammographic screening. The early detection of breast cancer reduces the mortality rate. A subtle type of breast cancer that often leads to misinterpretation by radiologists is architectural distortion. Though the existing computer aided diagnosis systems efficiently and effectively detect the presence of micro-calcification and masses, the diagnosis of architectural distortion lacks a promising method. This project attempts to detect and classify the regions of mammograms having architectural distortion. MIAS and DDSM database images are enrolled in this research work. 350 region of interests (ROIs) of each architectural distortion and normal cases were extracted. They were subjected to a filtering process, followed by contrast enhancement. Application of Gabor filter to the images resulted in orientation differences between the normal and abnormal images. Statistical features extracted from the resulting images were classified using extreme learning machine classifier. The experimental results obtained from extreme learning machine in comparison with support vector machine had an accuracy of 98.49% and 87.21% for MIAS and DDSM respectively. The accuracy of combined database of which is 85.38%.