Matrix inversion is a computationally complex operation used in numerous scientific and engineering domains, from signal reconstruction to control systems. One such application is the Least Squares (LS) problem in Orthogonal Matching Pursuit (OMP), a Compressive Sensing (CS) reconstruction algorithm. Matrix inversion can be performed through QR decomposition (QRD) or the Gaussian Elimination (GE) method. This study investigates the performance of different factorisation algorithms and their effectiveness through High-Level Synthesis (HLS). Through directive-based loop and control optimisations in HLS, the study evaluates latency, resource usage, and throughput. The investigations for both fixed and floating type representation, for different matrix sizes, are reported on a Zynq xc7z020-clg484-1 FPGA device. The optimized floating-point implementation achieves 8.8% higher operating frequency, 67% lower latency, approximately three times higher throughput, and 75% lower numerical error than the fixed-point implementation, while the fixed-point implementation significantly reduces hardware resource utilization, making both implementations suitable for compressive sensing applications with different performance and resource requirements. The optimized Modified Gram-Schmidt (MGS) method and Incremental GE method have better execution time and resource utilization as compared to other classical methods in previous studies, amidst trade-offs. The optimized inversion techniques apply to wireless communications and signal processing applications where resource constraints and precision trade-offs are critical.