The composition of VAG signals in the daily paradigm, where friction is intensified due to weight-bearing movements, is complex, making it very difficult to monitor the multi-grade osteoarthritis (OA) deterioration using VAG. It is easier to misclassify VAG signals present near the decision boundary of adjacent deterioration grades. Most of the existing literature solely addresses the binary classification of OA, which does not help doctors to develop treatment plans that consider the presence or absence of OA. As a result, we introduce a unique framework for daily paradigm fine-grained multi-grade OA degradation monitoring. We propose Confusion-Free Master-Slave (CF-MS) Classification and Multi-grade OA Deterioration Monitor (FMOADM) which comprises of Multiscale-temporal Feature Extraction (MTFE). To be more precise, multiscale-temporal discriminative features are extracted from the complex VAG signals using MTFE. Furthermore, center loss is added by CF-MS to clear up any confusion at the feature space boundary between adjacent degradation grades. For additional fine-grained classification, a master-slave structure has been proposed in which the slave classifier is made to update MTFE parameters while the master classifier incorporates a channel attention method. This means that by reducing boundary confusion and utilizing a multiscale-temporal discriminative feature, the proposed method guarantees fine-grained multi-grade OA monitoring performance. The VAG-OA dataset's experimental findings show that our approach performs better than analogous techniques in the daily paradigm. The proposed system outperformed the state-of-the-art approach by 8%, achieving 78% precision. In summary, the proposed framework enhances the effectiveness of multi-grade OA deterioration monitoring by enabling doctors to create personalized treatment regimens based on detailed monitoring data. It includes the monitoring of knee joint health in daily activities closer to being practical.