In the dynamic environment of social media, user engagement improvement is the key to relevance enhancement of content, platform sustainability, and enhanced user experience. Failure of static recommendation models that are the foundation of conventional engagement strategies to dynamically adapt to changing user behaviours translates into content fatigue and poorer retention rates. Scalability in handling enormous volumes of social media data, algorithmic prejudice, and data privacy are significant challenges. By utilizing advanced machine learning approaches for real-time content personalization and predictive engagement analysis, this research offers a Machine Learning-Driven User Engagement Optimization Framework (ML-UEOF) to solve these limitations. By combining ensemble learning models, the ML-UEOF is able to forecast engagement levels, and RNNs examine patterns of user interactions over time. Social media analytics, influencer marketing, targeted advertising, adaptive collection of content in real-time, and targeted advertising are among the few of the many uses for the ML-UEOF framework. A more engaging and personalized online experience is possible when social media networks employ this smart analytical methodology to increase user happiness, diversify content, and maintain engagement over the years. Evaluating critical performance indicators including engagement rate, content relevancy, user retention, and computational efficiency, the framework is subjected to thorough simulation analysis utilizing real-world datasets from prominent social media platforms.