Efficient traffic signal control is essential for reducing congestion and improving city mobility. This study proposes a hybrid optimization framework that combines Qlearning with Quadratic Unconstrained Binary Optimization (QUBO) to manage signal phases at a four-way intersection. The system accounts for realistic operational constraints, such as yellow light transition times, minimum green phase requirements, and measures to prevent starvation, ensuring fair service for all directions. The Q-learning agent changes its policy by interacting with the simulated environment, while the QUBO component guides phase selection, reducing total queue lengths and avoiding unnecessary phase changes. In this work, a custom traffic simulation with random vehicle arrivals and a visualization module for real-time monitoring of traffic conditions and signal behavior is considered. Experimental results show that the proposed method effectively balances efficiency and fairness. It dynamically adjusts green phases according to traffic demand in preventing long delays. This approach shows the potential of using reinforcement learning and combinatorial optimization in smart, adaptable traffic management systems.