Swift growth in vehicle to everything (V2X) communications has created new opportunities for advanced vehicular networking. Vehicle-to-vehicle communication plays a crucial role in supporting latency-critical and safety-related services under ultra-reliable low-latency communication (URLLC) requirements. However, the increasing number of vehicles and diverse quality-of-service demands make efficient resource management a challenging task. To address this, this work proposes a multi-agent deep reinforcement learning (DRL)-based resource allocation framework for joint channel and power optimisation in heterogeneous vehicular networks. Individual utility functions are designed for each vehicular user to capture heterogeneous traffic requirements. Unlike conventional single-agent DRL approaches, the proposed multi-agent deep deterministic policy gradient (MADDPG) method adopts centralised training with distributed execution, enabling better coordination among agents. Input and reward normalisation are applied to accelerate training convergence. Simulation results demonstrate that the proposed scheme outperforms duelling DQN, fixed power allocation, and other RL baselines, achieving up to 23 higher throughput and 15 fewer latency violations, validating its scalability and effectiveness for latency-sensitive V2X communications. Copyright © 2026 Inderscience Enterprises Ltd. This is an Open Access Article distributed under the CC BY license. (https://creativecommons.org/licenses/by/4.0/)