The increasing complexity of large language models (LLMs), along with their rising processing demands, latency, and carbon footprint, make AI deployment unsustainable and ineffective. In this paper, we introduce Eco-Transformers, a novel runtime optimization technique for improving Transformer model inference efficiency. The technique dynamically removes low-magnitude weights by masking and freezes neuron activations near zero. Crucially, these optimizations are done at runtime— without retraining or altering the model structure. With PyTorch forward hooks implementation, the framework is lightweight, architecture-independent, and completely compatible with real-time and low-resource environments. The proposed Eco-Transformers was tested on four transformer models: FalconRW-1B, GPT-Neo-1.3B, OPT-1.3B, and GPT2-XL. Eco-Transformers consistently reduced CO2 emissions and inference latency across all tested models while maintaining output quality as verified by BLEU, ROUGE-L, and perplexity scores. GPT2-XL achieved the greatest reduction in inference time, while GPT-Neo1.3B recorded the highest CO2 savings, making both models stand out in complementary aspects of efficiency. These findings validate Eco-Transformers as a cost-effective and efficient method for low-latency and carbon-efficient inference in transformer-based language models.