Traffic signal optimization is a critical component of modern urban traffic management systems. This paper proposes a novel approach to optimizing traffic signal control using Markovian Decision Processes (MDPs). By modeling the traffic flow at intersections as a stochastic process, the Markovian framework allows for the development of decision policies that minimize congestion and improve traffic flow. The system considers various factors such as traffic volume, waiting times, and signal timings, incorporating these into a dynamic model that adapts to real-time traffic conditions. A reinforcement learning algorithm is applied to iteratively improve the control policy, aiming to minimize the total delay and improve throughput across the network of traffic signals. Experimental results demonstrate that the proposed MDP-based approach outperforms traditional traffic signal control strategies, providing significant reductions in average waiting time and overall traffic congestion. This study highlights the potential of advanced decision-making models in creating more efficient, responsive traffic management systems in urban environments.