Large Language Models (LLMs) have trans-formed software development through AI-powered code generation, yet systematic comparisons of their capabilities remain limited. We present a comprehensive empirical evaluation of six leading LLM-based coding assistants—GPT-4, Claude 3.5 Sonnet, Gemini 1.5 Pro, CodeLlama-70B, DeepSeek Coder, and Mistral Large—across 1,847 code generation tasks spanning five programming languages and eight complexity tiers. Our evaluation framework assesses functional correctness (pass@k), code quality (maintainability, security), computational efficiency, and prompt robustness. Key findings reveal: (1) Claude 3.5 Sonnet achieves the highest overall pass@1 rate (84.7%) but GPT-4 excels in complex algorithmic tasks;
(2) all models exhibit significant performance degradation (18–34%) on adversarial prompt variations; (3) security vulnerability rates range from 3.2% (Claude) to 11.8% (CodeLlama); and (4) open-source models achieve 73–81% of proprietary model performance at substantially lower cost. We release our benchmark suite, CodeEval-1847, comprising novel problems to prevent data contamination. Our findings provide actionable guidance for practitioners selecting AI coding tools and highlight critical areas for model improvement.