Dynamic Scaling of AI-Driven Data Platforms: Resource Management for Generative AI Workloads - Mouna Reddy Mekala, (International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2025)
This article comprehensively analyzes dynamic scaling mechanisms for AI-driven data platforms, focusing on resource management challenges in generative AI workloads. The article examines the limitations of traditional scaling approaches and proposes a novel algorithm that combines predictive and reactive elements for optimal resource allocation. This article demonstrates significant improvements in resource utilization, cost efficiency, and system performance through an extensive evaluation across multiple cloud platforms, including Databricks and Amazon EMR. The article provides detailed platform-specific optimization strategies and implementation guidelines, offering organizations a robust framework for deploying and managing AI workloads in cloud environments.
Keywords: Dynamic Resource Management; Generative AI Workloads; Cloud Computing Optimization; Platform-Specific Scaling; AI Infrastructure Management