PySpark Training in Sofia

In this live, instructor-led training in Sofia, participants will explore how to harness the combined power of Python and Spark for big data analysis. The course emphasizes practical application through a sequence of hands-on exercises.

Upon completing this training, participants will be able to:

• Utilize Spark alongside Python to conduct comprehensive Big Data analysis.
• Complete exercises that reflect authentic, real-world data scenarios.
• Employ diverse tools and techniques for big data analysis using the PySpark framework.

This instructor-led live training (online or onsite) is designed for data engineers, data analysts, and data professionals who wish to use Databricks and PySpark to build scalable data pipelines and migrate existing SQL workflows.

This course offers a hands-on introduction to developing scalable data processing and Machine Learning workflows with PySpark. Attendees will gain insight into how Apache Spark functions within contemporary Big Data ecosystems and how to effectively manage large datasets by leveraging distributed computing principles.

This intensive three-day workshop is designed to help you build and refine high-performance data-processing workloads by leveraging PySpark, Pandas and Polars within Kubernetes-based environments.

Attendees will gain a functional grasp of how Spark applications operate on Kubernetes, and how specific configuration choices at the application level directly impact performance, scalability, resource utilisation and overall costs. Key optimisation topics covered include executor sizing, memory allocation strategies, dynamic allocation, partitioning logic, shuffle mechanics, resolving small-file issues and enhancing Parquet processing efficiency.

Additionally, the course tackles frequent challenges encountered with Pandas, such as memory constraints and out-of-memory errors, while introducing Polars as a high-performance solution for specific data-processing tasks. Through practical, hands-on labs, participants will learn to diagnose performance and memory bottlenecks, evaluate various configuration approaches and implement optimisation techniques in realistic ETL and machine learning contexts.

Throughout the training, the focus remains on practical decision-making: learning how to pinpoint bottlenecks, choose the right tools, configure Spark for maximum efficiency and strike a balance between performance and infrastructure costs.