Staff Software Engineer (Apache Spark Poweruser / Committer)

About

Seeking extraordinary software engineers with deep experience in scalable batch and streaming processing systems. You will have a passion to push the limits of distributed computing frameworks to get every ounce of performance out of them. Looking for engineers with in depth knowledge of Spark and other related existing frameworks/systems such as Iceberg, Parquet. Also someone who will be passionate by the prospect of working reciprocally with other groups internal to Apple and also communities outside Apple.

As an engineer on our team, you will advance our data infrastructure, driving innovation by implementing cutting-edge capabilities and tools that will power Apple Services Engineering. You'll collaborate closely with a dedicated team of open source data experts (including Spark, Kafka, Iceberg committers amongst others) to enhance and automate our infrastructure, ensuring seamless management across both data centers and cloud services. Your work will directly contribute to the resilience, scalability, and performance of the systems that support millions of users worldwide.

Requirements

• 8+ years of professional experience professional experience.
• A successful track-record or demonstrated aptitude as an engineer who has worked on distributed systems.
• Experienced Poweruser (at least 2 years of running production applications) of Apache Spark or Committer to Apache Spark.
• BS or MS degree in computer science or equivalent
• Good knowledge of Apache Iceberg and Apache Kafka.
• Experience working on or building connectors from Spark to any of the data sources.
• Knowledge of Yarn, Kubernetes or other compute substrate.