Data Scientist Databricks SME

About the Company

We are seeking a highly skilled Data Scientist with deep AI expertise to join our team. You will leverage the Databricks Data Intelligence Platform to design, build, and deploy advanced machine learning models and AI-driven solutions. This role is ideal for a Python expert who thrives at the intersection of big data, distributed computing, and cutting-edge artificial intelligence.

About the Role

This role involves leveraging advanced machine learning models and AI-driven solutions to address complex business problems.

Responsibilities

• AI/ML Model Development: Design and train machine learning models using various algorithms, including deep learning, NLP, and computer vision. • Databricks Orchestration: Build and optimize end-to-end AI/ML pipelines on Databricks, utilizing Unity Catalog for governance and MLflow for experiment tracking. • Generative AI & LLMs: Implement advanced AI patterns such as Retrieval-Augmented Generation (RAG) and fine-tune pre-trained models for specific enterprise tasks. • Python Expertise: Write production-quality, idiomatic PySpark and Python code that leverages Spark’s distributed nature. • Collaboration: Partner with Engineering and Product teams to translate business problems into scalable analytical solutions. • Insight Extraction: Perform exploratory data analysis (EDA) and extract meaningful insights from massive, complex datasets to drive strategic decisions.

Qualifications

• Education: MS or PhD in a quantitative field such as Computer Science, Statistics, or Math. • Experience: 5+ years of hands-on experience in data science or AI engineering in high-growth environments.

Required Skills

• Technical Proficiency: Expert-level Python (pandas, NumPy, scikit-learn, PySpark). • Extensive experience with Apache Spark for large-scale data processing. • Proficiency in SQL for data manipulation and querying in Lakehouse environments. • AI Foundations: Strong understanding of statistics, probability, and advanced ML lifecycle management (MLOps).

Preferred Skills

• Experience with Deep Learning frameworks (TensorFlow, PyTorch). • Familiarity with Cloud Platforms (AWS, Azure, or GCP) and their native data services. • Databricks certifications, such as Databricks Certified Machine Learning Professional.

Pay range and compensation package

[Pay range or salary or compensation]

Equal Opportunity Statement

We are committed to diversity and inclusivity.