Data Engineer - Databricks

TCS is Hiring Data Engineer - Databricks For Bangalore, Chennai and Hyderabad location

Experience : 5 - 8 years

Mode of interview : Virtual

Location : Bangalore, Chennai, Hyderabad

JOB DESCRIPTION:

Must have:

• Strong experience in programming languages such as Python, PySpark, or Scala for data engineering tasks. • Design, develop, and maintain scalable and robust data pipelines on Databricks • Hands-on experience with cloud technologies, expertise in AWS/Azure data engineering services to handle large-scale data processing and real-time data streams. • In-depth knowledge of data warehousing solutions for building and managing data warehouses. • Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using cloud services, ADF, Glue etc

Good to have:

• Strong understanding of data modeling concepts and techniques to create efficient and scalable data models. • Experience with version control systems such as Git for code management and collaboration. • Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information

Responsibilities:

• Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility. • Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data. • Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency • Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle. • Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.