Databricks Data Engineer

TATA Consultancy Services Ltd

Hyderabad, Telangana, India

Type: Full-Time Arrangement: On-Site
5 - 10 Yrs800000 - 1500000(Annually)

Posted: 2 Sept 2026

Job Description

We are looking for experienced Databricks Data Engineers to design, develop, and optimize scalable data pipelines and Lakehouse solutions for large-scale modernization, migration, and AI-ready data platform initiatives. The role demands strong expertise in PySpark, Databricks, Delta Lake, and cloud-based data engineering. Key responsibilities include building ETL/ELT pipelines, implementing Lakehouse & Medallion Architecture (Bronze-Silver-Gold), managing data governance and lineage via Unity Catalog, and optimizing performance using tools like Auto Loader and Structured Streaming. Preferred candidates will have hands-on experience with Azure/AWS cloud platforms, CI/CD workflows, and BFSI domain knowledge.

Required Qualifications

  • Databricks expertise
  • PySpark & Spark SQL proficiency
  • Python & SQL programming
  • Delta Lake & Delta Live Tables (DLT) knowledge
  • ETL/ELT pipeline development
  • Cloud platform exposure (Azure/AWS)
  • Data governance & lineage understanding

Preferred Qualifications

  • Azure Data Factory (ADF) or AWS Glue experience
  • Apache Airflow & DBT familiarity
  • CI/CD & Azure DevOps/Git workflow
  • Lakehouse & Medallion architecture implementation
  • BFSI domain experience

Skills Required

DatabricksPySparkSpark SQLPythonSQLDelta LakeDelta Live TablesAzure Data FactoryAWS GlueApache AirflowDBTUnity CatalogCI/CDAzure DevOpsGitETL/ELTPerformance TuningLakehouse ArchitectureData GovernanceCloud Computing