We are looking for experienced Databricks Data Engineers to design, develop, and optimize scalable data pipelines and Lakehouse solutions for large-scale modernization, migration, and AI-ready data platform initiatives. The role demands strong expertise in PySpark, Databricks, Delta Lake, and cloud-based data engineering. Key responsibilities include building ETL/ELT pipelines, implementing Lakehouse & Medallion Architecture (Bronze-Silver-Gold), managing data governance and lineage via Unity Catalog, and optimizing performance using tools like Auto Loader and Structured Streaming. Preferred candidates will have hands-on experience with Azure/AWS cloud platforms, CI/CD workflows, and BFSI domain knowledge.
Required Qualifications
Databricks expertise
PySpark & Spark SQL proficiency
Python & SQL programming
Delta Lake & Delta Live Tables (DLT) knowledge
ETL/ELT pipeline development
Cloud platform exposure (Azure/AWS)
Data governance & lineage understanding
Preferred Qualifications
Azure Data Factory (ADF) or AWS Glue experience
Apache Airflow & DBT familiarity
CI/CD & Azure DevOps/Git workflow
Lakehouse & Medallion architecture implementation
BFSI domain experience
Skills Required
DatabricksPySparkSpark SQLPythonSQLDelta LakeDelta Live TablesAzure Data FactoryAWS GlueApache AirflowDBTUnity CatalogCI/CDAzure DevOpsGitETL/ELTPerformance TuningLakehouse ArchitectureData GovernanceCloud Computing