Lead Data Engineer (ADB)

About the role

We are seeking a Lead Data Engineer (ADB) to join a high‑performing tax technology team within a global professional services firm. You will design, develop, and maintain enterprise‑grade data pipelines and AI‑driven tools that support tax advisory services across a worldwide client base.

Key responsibilities

• Define and enforce coding standards and best practices across the project.
• Conduct thorough code reviews to maintain high code quality.
• Design, develop, and maintain robust, scalable Spark applications using PySpark and Databricks.
• Write clean, efficient code and optimise it for performance and scalability.
• Collaborate with cross‑functional teams to deliver high‑quality software solutions.
• Identify and resolve technical issues to ensure reliability and performance.
• Create and maintain comprehensive documentation for code, processes, and workflows.

Required profile

• 5+ years of hands‑on software development experience.
• Practical experience developing SQL stored procedures and migrating them to Spark SQL or PySpark.
• Extensive expertise with PySpark on Databricks, including Delta tables and workflow automation.
• Proven track record of optimizing Spark performance in production environments.
• Strong analytical and problem‑solving abilities with a detail‑oriented mindset.

Required skills

• Python (including Pandas, Polars)
• PySpark, Spark SQL, Databricks
• Azure Cloud services (Service Bus, Data Lake, Blob Storage, Redis)
• SQL, Azure SQL, Delta Lake, Parquet
• Microservices architecture
• .NET 8, ASP.NET Core
• Angular 18, Kendo UI
• GitHub Enterprise, Copilot
• LangGraph, LangChain, RAG pipelines, multi‑modal LLMs