Data Platform Engineer

About the role

Block Labs is building a new data platform that will become the backbone for real‑time decisioning and predictive modelling across its Web3, AI and iGaming products. As a Data Platform Engineer you will own the end‑to‑end real‑time pipeline, delivering data to a unified analytical warehouse and a low‑latency feature‑serving layer.

Key responsibilities

• Design, build and maintain scalable data pipelines using AWS Glue (PySpark) or equivalent tools.
• Engineer and optimise the ClickHouse warehouse for sub‑second query performance.
• Implement data contracts that allow new operators to be onboarded via configuration changes only.
• Develop the feature‑serving layer that provides pre‑computed features to AI agents with millisecond latency.
• Integrate with third‑party databases, back‑office APIs, CRM, affiliates and acquisition platforms.
• Establish monitoring, alerting, freshness checks, anomaly detection and SLA enforcement for data contracts.
• Own CI/CD pipelines and infrastructure‑as‑code for all data workloads.
• Collaborate with data scientists, agent engineers, BI developers and infrastructure teams to translate requirements into production‑grade pipelines.

Required profile

• 3+ years of experience building and operating production data pipelines at scale.
• Hands‑on experience with both streaming and batch processing paradigms.
• Deep expertise in Apache Kafka (or Amazon MSK), including topic design, consumer groups, schema registry and troubleshooting.
• Strong SQL and columnar warehouse engineering skills; ClickHouse experience strongly preferred.

Required skills

• AWS Glue / PySpark
• Python
• Apache Kafka / Amazon MSK
• ClickHouse
• SQL
• CI/CD pipelines
• Infrastructure‑as‑Code (e.g., Terraform)
• Monitoring and alerting tools
• Data contract design