Job Summary:
We are seeking an experienced Senior Data Platform Engineer with over 15 years of expertise in managing and administering complex data and compute infrastructure across on-premises and cloud environments. The ideal candidate will have deep hands-on experience with Kubernetes (on-prem & Azure) , Spark Clusters , and CI/CD pipelines using Azure DevOps , with a strong background in modern data platform tools including Apache Ranger, Kafka, Airflow, Trino, Iceberg , and Datahub .
Key Responsibilities:
- Administer and maintain on-prem Kubernetes environments using in-house Kubernetes packages , as well as Azure Kubernetes Service (AKS) clusters.
- Provision, configure, monitor, and manage Apache Spark clusters for large-scale data processing.
- Implement and support CI/CD pipelines using Azure DevOps (ADO) for data platform components and infrastructure.
- Ensure secure and governed data access using Apache Ranger and other security tools.
- Support and manage distributed data systems such as Kafka, Trino, Airflow, Iceberg , and Datahub .
- Monitor system health, identify performance bottlenecks, and perform root cause analysis for infrastructure and platform issues.
- Collaborate with DevOps, Data Engineering, and Cloud teams to optimize the scalability, performance, and resilience of the platform.
- Maintain infrastructure as code (IaC) practices and participate in platform automation initiatives.
- Ensure compliance with internal and regulatory security standards.
Required Skills & Experience:
- Minimum 15 years of overall IT experience with at least 5–7 years in data platform administration .
- Strong hands-on experience in:
- On-prem Kubernetes clusters (in-house distributions)
- Azure Kubernetes Service (AKS)
- Apache Spark cluster administration
- Apache Ranger for data governance and access control
- Apache Kafka , Apache Airflow , Trino , Iceberg , Datahub
- Azure DevOps (ADO) – pipelines, repos, artifacts
- Proven experience with CI/CD for data platforms and infrastructure automation .
- Strong Linux administration and scripting skills (Bash, Python preferred).
- Deep understanding of networking, container orchestration, and cloud infrastructure (Azure preferred).
- Experience in implementing monitoring and alerting systems for distributed environments.
Nice to Have:
- Experience in highly regulated industries such as banking or financial services .
- Familiarity with data lakehouse architectures .
- Certifications in Azure , Kubernetes , or Big Data technologies .
When you apply, you voluntarily consent to the disclosure, collection and use of your personal data for employment/recruitment and related purposes in accordance with the Tech Aalto Privacy Policy, a copy of which is published at Tech Aalto’s website (https://www.techaalto.com/privacy/)
Confidentiality is assured, and only shortlisted candidates will be notified for interviews.