Find Your Spark

Launch the next phase of your career with Canada's leading tech companies

Data Platform Engineer

Optable

Optable

Software Engineering
Montreal, QC, Canada · Montreal, QC, Canada · Remote
Posted on May 28, 2025

Mission

Design, build, and operate reliable and scalable data pipelines and infrastructure that power internal analytics and business intelligence at Optable. This includes developing Spark-based ETLs and maintaining a high-quality data warehouse used by Engineering, Product, Sales, and Customer Success teams. Maybe others as well.

🛠️ Core Responsibilities

1. Development

  • Build and maintain batch Spark jobs on k8s to ingest and transform data from internal services and external integrations.
  • Implement resilient, testable, and performant Spark code (Scala).
  • Write high-quality data contracts and schemas with clear ownership and versioning.
  • Help design and evolve the core data warehouse schema optimized for BI and analytics.
  • Collect feedback from the clients of this product (Solutions, Sales, Product, etc) to understand reporting needs and help prioritize a backlog.
  • Use DAG orchestration (with Astro, a Managed Airflow service) for productizing pipelines.

2. Maintenance

  • Set up robust monitoring and alerting for data freshness, pipeline health, and SLA adherence (Prometheus metrics, Grafana Cloud for vizualisations, Grafana IRM for incidents).
  • Debug and resolve data quality issues proactively.
  • Be ready to participate actively in incident response related to data infrastructure.
  • Document pipelines and datasets to support self-serve access and discovery.
  • Contribute to internal tooling and CI/CD workflows to improve developer velocity and platform stability.