About the role

  • Senior Software Engineer working on big data solutions for a major social media platform. Contribute to building and scaling data processing frameworks with Java and Spark.

Responsibilities

  • Obtains tasks from the project lead or Team Lead (TL), prepares functional and design specifications, approves them with all stakeholders
  • Ensures that assigned area/areas are delivered within set deadlines and required quality objectives
  • Provides estimations, agrees task duration with the manager and contributes to project plan of assigned area
  • Evaluating existing data systems
  • Developing best practices for data coding to ensure consistency within the system
  • Updating and optimizing local and metadata models
  • Evaluating implemented data systems for variances, discrepancies, and efficiency
  • Troubleshooting and optimizing data systems
  • Understand the business drivers and analytical use-cases and translate these to data products
  • Design, implement and maintain pipelines that produce business critical data reliably and efficiently using cloud technology
  • Addresses area-level risks, provides and implements mitigation plan
  • Reports about area readiness/quality, and raise red flags in crisis situations which are beyond his/her AOR
  • Initiates and conducts code reviews, creates code standards, conventions, and guidelines
  • Suggests technical and functional improvements to add value to the product
  • Constantly improves his/her professional level
  • Collaborates with other teams
  • Design and implement Fine Grained Access Control system for Client’s big data platform (Spark, Flink, PyTorch, Ray and other ML/AI frameworks)
  • Improve and customize the internals of open source Spark to meet our challenges at scale and actively participate in open source community
  • Build and scale big data processing frameworks and infrastructure to process petabytes-scale datasets

Requirements

  • 5+ years of experience as Software Engineer with Solid Java coding skill with Spark
  • 3+ years of experience of building and support large scalable big data infrastructure/platform
  • Knowledge of big data technologies (e.g. Spark, Hadoop, Parquet/ORC, Flink)
  • Ability to organize and document solutions
  • A passion for streamlining systems and processes to make the difficult trivial
  • Strong OOP skills
  • Effective communication (oral & written), collaboration, and interpersonal skills
  • Result oriented approach
  • Nice to have: Experience with AWS & K8S

Benefits

  • Paid time off based on employee grade (A-F), defined by policy: Vacation: 12-25 days, depending on grade
  • Company paid holidays
  • Personal Days
  • Sick Leave
  • Medical, dental, and vision coverage (or provincial healthcare coordination in Canada)
  • Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)
  • Life and disability insurance
  • Employee assistance programs
  • Other benefits as provided by local policy and eligibility

Job type

Full Time

Experience level

Senior

Salary

CA$54,300 - CA$104,720 per year

Degree requirement

Bachelor's Degree

Tech skills

AWSCloudHadoopJavaKubernetesOpen SourcePyTorchRaySpark

Location requirements

OnsiteMississaugaCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.