Data Engineer with Apache Iceberg

Posted via LinkedIn Recruiter (not a company profile)

Posted 6 days ago

Apply Now

About the role

  • Data Engineer with Apache Iceberg experience needed for hybrid role in Mississauga. Requires big data pipeline skills and Spark proficiency.

Responsibilities

  • Data Engineer role building and optimizing big data pipelines, architectures, and datasets. Requires expertise in Apache Spark, Hadoop ecosystem, real-time stream processing with Apache Flink, and data lake table formats specifically Apache Iceberg.

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related quantitative field. Proven experience as Data Engineer building big data pipelines. Strong proficiency in Python (with extensive pySpark), Java, or Scala. Expertise in Apache Spark, Spark SQL, DataFrames, RDDs. Hands-on experience with Apache Hadoop ecosystem (HDFS). Experience with real-time stream processing using Apache Flink. Familiarity with Apache Iceberg for managing large datasets. Experience with relational SQL and NoSQL databases. Solid understanding of ETL/ELT processes, data warehousing, data modeling. Good to have: cloud platforms (AWS, Azure, GCP) and their data services. Excellent problem-solving, analytical, and communication skills.

Job title

Job type

Permanent

Experience level

Not specified

Salary

Not specified

Degree requirement

Bachelor's or Master's degree

Tech skills

Apache IcebergApache SparkPythonpySparkJavaScalaSpark SQLDataFramesRDDsApache HadoopHDFSApache FlinkSQLNoSQLETL/ELTAWSAzureGCP

Location requirements

Linkedin Recruiter PostMississaugaOntario Mississauga

Report this job

Found something wrong with the page? Please let us know by submitting a report below.