data engineer
Title posted on indeed.com -
Cloud Data Engineer
Posted on
October 09, 2024
by
Employer details
United It Solutions Inc
Job details
*Job Description :*
This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise.
*Major Responsibilities:*
* Build and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity.
* Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data.
* Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling.
* Proficient in developing programs in Python and SQL
* Experience with Data warehouse Dimensional data modeling.
* Working with event based/streaming technologies to ingest and process data.
* Working with structured, semi structured and unstructured data.
* Optimize Databricks jobs for performance and scalability to handle big data workloads.
* Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
* Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
* Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process.
* Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.
*Skills:*
* 5+ years Python coding experience.
* 5+ years - SQL Server based development of large datasets
* 5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark.
* Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake.
* Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
* Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills.
* Experience with Cloud based data architectures, messaging, and analytics.
* Cloud certification(s).
* Any experience with Airflow is a Plus.
Job Type: Full-time
Expected hours: 40 per week
Schedule:
* Monday to Friday
Experience:
* Python: 7 years (required)
* SQL: 8 years (required)
* ADF: 8 years (required)
* Snowflake: 8 years (required)
Work Location: In person
-
LocationMississauga, ON
-
Workplace information
On site
-
SalaryNot available
-
Terms of employment
Full time
-
Starts as soon as possible
- vacancies
1 vacancy
- Source
indeed.com
#9480282812
Important notice: This job posting has been provided by a partner site. Job Bank is not responsible for this content.
Report a problem with this job posting
Thank you for your help!
You will not receive a reply. For enquiries, please contact us.