Job Title: Azure Data Engineer Azure Data Factory (ADF)
Work Location: Redmond,WA-98052
Job Details:
Must Have Skills:
Python, SQL, (bonus: Scala), Apache Spark, PySpark, Spark SQL, Azure Databricks (notebooks, Jobs/Workflows, clusters), Azure Synapse, Delta Lake, Parquet, ADLS Gen2
Nice to have skills:
ETL/ELT, MERGE/upsert, partitioning, Spark performance tuning, Git, Azure DevOps / CI/CD
Detailed Job Description
Techniques (Data Engineering Practices)
ETL / ELT
Extract data from source systems
Transform/clean/reshape into usable form
Load into target tables (ELT = transform after loading, inside the platform)
MERGE / Upsert
Update existing rows when records already exist
Insert new rows when they don't
Handled in a single atomic operation (core Delta Lake pattern)
Partitioning
Split large tables by a column (e.g., date, source type)
Enables partition pruning - scan only relevant data
Improves query speed and reduces cost
Spark Performance Tuning
Handle data skew and manage shuffles
Caching / persistence of reused datasets
Right-size partitions; use broadcast joins
Cluster sizing and resource optimization Orm
Minimum years of experience
8-10 years
...a difference in peoples lives through food. We succeed in the industry by ensuring we deliver a large variety of exciting and quality products... ...About the Role: Mollie Stones Markets is looking for a FT Wine and Spirits APC Clerk at our Pacific Heights locations. The...
...to talk to you! We are hiring newly experienced and seasoned delivery drivers. Our Service and Delivery Drivers are responsible for delivering... ...NTW vehicle ~ Must be 21 years or older ~ Ability to lift up to 50lbs repetitively while bending and twisting and ability...
...managing corrective actions. ~ Familiarity with environmental programs involving stormwater, waste, and chemical management. ~ Experience and willingness to work in outdoor industrial environment. ~ Valid driver's license and ability to travel between sites....
The Laundry Attendant is responsible for providing a constant supply of clean linens for the entire hotel. This role requires attention to detail and the ability to work efficiently in a fast-paced environment. This role involves inspecting, washing, folding, and storing...
...We are looking for an experienced Materials Scientist or Materials Engineer to analyse experimental results, review technical literature... ...experimental results, technical datasets, and materials-science research. Conduct literature reviews on materials, methodologies...