Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

bmJ2K2Z1dFpleFZXSVpDaVFVM0hRYk5Ednc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

MetroSys

Part-Time Cisco Network Engineer Job at MetroSys

 ...Job Summary: We are seeking an experienced Cisco Network Engineer to provide part-time support (1020 hours per month) for our network infrastructure in Dallas, TX. This role is ideal for a senior engineer who enjoys flexible, fractional engagements and can bring... 

Mach Industries

Director, Navy Job at Mach Industries

 ...Role Lead strategy, capture, and program development across Navy acquisition channels. Oversee end-to-end program execution for joint...  ...execution. Drive relationships with senior uniformed and civilian leadership in the Pentagon and across the Fleet. Monitor emerging... 

Global LT

German Language Teacher for a child in Blythewood, SC Job at Global LT

 ...Join the Global LT community and help provide language lessons to business professionals and their families worldwide. We are hiring GermanLanguage Teachers to provide in-person customized lessons to our clients in Blythewood, SC. Job Information: Ref. #186967... 

Talent Span Partners

Commercial Service Plumber Job at Talent Span Partners

 ...We provide our clients with complete plumbing services for a wide range of job types; from single-family homes to mid-rise condominium complexes, townhouse complexes, hotels, retail spaces, as well as educational, religious, and government facilities. To qualify... 

People Placers

Travelling Physical Therapist Job at People Placers

 ...We are currently seeking a motivated Travelling Physical Therapist to work on a project-basis for a highly reputable rehabilitation and physical therapy provider that has been top 20% for orthopedic conditions and top 15% for pelvic floor specialization nationally...