Position : Senior Data Engineer Azure
Location : Hyderabad Hybrid role
Duration : Fulltime
HACKER RANK TEST AND F2F INTERVIEW AFTER 1ST ROUND OF VIRTUAL INTERVIEW
Senior Data Engineer
We are recruiting for an experienced Senior Data Engineer and we’ll give you the opportunity to grow and develop your capabilities as well as supporting you through Microsoft certifications. You’ll have experience in:
· Developing low to high complexity, secure, governed, high quality and efficient data pipelines from a variety of on and off premise, internal and external data source.
· Ensuring that data is cleansed, mapped, transformed and optimised for storage to meet requirements for business and technical use cases.
· Building data observability and data quality by design into all Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies.
· Take all requirements into account when creating a proposal
· Understanding and evaluating the organisational impact of new technologies and digital services
· Troubleshooting and resolving issues
You may be building solutions that transform data into data lake storage areas, physical database models and reporting structures across data lakes, data warehouse, business intelligence systems and analytics applications.
You may also be required to carry out unit testing of own code and peer testing of others to ensure appropriate quality.
Experience
· Experience of working with large, event based, datasets in an enterprise environment.
· Minimum 4 years within an Azure or SQL Data Engineering environment
· Engineering and database skills in SQL Server and/or Azure (Data Factory, Synapse) and the associated ETL technologies.
· Experience developing, driving & maintaining Data Integration Roadmaps and best practices.
· Overall responsibility of all technical business requirements gathering deliverables and implementation.
· Ability to deliver high to low-level designs for Data Solutions and have a good understanding of the full data lifecycle.
· Hands-on experience in designing and developing scripts for custom ETL processes and automation in Azure Data Factory, Azure Databricks, Azure Synapse, Python, Pyspark etc.
· Hands-on experience designing and developing stored procedures/functions and leveraging ETL packages/workflows
· Expertise in Cloud related Big Data integration and infrastructure tech stack using Azure Databricks & Apache Spark framework
· Well-versed with scripting languages such as Python
· Experience of Continuous Integration / Continuous Delivery (CI/CD).
· Experience of working in an agile environment and self-managing teams.
· Understanding and practical knowledge of testing techniques and tools
Desired technologies
· Big Data
· Azure
· Spark
· Python
. Azure Databricks
· Azure Synapse
· Azure Data Factory
· SQL Server
AZURE SYNAPSE,SPARK,DATABRICKS
Additional InformationAll your information will be kept confidential according to EEO guidelines.
