The Azure Data Engineer will implement big data pipelines using Azure technologies, collaborate with stakeholders, and utilize tools like Data Factory and Databricks.
This is a remote position.
Description :
- At least 2 years hands on experience working with an Agile data engineering team working on big data pipelines using Azure in a commercial environment.
- Dealing with senior stakeholders/leadership
- Understanding of Azure data security and encryption best practices. [ADFS/ACLs]
- Data Bricks - experience writing in and using data bricks Using Python to transform, manipulate data.
- Data Factory - experience using data factory in an enterprise solution to build data pipelines. Experience calling rest APIs.
- Synapse/data warehouse - experience using synapse/data warehouse to present data securely and to build & manage data models.
- Microsoft SQL server - We- d expect the candidate to have come from a SQL/Data background and progressed into Azure
- PowerBI - Experience with this is preferred
Additionally :
- Experience using GIT as a source control system
- Understanding of DevOps concepts and application
- Understanding of Azure Cloud costs/management and running platforms efficiently
Requirements
- 3 to 6 years of experience.
- Must Have: Azure services like Azure Data Factory, Databricks, Azure Data lakes, Synapse/DW. and SQL
- Good to have: Python, PowerBI, Git
Similar Jobs
Information Technology • Professional Services • Software • Consulting
Design, build and maintain secure, high-quality data pipelines on Azure (Data Factory, Databricks, Synapse). Cleanse, transform and optimise large event-based datasets, implement data quality/observability, develop ETL scripts, stored procedures, and CI/CD pipelines, and participate in testing and troubleshooting.
Top Skills:
SparkAzureAzure Data FactoryAzure DatabricksAzure SynapseCi/CdETLPysparkPythonSparkSQL Server
Software
Design, build, and maintain batch and real-time ETL/ELT data pipelines and workflows. Automate ingestion, aggregation and logging; ensure data accuracy, security, and compliance. Implement data warehouse and lake solutions, mentor junior engineers, and collaborate with stakeholders to meet business requirements.
Top Skills:
AdfAdls Gen2AirflowAthenaAWSAws QuicksightAzureAzure Sql DbAzure SynapseBigQueryCassandraData LakesData WarehousesEc2EmrFivetranGCPGlueGlue EtlHadoopHiveIamJavaLambdaMapreduceMatillionMongoDBNeo4JNumpyPandasPower BIPythonRdsRedisRedshiftS3ScalaSnowflakeSparkSQLStitchTableau
Artificial Intelligence • Big Data • Information Technology • Consulting
The Azure Data Engineer will build data pipelines and dataflows, implement security modules, and work with Azure services while supporting business requirements and data analysis.
Top Skills:
Azure Data FactoryAzure Data LakeC#C++ .NetMicrosoft Sql Server 2012Microsoft Sql Server 2016PowershellSql DatabaseSsisVisual Basic .Net
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
