Capco Logo

Capco

Pyspark Azure Synapse Data Engineer - Bangalore/ Chennai/ Pune/ Hyderabad/ Mumbai/ Gurgaon

Sorry, this job was removed at 11:23 a.m. (IST) on Wednesday, Oct 09, 2024
India
India

Job Title: Data Engineer with Pyspark, Azure Synapse

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the  British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.


Key Skills: Data Engineering, Pyspark, ADLS, Synapse, Hadoop, CI/CD

Technical Requirement:

Role: Data Engineer / Azure Data Engineer

Responsibilities

  • Responsible for designing, building, and maintaining data pipelines and infrastructure on Microsoft Azure
  • Extract, transform, and load data from various sources including databases, APIs, and flat files.
  • Stay up to date with the latest advancements in Azure data technologies and best practices.
  • Propose suitable data migration sets to the relevant stakeholders.
  • Assist teams with processing the data migration sets as required.
  • Assist with the planning, tracking and coordination of the data migration team and with the migration run book and the scope for each customer.

Role Requirements

  • Strong Data Analysis experience in Financial Services
  • Knowledge of and experience using data models and data dictionaries in a Banking and Financial Markets context
  • Demonstrate a continual desire to implement “strategic” or “optimal” solutions and where possible, avoid workarounds or short-term tactical solutions.
  • Working with stakeholders to ensure that negative customer and business impacts are avoided.
  • Manage stakeholder expectations and ensure that robust communication and escalation mechanisms are in place across the project portfolio.
  • Good understanding of the control requirement surrounding data handling

Experience/Skillset

  • Proven experience working with Microsoft Azure Data Services, including Data Factory, Data Lake Storage, Synapse Analytics, and Azure SQL Database.
  • Strong knowledge of data warehousing, data modeling, and ETL/ELT principles.
  • Hands-on experience in Airflow, Elastic
  • Perform data cleansing, transformation, and integration tasks to ensure data quality and consistency.
  • Develop and maintain data models for data warehousing, data lakes, and other data storage solutions.
  • Proficiency in programming languages like Python, PySpark, Scala, or SQL.
  • Knowledge of and experience using data models and data dictionaries in a Banking and Financial Markets context
  • Knowledge of CI/CD processes for application software integration and deployment, including Maven, Git, and Jenkins
  • Implement and maintain security best practices for data storage and access.
  • Monitor and troubleshoot data pipelines and data quality issues.
  • Knowledge of COBOL copy book is preferable to migrate the COBOL copy book solution to Azure data solution.
  • Document and communicate technical solutions effectively.
  • Enthusiastic and energetic problem solver to join an ambitious team.
  • Business analysis skills, defining and understanding requirements.
  • Attention to detail.
  • Good knowledge of SDLC and formal Agile processes, a bias towards TDD and a willingness to test products as part of the delivery cycle.
  • Ability to communicate effectively in a multi-program environment across a range of stakeholders.
  • Strong verbal and written communication skills

If you are keen to join us, you will be part of an organization that values your contributions, recognizes your potential, and provides ample opportunities for growth. For more information, visit www.capco.com. Follow us on Twitter, Facebook, LinkedIn, and YouTube.

Similar Jobs at Capco

6 Days Ago
Remote
Pune, Maharashtra, IND
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Data Architect will design and develop data architecture solutions in the Finance sector, collaborating with stakeholders to enhance data management and resolve issues. Responsibilities include creating data models, maintaining documentation, and ensuring compliance with standards and regulations. This role involves continuous improvement of data architectures and effective communication with technical and non-technical audiences.
Top Skills: AgileData ArchitectureScrum
6 Days Ago
Remote
Pune, Maharashtra, IND
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Sr. Data Engineer will work on projects involving PySpark and Scala with a focus on data analysis and debugging. They will utilize their skills in Spark, GIT, and familiar CICD tools to manage the Big Data Application Life Cycle while ensuring efficient incident management using Control-M and Service Now.
Top Skills: PysparkScalaSpark
7 Days Ago
Remote
Hybrid
Chennai, Tamil Nadu, IND
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves analyzing credit risk, managing regulatory reporting such as IFRS9 and Basel 3, and delivering business transformation projects. Key responsibilities include executing complex change management initiatives and performing data and business analysis with a focus on finance-related regulations.
Top Skills: SQL

What you need to know about the Hyderabad Tech Scene

Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account