Lead Data Engineer - Python, Pyspark

Job Description
Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.
As a Lead Data Engineer at JPMorganChase within the Payments Technology team, you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm's business objectives.
Job Responsibilities

• Lead cross-functional collaboration: Partner with stakeholders across all JPMorgan lines of business and functions to deliver robust software and data engineering solutions.
• Drive innovation and architecture: Spearhead the experimentation, design, development, and production deployment of advanced data pipelines, data services, and data platforms that directly support business objectives.
• Architect scalable data solutions: Design and implement highly scalable, efficient, and reliable data processing pipelines, leveraging advanced analytics to generate actionable business insights and optimize outcomes.
• Integrate security architecture: Proactively address opportunities to unify physical, IT, and data security architectures, ensuring comprehensive access management and data protection.

Required Qualifications, Capabilities, and Skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Extensive experience in data technologies, with formal training or certification in large-scale technology program management.
• Advanced programming expertise: Proficient in Java and Python, with a strong track record of building and optimizing data frameworks and solutions.
• Comprehensive data lifecycle knowledge: Deep experience in architecting and managing data frameworks, including data lakes, and overseeing the full data lifecycle.
• Batch and real-time processing: Proven expertise in developing batch and real-time data processing solutions using Spark or Flink.
• Cloud data processing: Hands-on experience with AWS Glue and EMR for scalable data processing tasks.
• Databricks proficiency: Demonstrated ability to leverage Databricks for advanced analytics and data engineering.
• Service development and deployment: Skilled in building services using Spring Boot or Flask, and deploying them on AWS EKS or Kubernetes.
• Database management: Strong working knowledge of both relational and NoSQL databases, with experience in ETL pipeline development for batch and real-time processing, data warehousing, and NoSQL solutions.

Preferred Qualifications, Capabilities, and Skills

• Cloud and containerization: Expertise in Amazon Web Services (AWS), Docker, and Kubernetes for cloud-native and containerized data solutions.
• Big data technologies: Advanced experience with Hadoop, Spark, and Kafka for distributed data processing and streaming.
• Distributed systems: Proven ability to design and develop distributed systems for large-scale data engineering applications.

About Us
JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world's most prominent corporate, institutional and government clients under the J.P. Morgan and Chase brands. Our history spans over 200 years and today we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing and asset management.
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
About the Team
J.P. Morgan's Commercial & Investment Bank is a global leader across banking, markets, securities services and payments. Corporations, governments and institutions throughout the world entrust us with their business in more than 100 countries. The Commercial & Investment Bank provides strategic advice, raises capital, manages risk and extends liquidity in markets around the world.