Company Description
About Experian
Experian is the world's leading global information services company. During life's big moments — from buying a home or a car to sending a child to college to growing a business by connecting with new customers — we empower consumers and our clients to manage their data with confidence. We help individuals to take financial control and access financial services, businesses to make smarter decisions and thrive, lenders to lend more responsibly, and organizations to prevent identity fraud and crime.
We have 17,800 people operating across 44 countries, and every day we're investing in new technologies, talented people and innovation to help all our clients maximize every opportunity. We are listed on the London Stock Exchange (EXPN) and are a constituent of the FTSE 100 Index.
Learn more at www.experianplc.com or visit our global content hub at our global news blog for the latest news and insights from the Group.
Job Description
We are seeking a highly skilled and experienced Engineering Manager to lead and manage our big data and cloud-based solutions team. The ideal candidate will have deep expertise in AWS services like EMR, Athena, Glue, S3, and Apache Airflow, along with strong proficiency in Big Data Technologies using Python and PySpark.
As an Engineering Manager, you will be responsible for designing, developing, and optimizing large-scale Batch data processing applications, while also leading a team of talented engineers. You will work closely with cross-functional teams to deliver scalable and reliable solutions that enable high-performance data processing and analytics in the cloud.
Key Responsibilities:
- Lead, mentor, and manage a team of engineers working on cloud-based big data applications and services.
- Design, develop, and maintain large-scale Batch data processing pipelines using AWS services (EMR, Athena, Glue, S3) and Apache Airflow.
- Architect and implement data solutions using Python, PySpark, and big data frameworks to handle complex datasets efficiently.
- Collaborate with product management, Business, Platform Infrastructure and DevOps teams to define project requirements and deliver high-quality, scalable solutions.
- Ensure the team's development efforts follow best practices in coding, security, and data management.
- Optimize performance, reliability, and cost-effectiveness of data pipelines and cloud infrastructure.
- Drive continuous improvement in engineering processes, ensuring the team’s deliverables meet quality and performance benchmarks.
- Stay updated on emerging technologies and industry trends, and guide the adoption of new tools and processes to enhance engineering efficiency.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 8 - 10 years of experience in software engineering, with at least 3+ years in a leadership or management role.
- Strong hands-on experience with AWS services including EMR, Athena, Glue, S3, and Apache Airflow.
- Expertise in Big Data technologies, especially with Python, PySpark, and distributed data processing frameworks.
- Proven experience in designing and developing scalable and reliable data pipelines and applications.
- Solid understanding of data architecture, data modeling, and ETL processes.
- Strong leadership, communication, and project management skills, with the ability to collaborate effectively with cross-functional teams.
- A background in cloud infrastructure and DevOps practices for big data environments is a plus.
Additional Information
- Experience with other cloud platforms or tools, such as Google Cloud Platform (GCP) or Azure.
- Knowledge of data warehousing, data lakes, and streaming data technologies.
- Experience working in an Agile/Scrum development environment.
Experian Careers - Creating a better tomorrow together
Find out what its like to work for Experian by clicking here
