Senior Data Engineer-AWS, Python

Responsibilities:

• Team & Operational Management
• Team Management: Manage the team of 3-4 Data Engineers.
• Roster & Scheduling: Create, manage, and maintain team rosters and schedules to ensure full 8-hour coverage, 7 days a week, including rotational weekend and holiday coverage.
• Availability: Must be able to work a day shift and manage a team that provides 7-day-a-week coverage, including weekends and holidays on rotation.
• Coverage Assurance: Handle all team leave requests (planned and unplanned). You are responsible for ensuring coverage is always maintained, especially during medical or other unexpected absences.
• Performance & SLAs: Act as the primary operational point of contact, ensuring the team executes all tasks efficiently, makes sound decisions, and adheres to all agreed-upon SLAs.
• Technical Leadership & Engineering
• Final Escalation Point: Serve as the final technical escalation point for the team; personally handle the most critical job failures and system issues.
• Expert RCA: Lead all high-priority Root Cause Analysis (RCA) efforts, documenting solutions and implementing permanent fixes to prevent recurrence.
• Strategic Improvement: Drive the team's efforts in performance tuning and AWS Glue cost optimization.
• Automation Strategy: Define and lead the development of new automation jobs (using Glue, Lambda, and Step Functions) to reduce manual support tasks.
• Complex Data Management: Oversee all data integrity and synchronization tasks, including CDC reloads and write-backs to external databases like Teradata.
• Knowledge Transfer: Own the knowledge transfer process to internal resources and ensure the team maintains excellent documentation and runbooks.

Qualifications:

• Technical Expertise: In-depth, expert-level knowledge and extensive hands-on experience with AWS S3, Glue, Lambda, Step Functions and other related services.
• Demonstrated expertise with a minimum of 5-8 years of relevant experience.
• Deep understanding of Data engineering, ETL principles and data warehouse design.
• Optimization Experience: Demonstrable experience in system performance tuning and cloud cost optimization, specifically with AWS Glue.
• Automation: Proven ability to design and develop automation solutions from the ground up.
• Problem-Solving: Expert-level troubleshooting and RCA skills.
• Leadership: Ability to work independently, make critical decisions, and provide technical guidance.
• Availability: Must be able to work a day shift that includes coverage for weekends and holidays on a rotational basis.
• Database Skills: Strong experience with Change Data Capture (CDC), managing database synchronization, and working with Teradata.
• Preferred Skills: AWS Certified.