Designs and optimizes scalable data pipelines using Python and PySpark, develops secure REST APIs, and deploys cloud applications on AWS. The role involves processing large datasets, managing distributed computing solutions, troubleshooting technical issues, optimizing performance, and collaborating with cross-functional teams to deliver reliable software and data engineering solutions.
Python / PySpark Developer
Summary
We are seeking a highly skilled Python and PySpark Developer to join our IT Services team. In this pivotal role, you will design, develop, and optimize scalable data processing solutions leveraging the power of Python and the PySpark framework. This position is critical for driving our data engineering initiatives, ensuring efficient handling of large datasets, and supporting robust application architectures within our cloud environment. The ideal candidate will possess deep expertise in distributed computing and API integration to deliver high-performance software that meets complex business requirements.
Responsibilities
- Architect and implement robust data pipelines using Python and PySpark to process and transform large-scale datasets efficiently.
- Design, develop, and maintain secure and scalable RESTful APIs to facilitate seamless data exchange between systems.
- Deploy, monitor, and manage cloud infrastructure and applications on AWS, ensuring high availability and performance.
- Collaborate with cross-functional teams to define technical specifications and translate business needs into functional software solutions.
- Optimize code for maximum speed, scalability, and reliability while adhering to best coding practices and security standards.
- Troubleshoot and resolve complex technical issues related to data processing, API connectivity, and cloud deployments.
Requirements
Requirements:
- Minimum of 6 to 9 years of professional experience in software development, with a strong focus on data engineering and backend systems.
- Proven expertise in Python programming, including mastery of its libraries and frameworks for data manipulation.
- Extensive hands-on experience with Apache PySpark for building distributed data processing applications.
- Solid understanding and practical application of AWS cloud services for deployment and infrastructure management.
- Demonstrated proficiency in designing and consuming REST APIs within enterprise-level applications.
- Ability to work effectively in a dynamic IT Services environment, delivering high-quality code under tight deadlines.
Similar Jobs
Edtech • HR Tech • Information Technology • Professional Services
Designs, builds, and optimizes scalable ETL and ELT pipelines using Python and PySpark. Develops high-performance applications for large-scale data processing, integrates AWS data services, writes advanced SQL, and builds REST APIs. Ensures cloud-based data solutions are secure, scalable, highly available, and compliant while collaborating with cross-functional teams to deliver business-focused data engineering solutions.
Top Skills:
AWSEltETLPysparkPythonRelational DatabasesRest ApisScalaSQL
Fintech • Mobile • Payments • Software • Financial Services
Conduct customer and business onboarding through CDD and EDD reviews, assess identity documents, ownership structures, source of wealth, adverse media, and transaction activity. Maintain accurate KYC records, perform risk assessments, identify money laundering and fraud indicators, escalate risks, and file Suspicious Activity Reports when required. Communicate with customers, meet strict KPIs, provide process feedback, and work flexibly across shifts.
Fintech • Mobile • Payments • Software • Financial Services
Recruiter responsible for end-to-end hiring for Wise’s Servicing Operations teams in Hyderabad. The role manages candidate experiences, partners with hiring managers and sourcers, builds talent networks, uses recruiting data and dashboards to optimize processes, and supports talent initiatives such as referrals, talent mapping, and salary benchmarking.
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
