Designs, develops, and maintains scalable data pipelines, data architectures, quality processes, and integrations across varied sources. Troubleshoots data issues, monitors solutions to meet SLAs, documents infrastructure, and collaborates with stakeholders and cross-functional teams. Requires strong SQL, database management, dimensional and relational modeling, and Python or Java experience, with preferred expertise in cloud data warehouses, big data, ETL tools, orchestration, BI dashboards, and Agile delivery.
Responsibilities:
- Design, develop and maintain scalable and efficient data pipelines
- Analyze and Develop data quality and validation procedures.
- Work with stakeholders to understand the data requirements and provide solutions.
- Troubleshoot and resolve data issues in a timely manner
- Collaborate with cross-functional teams to ingest data from various sources.
- Continuously evaluate and improve data architecture and processes.
- Own, monitor, and improve solutions to ensure SLAs are met
- Develop and maintain documentation for Data infrastructure and processes.
- Executes projects using Agile Scrum methodologies. Team player.
What you will bring:
Basic Qualifications
- Bachelor's Degree in Computer Science, Data Analytics, Information Systems, etc.
- Experience developing data pipelines in one of the following languages: Python or Java
- 5+ years dimensional and relational data modeling experience
- Excellent SQL and database management skills
Preferred Qualifications
- 5+ years in data warehouse engineering (OLAP) Snowflake, BigQuery, Teradata, Redshift
- 5+ years with transactional databases (OLTP) Oracle, SQL Server, MySQL
- 5+ years with big data, Hadoop, Data Lake, Spark in a cloud environment(AWS)
- 5+ years with commercial ETL tools – dbt, Matillion, and SSIS/ADF
- 5+ years delivering ETL solutions from source systems, databases, APIs, flat-files, JSON
- Experience developing Entity Relationship Diagrams with Erwin, SQLDBM, or equivalent
- Experience working with job scheduling and monitoring systems (Airflow, Datadog, AWS SNS)
- Experience with Claude Code, Cortex Coco, VSCode copilot, AI enabled SDLC development models.
- Experience building BI Dashboards with tools like Tableau
- Experience in any of the financial, Marketing, Sales, accounts payable, accounts receivable, invoicing etc.. Domains are a plus.
- Experience managing work assignments using tools like Jira and Confluence
- Experience with Scrum/Agile methodologies is a plus.
- Ability to work independently and as part of a team
- Excellent analytical and problem solving skills
Similar Jobs
Information Technology • Professional Services • Software • App development
Lead the design, implementation, maintenance, and monitoring of scalable Big Data pipelines. Build Java/Scala-based systems using distributed processing frameworks, ETL tools, and cloud-based infrastructure. Review and optimize existing pipelines, troubleshoot production issues, lead technical teams, coordinate with US and India engineers, and deliver new modules and features using Agile practices.
Top Skills:
AkkaApache AirflowApache NifiSparkApache StormGitGradleHadoopJavaJIRAMavenSbtScala
Information Technology • Security • Cybersecurity
Design, build, and operate high-throughput data platform components and pipelines using Spark and Kafka. Debug, tune, and optimize jobs for massive transaction volumes, collaborate on architecture and standards, and mentor junior engineers.
Top Skills:
Apache KafkaSparkJavaPythonScala
Information Technology • Security • Cybersecurity
Lead design, development, and support of a scalable, distributed SaaS security risk platform. Provide technical leadership on data platform, pipelines, and architecture; prototype and implement big data solutions; ensure performance, scalability, security and compliance; mentor engineers; collaborate with product, services and sales; perform benchmarking, troubleshooting, and guide cross-team delivery.
Top Skills:
AirflowApache FlinkSparkCassandraData LakeElasticsearchHadoopJavaKafkaKappa ArchitectureLambda ArchitectureMessage QueueMicroservicesNoSQLOraclePrestoRdbmsRedisRule EngineScalaSolrUnix
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
