Design, build, and maintain scalable data pipelines and models using Big Data and cloud technologies. Ensure data quality, security, SLA adherence, and collaborate with analytics and ML teams. Monitor, optimize, and document ETL/ELT processes.
Role- Data Engineer
Experience- 3-5 Yrs
Location - Hyderabad & Bangalore (Hybrid)
Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines to support various data processing and analytics needs.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
- Ensure data quality and integrity by implementing best practices for data validation, cleansing, and transformation.
- Develop and optimize data models to support reporting, analytics, and machine learning applications.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability of data processing systems.
- Work with Hadoop and related technologies to manage and process large datasets.
- Implement and maintain data security and privacy measures in compliance with organizational and regulatory requirements.
- Integrate diverse data sources, both structured and unstructured, into a cohesive data ecosystem.
- Document data engineering processes, standards, and best practices for knowledge sharing and continuity.
Required Skills :
- Experience in Big data Distributed ecosystems:- Hadoop, Hive Excellent knowledge of HQL/PrestoQL:- Optimisations, complex aggregations, performance tuning
- Experience building data processing frameworks, and big data pipelines. Solid understanding of DWH architecture, ELT/ETL processes, and data structures
- Basic Understanding of Python:- ETL
- Keep the SLA of maintained datasets(Tier 1, 2) to above 99%. (On-call)
- Maintain the SLAs for Tier 3 tables as decided
- Experience with big data technologies (e.g., Hadoop, Spark) and data warehousing solutions (e.g., Redshift, BigQuery).
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with ETL tools and frameworks (e.g., Apache NiFi, Talend, Informatica).
- Familiarity with cloud-based data platforms (e.g., AWS Glue, Azure Data Factory, Google Dataflow).
About Company :
Nineleaps employs AI stratagems to help global organizations breathe and oxygenate data into valuable outcomes across product and platform engineering functions. We thrive by creating high-performance business applications utilizing unique solutions with Cloud, AIOps, our NineX development platform, and other proprietary frameworks that have helped Global Startups and Fortune 500 companies experience value-driven collaboration with us. From catering to global applications through a single code base to deriving actionable insights from petabyte-scale data, our web of capabilities has helped us catalyze transformations while emphasizing human experiences.
Websitehttps://www.nineleaps.com/
Similar Jobs
Healthtech • Software
As a Data Engineer, you will develop and maintain scalable data pipelines using AWS, support data processing, and collaborate with teams to deliver high-quality data solutions.
Top Skills:
AirflowApache IcebergAws AthenaAws GlueAws S3DbtParquetPysparkPythonSparkSQL
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Lead and manage software product development and engineering teams, collaborate with product/design/support, mentor staff, allocate resources, solve complex problems, and ensure high-quality, standards-compliant solutions.
Top Skills:
Agile DevelopmentAIC++JavaJavaScriptOpen Source ToolsServicenowServicenow Ai
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Senior client-facing leader who drives large-scale enterprise data transformation programs for financial services, owning delivery, governance, stakeholder engagement, migration, MDM/reference data, data quality, remediation, and mentoring global teams to meet aggressive timelines and quality targets.
Top Skills:
Data GovernanceData MigrationData QualityEnterprise Data ModernizationMdmReconciliationReference Data
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
