Design, build, and maintain scalable data pipelines and models using Big Data and cloud technologies. Ensure data quality, security, SLA adherence, and collaborate with analytics and ML teams. Monitor, optimize, and document ETL/ELT processes.
Role- Data Engineer
Experience- 3-5 Yrs
Location - Hyderabad & Bangalore (Hybrid)
Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines to support various data processing and analytics needs.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
- Ensure data quality and integrity by implementing best practices for data validation, cleansing, and transformation.
- Develop and optimize data models to support reporting, analytics, and machine learning applications.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability of data processing systems.
- Work with Hadoop and related technologies to manage and process large datasets.
- Implement and maintain data security and privacy measures in compliance with organizational and regulatory requirements.
- Integrate diverse data sources, both structured and unstructured, into a cohesive data ecosystem.
- Document data engineering processes, standards, and best practices for knowledge sharing and continuity.
Required Skills :
- Experience in Big data Distributed ecosystems:- Hadoop, Hive Excellent knowledge of HQL/PrestoQL:- Optimisations, complex aggregations, performance tuning
- Experience building data processing frameworks, and big data pipelines. Solid understanding of DWH architecture, ELT/ETL processes, and data structures
- Basic Understanding of Python:- ETL
- Keep the SLA of maintained datasets(Tier 1, 2) to above 99%. (On-call)
- Maintain the SLAs for Tier 3 tables as decided
- Experience with big data technologies (e.g., Hadoop, Spark) and data warehousing solutions (e.g., Redshift, BigQuery).
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with ETL tools and frameworks (e.g., Apache NiFi, Talend, Informatica).
- Familiarity with cloud-based data platforms (e.g., AWS Glue, Azure Data Factory, Google Dataflow).
About Company :
Nineleaps employs AI stratagems to help global organizations breathe and oxygenate data into valuable outcomes across product and platform engineering functions. We thrive by creating high-performance business applications utilizing unique solutions with Cloud, AIOps, our NineX development platform, and other proprietary frameworks that have helped Global Startups and Fortune 500 companies experience value-driven collaboration with us. From catering to global applications through a single code base to deriving actionable insights from petabyte-scale data, our web of capabilities has helped us catalyze transformations while emphasizing human experiences.
Websitehttps://www.nineleaps.com/
Similar Jobs
Healthtech • Software
As a Data Engineer, you will develop and maintain scalable data pipelines using AWS, support data processing, and collaborate with teams to deliver high-quality data solutions.
Top Skills:
AirflowApache IcebergAws AthenaAws GlueAws S3DbtParquetPysparkPythonSparkSQL
Cloud • Information Technology • Security • Software • Cybersecurity
As a Principal Site Reliability Engineer at Zscaler, you'll manage cloud infrastructure and enhance automation while ensuring compliance and operational excellence.
Top Skills:
AnsibleAWSKubernetesLinuxPythonTerraform
Healthtech • Software
Lead the design and delivery of a cloud-native healthcare data platform on AWS. Architect and implement scalable batch and near-real-time pipelines, lead Apache Iceberg lakehouse engineering, build reusable ingestion/transformation/validation frameworks, drive data quality, observability, and operational excellence, review architectures, conduct code reviews, mentor engineers, and collaborate across platform, analytics, and business teams.
Top Skills:
AirflowApache IcebergAthenaAws S3DbtDelta LakeGlueGlue CatalogHudiIamLake FormationPysparkPythonSparkSQLStep Functions
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
