Design, build, and maintain scalable data pipelines and models using Big Data and cloud technologies. Ensure data quality, security, SLA adherence, and collaborate with analytics and ML teams. Monitor, optimize, and document ETL/ELT processes.
Role- Data Engineer
Experience- 3-5 Yrs
Location - Hyderabad & Bangalore (Hybrid)
Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines to support various data processing and analytics needs.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
- Ensure data quality and integrity by implementing best practices for data validation, cleansing, and transformation.
- Develop and optimize data models to support reporting, analytics, and machine learning applications.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability of data processing systems.
- Work with Hadoop and related technologies to manage and process large datasets.
- Implement and maintain data security and privacy measures in compliance with organizational and regulatory requirements.
- Integrate diverse data sources, both structured and unstructured, into a cohesive data ecosystem.
- Document data engineering processes, standards, and best practices for knowledge sharing and continuity.
Required Skills :
- Experience in Big data Distributed ecosystems:- Hadoop, Hive Excellent knowledge of HQL/PrestoQL:- Optimisations, complex aggregations, performance tuning
- Experience building data processing frameworks, and big data pipelines. Solid understanding of DWH architecture, ELT/ETL processes, and data structures
- Basic Understanding of Python:- ETL
- Keep the SLA of maintained datasets(Tier 1, 2) to above 99%. (On-call)
- Maintain the SLAs for Tier 3 tables as decided
- Experience with big data technologies (e.g., Hadoop, Spark) and data warehousing solutions (e.g., Redshift, BigQuery).
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with ETL tools and frameworks (e.g., Apache NiFi, Talend, Informatica).
- Familiarity with cloud-based data platforms (e.g., AWS Glue, Azure Data Factory, Google Dataflow).
About Company :
Nineleaps employs AI stratagems to help global organizations breathe and oxygenate data into valuable outcomes across product and platform engineering functions. We thrive by creating high-performance business applications utilizing unique solutions with Cloud, AIOps, our NineX development platform, and other proprietary frameworks that have helped Global Startups and Fortune 500 companies experience value-driven collaboration with us. From catering to global applications through a single code base to deriving actionable insights from petabyte-scale data, our web of capabilities has helped us catalyze transformations while emphasizing human experiences.
Websitehttps://www.nineleaps.com/
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
The Sr. Data Engineer at Micron will design and implement data management systems, develop ETL processes, and collaborate with cross-functional teams to prepare data for analysis.
Top Skills:
Apache Ni-FiAWSAzureGCPLinuxMS OfficeNoSQLSnowflakeSparkSQLUnix
Information Technology
The Senior Data Engineer focuses on ETL processes, data modeling, and utilizing Databricks for data management and transformation.
Top Skills:
Data WarehousingDatabricksETLPlsqlPythonSQLStored ProceduresT-Sql
Artificial Intelligence • Big Data • Cloud • Security • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Lead design, implementation, optimization, and maintenance of performance-critical data-path components. Improve concurrency, throughput, latency, scalability, and reliability; diagnose complex defects and escalations; partner with QA and Performance Engineering; define test strategies; influence architecture, coding standards, and cross-team engineering practices.
Top Skills:
Ai-Assisted Engineering ToolsApplication-Level ProtocolsCC++CachingConsistency ModelsDatabasesDiagnostic ToolsDistributed SystemsFile SystemsGoLinuxMetadata ServicesNasNetworkingObject StorageProfilingPythonReplicationSnapshotsStorage SystemsTracing
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
