Data Engineer
Experience Level: MID
Position Overview:
This role will be responsible for expanding our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler, who enjoys optimizing data systems and building them from the ground up.
This position will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
Roles and Responsibilities:
- Design, implement, and maintain scalable data pipelines for batch and real-time data processing using tools like Apache Airflow, Apache Kafka, and Azure Data Factory.
- Build ETL/ELT workflows, automate data movement, and integrate data from multiple sources (APIs, databases, third-party systems).
- Work with cloud platforms (e.g., AWS, Azure) and tools like Snowflake, Amazon Redshift, Azure Data Lake, and S3 to manage large-scale data storage.
- Ensure data quality through automated testing, monitoring, and validation of data pipelines.
- Collaborate with Data Analysts, Data Scientists, and other stakeholders to understand and deliver on data needs.
- Optimize and tune data pipelines for performance, scalability, and cost efficiency.
- Maintain documentation and adhere to best practices for data engineering and architecture.
- Continuously stay up-to-date with new technologies and industry trends, improving systems and processes as needed.
Must-Have Skills:
- 4+ years of experience in Data Engineering, with expertise in building data pipelines and managing cloud-based data infrastructure.
- Proficiency in Python, SQL, and Java/Scala for data processing and transformation.
- Experience with cloud platforms (e.g., AWS, Azure) and tools like Snowflake, Redshift, S3, Azure Data Factory.
- Hands-on experience with data orchestration tools like Apache Airflow and Apache Kafka for real-time data workflows.
- Strong knowledge of ETL/ELT processes, data modeling, and data quality practices.
- Ability to troubleshoot, optimize, and maintain high-performance data pipelines.
- Experience working in cross-functional teams and collaborating with Data Scientists and Data Analysts.
Nice-to-Have Skills:
- Familiarity with data governance and privacy regulations (e.g., GDPR, CCPA).
- Experience with NoSQL databases (MongoDB, Cassandra), Hadoop, or Apache Spark.
- Knowledge of machine learning pipelines or integration with MLflow, TensorFlow, PyTorch.
- Experience with BI tools like Power BI, Tableau, or Looker for data visualization.
Qualification:
- Bachelor / Masters or equivalent degree in Computer Science or related field.
Location:
- Hyderabad / Bangalore
About Nomiso:
Nomiso is a product and services engineering company. We are a team of Software Engineers, Architects, Managers, and Cloud Experts with expertise in Technology and Delivery Management.
Our mission is to Empower and Enhance the lives of our customers, through efficient solutions for their complex business problems.
At Nomiso we encourage entrepreneurial spirit - to learn, grow and improve. A great workplace, thrives on ideas and opportunities. That is a part of our DNA. We’re in pursuit of colleagues who share similar passions, are nimble and thrive when challenged. We offer a positive, stimulating and fun environment – with opportunities to grow, a fast-paced approach to innovation, and a place where your views are valued and encouraged.
We invite you to push your boundaries and join us in fulfilling your career aspirations!
We are an equal opportunity employer and are committed to diversity, equity, and inclusion. We do not discriminate on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability status, or any other protected characteristics.
