OneMagnify Logo

OneMagnify

Databricks/PySpark offshore Developer

Job Posted 2 Days Ago Posted 2 Days Ago
Be an Early Applicant
Chennai, Tamil Nadu
Mid level
Chennai, Tamil Nadu
Mid level
The role involves designing and developing data ingestion pipelines, optimizing Databricks jobs, mentoring junior developers, and automating ETL processes. Candidates will collaborate with various stakeholders, document best practices, and support software applications while performing data migration and ensuring thorough unit testing.
The summary above was generated by AI

Looking for an offshore Lead Databricks/PySpark Developer who is willing to learn new technologies if needed and able to work with team. This position is long term and will likely be renewed annually.

ESSENTIAL DUTIES AND RESPONSIBILITIES

  • Design and development of data ingestion pipelines (Databricks background preferred). 
  • Performance tune and optimize the databricks jobs 
  • Evaluated new features and refractors existing code  
  • Mentor junior developers and makes sure all patterns are documented 
  • Perform data migration and conversion activities. 
  • Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures. 
  • Collaborate with Business Analysts, Architects and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments). 
  •  Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform. 
  •  Maintain and support the application. 
  •  Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary 
  •  Ensures to understand the requirements thoroughly and in detail and identify gaps in requirements 
  •  Ensures that detailed unit testing is done, handles negative scenarios and document the same 
  •  Work with QA and automation team. 
  •  Works on best practices and documenting the process 
  •  code merges and releases (Bitbucket) 
  •  Works with architect and manager on designs and best practices 
  •  Good data analysis skills
     

EDUCATION AND/OR EXPERIENCE

  • Must be a team player. 
  • Must have following 

SCALA 

SQL 

Spark/Spark Streaming 

Big Data Tool Set 

Linux 

Python/PySpark 

Kafka 

  •  Experience collaborating with dev team, project managers, and engineers. 
  •  Excellent communication and teamwork skills

About us

Whether it’s awareness, advocacy, engagement, or efficacy, we move brands forward with work that connects with audiences and delivers results. Through meaningful analytics, engaging communications and innovative technology solutions, we help clients tackle their most ambitious projects and overcome their biggest challenges.

We are an equal opportunity employer

We believe that Innovative ideas and solutions start with unique perspectives. That’s why we’re committed to providing every employee a workplace that’s free of discrimination and intolerance. We’re proud to be an equal opportunity employer and actively search for like-minded people to join our team.

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform job functions, and to receive benefits and privileges of employment. Please contact us to request accommodation.

  • Safeguard the company’s assets.
  • Adhere to the company’s compliance program.
  • Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices.
  • Maintain a focus on customer-service, efficiency, quality, and growth.
  • Collaborating with additional team members
  • Other duties as assigned.

Similar Jobs

6 Hours Ago
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Mid level
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
The Consultant, AEM Development is responsible for enhancing and developing AEM components, optimizing website performance, integrating APIs, and collaborating with cross-functional teams ensuring adherence to best practices in SEO and web accessibility. This role requires hands-on expertise in Adobe Experience Manager, including its architecture and components, along with proficiency in Java and web technologies.
Top Skills: Adobe Experience ManagerAemaacsCSSHtlJavaJavaScriptJcrJqueryOsgiSeoSling
6 Hours Ago
2 Locations
Junior
Junior
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
The Senior Associate, Data Solutions Engineering will develop and maintain data warehouse/data lake solutions, build data pipelines, curate and model data for analytics, and collaborate with various teams to provide insights in support of Pfizer's commercial objectives.
Top Skills: ETLJavaPythonRdbmsSnowflakeSQL
Mid level
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
As a Senior Associate in AI and Data Science Full Stack Engineering at Pfizer, you will implement AI solutions at scale, develop data science workflows, and create software solutions. Responsibilities include building data applications, maintaining infrastructure, and collaborating on AI and data science model integration.
Top Skills: AngularjsAnsibleAWSCloudFormationDashDataiku Data Science StudioDockerGitNoSQLPower BIPythonRReactRestful ApisSQLStreamlitTableauTerraformVue

What you need to know about the Hyderabad Tech Scene

Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account