Designs Databricks data ingestion pipelines using PySpark and PySQL, develops and deploys Databricks Apps, and configures permissions, tokens, and service principals. Builds AI applications with FastAPI, React, Claude APIs, prompt engineering, vision processing, retrieval augmented generation, guardrails, graph analysis, and caching. Performs scenario-based testing, translates user requirements into code changes, and integrates custom notifications through APIs and webhooks.
Databricks
- Design and build pipelines for ingesting data into Databricks from various sources like SAP, website scraping. Core technical skill will be PySpark and PySQL
- Write and do scenario-based testing, edge cases and discuss with the stake holders to finalize the necessary code changes and acceptance criteria.
- Understanding of the Databricks Unity Catalog permissions model and how to use OBO tokens and configure Service Principals for Databricks Agents,Genie Spaces, Machine Learning and Foundation Models.
- Knowledge of Databricks Apps. Ability to build and Deploy Databricks Apps using Visual Studio Code.
- Sending custom notifications from Databricks using APIs to custom Team Channel Webhooks
AI:
- Ability to use FastAPI /React front end and build minimal Single Page Applications(SPAs) using Chat UI interfaces and Claude Models/APIs
- Ability to use defined System Prompts, User Prompts etc..(i..e,) skills in prompt engineering.
- Basic Knowledge of Token Economics and ability use rule enforcement in prompts in addition to standard NLP language.
- Ability to reengineer code and see what it does and map it to the requirement specifications from users
- Ability to use Graphs to detect inter dependencies between rules and circular references.
- Ability to use both Claude Vision API in addition to standard text processing.
- Understand and develop Retrieval Augmented Generation specifically to Databricks like AI Search indexes, Hierarchical Chunking etc..
- Know how to implement basic guard rails related to AI safety
- Understand various caching mechanisms
Tools:
- Ability to work with Claude Code and Genie.
Similar Jobs
Automotive
Build scalable Python backend services, APIs, cloud-native applications, and AI-powered solutions. Develop LLM, RAG, agentic, and multi-agent systems; deploy workloads with Docker and Kubernetes; maintain CI/CD pipelines; and operate applications across major cloud platforms. Collaborate cross-functionally to deliver secure, tested, observable, and production-ready software while optimizing performance, scalability, latency, and cost.
Top Skills:
Amazon Q DeveloperArgocdAWSAzureClaude CodeCursorDockerFaissGitGithub ActionsGithub CopilotGitlab CiGoogle AdkGoogle Cloud PlatformGraphQLHelmJenkinsKubeflowKubernetesLangchainLanggraphLarge Language ModelsMicroservicesMilvusMlflowNosql DatabasesPineconePythonRelational DatabasesRestTerraformTorchserveTriton Inference ServerVllmWeaviateWeights & BiasesWindsurf
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Leads the architecture, development, and productionization of AI/ML and GenAI solutions for healthcare operations. Responsibilities include building clinical and claims data pipelines, developing LLM and RAG applications, implementing MLOps and LLMOps, ensuring HIPAA-compliant governance, translating business needs into technical roadmaps, communicating with stakeholders, and mentoring engineers.
Top Skills:
AirflowAWSAzureAzure MlBigQueryCi/CdDaskDatabricksDockerFaissGCPGitGoHugging FaceJavaKafkaKubernetesLangchainLlamaindexMlflowNeo4JNumpyPandasPgvectorPineconePrefectPytestPythonPyTorchSagemakerScalaSnowflakeSparkSQLTensorFlowTerraformTransformersVertex Ai
Artificial Intelligence • Fintech • Hardware • Information Technology • Sales • Software • Transportation
Build and operate production software and AI systems across cloud infrastructure. Responsibilities include developing APIs and services, operating RAG and inference platforms, implementing CI/CD and observability, managing AWS or GCP environments with Kubernetes and Terraform, optimizing model serving and GPU usage, controlling cloud and AI costs, and partnering with stakeholders on reliability, security, governance, and technical tradeoffs.
Top Skills:
AWSCi/CdFinopsGCPGoJavaKubernetesLlmsNext.JsNode.jsPythonPyTorchRagReactTensorFlowTerraformTypescriptVector Databases
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
