Designs and executes data quality testing for end-to-end ETL/ELT pipelines. Responsibilities include source-to-target validation, reconciliation, business rule verification, SQL and Python automation, Databricks and Delta Lake testing, CI/CD integration, failure analysis, and data integrity validation. The role also evaluates emerging AI and RAG pipelines and collaborates with data engineering teams to resolve discrepancies.
Job Description: Data Quality Engineer (Data QE)
Location: Hyderabad, India
Experience: 5+ Years
Company: Anblicks
About AnblicksAnblicks is a Great Place to Work® Certified™ Data and AI company, helping enterprises accelerate digital transformation through modern data platforms, cloud engineering, analytics, and AI-driven solutions.
Role OverviewWe are looking for a skilled Data Quality Engineer with strong expertise in SQL, Python, ETL/ELT testing, and data test automation. The candidate will ensure data accuracy, integrity, and reliability across complex data pipelines using modern platforms such as Databricks and Snowflake. Exposure to Generative AI, LLMs, RAG pipelines, and AI agents is an added advantage.
Key Responsibilities- Design and execute data quality testing strategies for end-to-end data pipelines.
- Perform ETL/ELT testing, source-to-target validation, data reconciliation, and business rule verification.
- Develop advanced SQL queries and Python scripts for data validation and test automation.
- Validate Databricks workflows, Delta tables, and Bronze, Silver, and Gold data layers.
- Test incremental loads, CDC, schema changes, data integrity, and failure-recovery scenarios.
- Build reusable, configuration-driven data quality automation frameworks.
- Integrate automated tests into CI/CD pipelines and workflow orchestration tools.
- Investigate data discrepancies, perform root cause analysis, and collaborate with data engineering teams.
- Leverage AI-assisted tools to improve test case generation, SQL development, and failure analysis.
SQL & Python
- Strong SQL skills, including complex joins, CTEs, window functions, and aggregations.
- Proficiency in Python for data validation, reconciliation, and automation.
- Exposure to PySpark for large-scale data processing and validation.
Data Quality & Engineering
- Strong understanding of ETL/ELT testing, data warehousing, data modelling, and data profiling.
- Experience with source-to-target validation, business rules, and data integrity checks.
- Knowledge of batch/streaming pipelines and formats such as JSON, CSV, and Parquet.
Databricks & Cloud Platforms
- Hands-on experience with Databricks notebooks, Spark SQL, jobs, and workflows.
- Knowledge of Delta Lake, schema evolution, and Medallion Architecture.
- Exposure to Snowflake, Unity Catalog, and cloud data platforms is desirable.
Automation & CI/CD
- Experience building reusable SQL/Python-based test automation frameworks.
- Knowledge of PyTest or similar testing frameworks.
- Experience with Git, Jenkins, Azure DevOps, or equivalent CI/CD tools.
- Familiarity with Databricks Workflows or Apache Airflow.
- Understanding of Generative AI, LLMs, and AI-assisted testing.
- Exposure to RAG pipeline validation, including document quality, metadata, retrieval relevance, and response accuracy.
- Familiarity with LLM evaluation, AI agents, and responsible AI practices is an advantage.
- Experience with Great Expectations, Soda, or Deequ.
- Knowledge of Kafka, data observability, and anomaly detection.
- Exposure to AWS, Azure, GCP, RAG evaluation tools, and data governance.
Anblicks Hyderabad, Telangana, IND Office
Raidurg Panmakhta, Serilingampally Mandal, Ranga Reddy, , Hyderabad, Telangana , India, 500081
Anblicks Hyderabad, Telangana, IND Office
Hyderabad, Hyderabad, India
Similar Jobs
Edtech • HR Tech • Information Technology • Professional Services
Designs and executes ETL testing strategies, performs data reconciliation and integrity assessments, optimizes SQL queries, validates data across PostgreSQL, MSSQL, and Snowflake, and develops automated data quality testing suites to ensure reliable data pipelines and reporting.
Top Skills:
ETLMicrosoft Sql Server (Mssql)PostgresSnowflakeSQL
Artificial Intelligence • Information Technology • Software • Consulting
Ensure enterprise data quality, accuracy, integrity, and reliability across source systems, data warehouses, Medallion Architecture layers, Snowflake, and Power BI. Perform source-to-target validation, ETL/ELT testing, reconciliation, profiling, governance validation, lineage checks, and report testing. Develop automated data-quality frameworks and reusable test suites using SQL, Python, PySpark, and Soda, integrating them into Azure DevOps or GitHub Actions CI/CD pipelines.
Top Skills:
Azure Blob StorageAzure DevopsBigQueryGithub ActionsGoogle Analytics 4Microsoft Sql ServerPostgresPower BIPysparkPythonSnowflakeSodaSQL
Fintech • Mobile • Payments • Software • Financial Services
Conduct customer and business onboarding through CDD and EDD reviews, assess identity documents, ownership structures, source of wealth, adverse media, and transaction activity. Maintain accurate KYC records, perform risk assessments, identify money laundering and fraud indicators, escalate risks, and file Suspicious Activity Reports when required. Communicate with customers, meet strict KPIs, provide process feedback, and work flexibly across shifts.
What you need to know about the Hyderabad Tech Scene
Because of its proximity to leading research institutions and a government committed to the city's growth, Hyderabad's tech scene is booming. With plans to establish India's first "AI city," the city is on track to become one of the world's most anticipated tech hubs, with companies like TransUnion, Schrödinger and Freshworks, among others, already calling the city home.
