QA Data Science Engineer

Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!

Job Description

We are seeking a Data Science focused QA engineer to develop next-generation Security Analytics products. You will work closely with Data scientists,   engineers   and product managers to design and   optimize   AI driven security solutions.

As   QA   engineer, the ideal candidate has a strong background in Backend engineering, system integrations,   ML,AI   and data pipelines.

Responsibilities (QA Engineer – Data Science / ML)

Establish QA best practices for Traditional ML and Generative AI workflows, including:

Functional and regression testing of ML pipelines using   pytest   and Airflow/ Dagster   test utilities   and API testing tools (e.g., Postman,   pytest-httpx ).

Validate data contracts, schemas, and API compatibility across services using   Pandera , and custom validation rules .

Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions

Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests .

Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).

Implement LLM-specific testing, including:

Prompt and response validation, determinism checks, and regression testing using   LangSmith .

Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and /or   rule-based checks .

Cost, token usage, and timeout monitoring for GenAI workflows

Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.

Requirements:

BS or MS  in  Computer Science or a related field .

2-5 years  of experience in Data   or Machine   Learning  projects .

Familiarity and  experience   of GenAI applications   and tools - PyTorch ,  LangChain ,  vLLM  etc.

Demonstrates  a commitment to continuous learning in this rapidly evolving field.

Tools listed in   the responsibilities   section.