Senior Test Engineer- AI Testing

Responsabilities:
• Design and execute test strategies for machine learning models (classification, regression, NLP, CV, etc.).
• Validate model accuracy, fairness, bias detection, explainability, and robustness under varying datasets.
• Perform adversarial testing and edge-case validation for AI systems.
• Test autonomous agents, multi-step reasoning paths, and state transitions.
• Validate correctness, grounding, consistency, and safety of LLM outputs.
• Evaluate prompt robustness and behavioral variations across scenarios.
• Validate retrieval accuracy, grounding quality, and hallucination reduction.
• Test vector store behavior, document chunking logic, and retriever configurations.
• Validate fallback behaviors when tools or external services fail.
• Execute adversarial, prompt-injection, and red team style testing.
• Validate compliance with data privacy, business rules, and insurance guidelines.
• Benchmark LLM latency, throughput, and multi-agent performance.
• Validate concurrency handling, degradation logic, and retry mechanisms.
• Document test cases, behavior maps, prompt variations, and evaluation reports.
• Work closely with AI engineers and SMEs to refine agent workflows.
• Functional & Non-Functional Testing:

o Conduct API testing for AI services and model endpoints.

o Validate performance, scalability, and latency of AI inference services.

o Perform security testing for data pipelines and model-serving endpoints.
• Quality Governance & Compliance:

o Ensure compliance with AI ethics, data privacy, and regulatory standards.

o Establish KPIs for model quality (precision, recall, F1-score, drift detection).

o Document test strategies, results, and provide audit-ready evidence.

Collaboration & Leadership:

o Work closely with data scientists, ML engineers, and DevOps teams to integrate testing into the AI lifecycle.

o Mentor junior testers and contribute to building reusable test accelerators.

o Represent the testing function in client discussions, providing insights on AI quality assurance.
• Develop automated test frameworks for AI/ML pipelines.
• Implement synthetic data generation for edge-case testing.
• Conduct bias and fairness testing across diverse datasets.
• Validate monitoring dashboards for model drift and performance degradation.
• Perform regression testing for retrained models.
• Create reusable test assets for AI projects (scripts, datasets, frameworks).
• Provide detailed defect analysis and SLA-driven closure.