AI/ML Test Engineer

Ushur is transforming the way enterprises communicate and engage with customers. Fueled by consumer’s self-service demands, enterprises are modernizing customer engagement and experience models. Ushur is fast becoming the platform of choice for Customer Experience Automation™, enabling these enterprises to leapfrog their digital native counterparts and deliver delightful customer and employee experiences. With cutting-edge Conversational AI, Machine Learning and Intelligent Process Automation technologies, Ushur has enabled Fortune 100 enterprises including some of the world’s most well known brands in healthcare, insurance, banking and financial services sectors to automate their customer engagement. Cloud-native, 100% no-code and purely workflow-driven, Ushur empowers citizen developers within business operations teams to build AI-powered, fully-automated and omni-channel experience to digitally transform customer journeys end-to-end.

Role: Staff AI/ML Test Engineer

Exp: 3-7 Yrs

SKill: Mandatory : AI/ML Testing/Chatbot Testing with Python

Secondary: LLM

What You’ll Do:

Test Planning and Strategy:

• Develop and execute comprehensive plans and cases for AI/ML models.
• Define strategies that cover various aspects of model performance, including accuracy, robustness, scalability, and fairness.
• Ability to create cases for various use cases such as document extraction, classification and conversational use cases
• Good Python programming skills to create samples for each category (classification, extraction and conversational)
• Some familiarity with annotation/ground truth creation tools to help with case creation

Model Validation and Verification:

• Validate the correctness and performance of ML models using appropriate metrics.
• Perform rigorous to ensure models meet functional and non-functional requirements.

Automation and Scripting:

• Automate repetitive tasks and develop scripts.
• Maintain and improve automation frameworks.

Data Quality and Integrity:

• Ensure the quality and integrity of training and datasets.
• Verify data preprocessing and feature engineering pipelines.

Performance Testing:

• Conduct performance to assess the scalability and efficiency of AI systems.
• Monitor and analyze the runtime performance of models.

Documentation and Reporting:

• Document cases, results, and any identified issues.
• Generate reports and provide feedback to the development team.

Collaboration:

• Work closely with data scientists, developers, and other stakeholders to understand model requirements and design.
• Provide support during the deployment and maintenance phases.

Continuous Improvement:

• Stay updated with the latest AI/ML tools and techniques advancements.
• Continuously improve processes and methodologies.

The experience you need:

Educational Background:

• Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field.
• Advanced degrees or AI/ML or Data Science certifications are advantageous.

Technical Skills:

• Proficiency in programming languages such as Python
• Experience with Natural Language Processing (NLP) and Chatbot
• Understanding different ML algorithms and their applications (supervised, unsupervised, reinforcement learning).
• Knowledge of data processing and feature engineering techniques.
• Familiarity with software tools and methodologies (e.g., unit , integration , regression ).
• Experience with automated tools and frameworks.
• Understanding/exposure to LLMs, Prompt Engineering and Prompt Tuning

Machine Learning and AI Testing:

• Knowledge of model evaluation metrics and validation techniques (accuracy, precision, recall, F1-score, ROC-AUC).
• Ability to develop strategies for AI/ML models, including black-box and white-box approaches.
• Experience with A/B , cross-validation, and hyperparameter tuning.

Software Development and DevOps:

• Understanding of software development lifecycle (SDLC) and agile methodologies.
• Experience with version control systems (e.g., Git).
• Familiarity with CI/CD pipelines and tools like Jenkins, Docker, Kubernetes.

Analytical and Problem-Solving Skills:

• Strong analytical and problem-solving skills to identify and troubleshoot issues in ML models and AI systems.
• Ability to interpret and analyse complex data sets.

Soft Skills:

• Good communication skills to collaborate with data scientists, developers, and other stakeholders.
• Attention to detail and a proactive approach to identifying potential issues.