Gen AI Security Researcher

ActiveFence is the leading tool stack for Trust & Safety teams, worldwide. By relying on ActiveFence’s end-to-end solution, Trust & Safety teams – of all sizes – can keep users safe from the widest spectrum of online harms, unwanted content, and malicious behavior, including child safety, disinformation, fraud, hate speech, terror, nudity, and more.

Using cutting-edge AI and a team of world-class subject-matter experts to continuously collect, analyze, and contextualize data, ActiveFence ensures that in an ever-changing world, customers are always two steps ahead of bad actors. As a result, Trust & Safety teams can be proactive and provide maximum protection to users across a multitude of abuse areas, in 70+ languages.

Backed by leading Silicon Valley investors such as CRV and Norwest, ActiveFence has raised $100M to date; employs 300 people worldwide, and has contributed to the online safety of billions of users across the globe.

As a Red Team Specialist focused on Generative AI Models, you will play a critical role in enhancing the security and integrity of our cutting-edge AI technologies. Your primary responsibility will be to conduct simulated cyber attacks and penetration testing on our generative AI systems, including but not limited to language models, image generation models, and any related infrastructure. The goal is to identify vulnerabilities, assess risks, and provide actionable insights to fortify our AI models against potential threats.

Key Responsibilities:

• Simulated Cyber Attacks: Conduct sophisticated and comprehensive simulated attacks on generative AI models and their operating environments to uncover vulnerabilities.
• Vulnerability Assessment: Evaluate the security posture of AI models and infrastructure, identifying weaknesses and potential threats.
• Risk Analysis: Perform thorough risk analysis to determine the impact of identified vulnerabilities and prioritize mitigation efforts.
• Mitigation Strategies: Collaborate with development and security teams to develop effective strategies to mitigate identified risks and enhance model resilience.
• Security Protocols Development: Aid in the creation and refinement of security protocols and best practices for the development and deployment of generative AI models.
• Research and Innovation: Stay abreast of the latest trends and developments in AI security, ethical hacking, and cyber threats. Apply innovative testing methodologies to ensure cutting-edge security practices.
• Documentation and Reporting: Maintain detailed documentation of all red team activities, findings, and recommendations. Prepare and present reports to senior management and relevant stakeholders.
• Training and Awareness: Provide guidance and training to technical and non-technical teams on security awareness and best practices related to generative AI.

Required Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Security, or a related field.
• At Least 5 years of experience in offensive cyber security, particularly in web application and API security, vulnerability assessment, and ethical hacking.
• Strong understanding of AI technologies and their underlying architectures, especially generative models like GPT, DALL-E, etc.
• Knowledge on AI vulnerabilities and possible mitigation strategies
• Familiarity with security frameworks, compliance standards, and ethical guidelines in AI.
• Proficiency in programming and scripting languages relevant to AI and cybersecurity (e.g., Python, JavaScript).
• Excellent analytical, problem-solving, and communication skills.
• Certifications in offensive cybersecurity (e.g., OSWA, OSWE, OSCE3, SEC542, SEC522) are highly desirable.

Preferred Skills:

• Experience with machine learning development frameworks and environments.
• Knowledge of data privacy regulations relevant to AI technologies, such as GDPR, CCPA, etc.
• Ability to work in a fast-paced, ever-changing environment.