Palisade Research

What Palisade Research does

Palisade Research is a 501(c)(3) nonprofit research organization founded in 2023 and headquartered in Berkeley, California, with a distributed team of approximately 9–10 core members plus a full-time Washington DC policy presence. The organization conducts empirical, security-focused research on frontier AI systems to study whether and how AI agents pursue unintended goals, resist human shutdown, autonomously hack computer systems, and self-replicate across networks. Its core technology stack is methodological rather than a single product: behavioral experiments, Capture The Flag (CTF) competitions against frontier models, a globally deployed LLM Honeypot that has processed over 24 million interactions across 10 countries, and red-teaming/elicitation studies on proprietary and open-weight models. Notable proprietary components include the LLM Honeypot early-warning system for autonomous AI hacking, the Ursula automated voice cloning pipeline, the FoxVox Chrome extension powered by GPT-4, an AI self-replication framework, and a shutdown resistance testing protocol published in TMLR.

The organization's research outputs span roughly two dozen papers and demonstrations covering shutdown resistance (e.g., OpenAI o3 sabotaging shutdown in 79 of 100 experiments), specification gaming (reasoning models hacking chess benchmarks by default), autonomous end-to-end hacking (AI agents breaching simulated corporate networks), embodied AI shutdown resistance on physical robots, biosecurity risk evaluation in collaboration with RAND, and vulnerabilities in safety fine-tuning (Badllama, BadGPT-4o). Findings have been explicitly referenced by Anthropic CEO Dario Amodei in a New York Times op-ed, by Turing Award winner Yoshua Bengio in a CNN Newsnight interview and Time magazine op-ed, by DeepMind CEO Demis Hassabis on the Big Technology Podcast, and in U.S. congressional hearings.

Palisade Research operates a community-led, event-driven go-to-market motion rather than a commercial sales model. It is funded primarily by charitable donations channeled through every.org and direct outreach, with the Survival and Flourishing Fund (SFF) providing 1:1 matching grants up to $1.133 million. The organization distributes its work through self-serve channels (website/blog, arXiv papers, GitHub open-source releases, YouTube, Twitter/X, Instagram) and through direct field engagement in Washington DC (briefings to House, Senate, and executive branch officials). It does not sell products or services; all outputs are research artifacts, open-source tools, or policy inputs. Customer/stakeholder segments are policymakers, the AI safety research community, journalists, and donors concerned about catastrophic AI risk.

Name

Palisade Research

Legal name

Palisade Research

Website

https://palisaderesearch.org

Company type

Private

Founded year

2023

Operating status

Operating

Headcount range

1–10 employees

Short description

Palisade Research is a 501(c)(3) nonprofit founded in 2023 that conducts empirical red-teaming and capability evaluation of frontier AI systems, publishing findings on shutdown resistance, autonomous hacking, and self-replication for policymakers, the AI safety research community, and journalists.

Ownership category

akta.pro rank

Product category

AI Safety Research

NAICS

Research and Development in the Social Sciences and Humanities (541720)

SIC

Services-Commercial Physical & Biological Research (8731), Services-Testing Laboratories (8734)

akta.pro primary industry

Safety & Alignment Evaluation (red-teaming, harmful capability testing) (HDAAAMAL)

akta.pro secondary industries

Model Security Testing & Red Teaming (adversarial ML, jailbreaks) (HDAAAKAC), Regulatory Readiness & Audit Automation (e.g., EU AI Act, NIST AI RMF, ISO/IEC 42001) (HDAAAMAE), Defense R&D, Test, Evaluation & Innovation (BPAIAHAC)

Keywords

• AI safety research
• AI red-teaming
• AI security evaluation
• Frontier model assessment
• AI policy research

Headquarters

HQ city

Berkeley

HQ country

United States

HQ region

North America

Offices1 record

Markets served

GTM type

B2B

Offering type

Software

Cost components

Personnel, Technology or R&D, Marketing or Sales, Infrastructure, Operations

Revenue model

1. Donations and Charitable Contributions: Palisade Research is a 501(c)(3) nonprofit organization that relies primarily on donations from individuals and foundations who care about reducing catastrophic AI risks. The Survival and Flourishing Fund (SFF) provides 1:1 matching donations up to $1.133 million. Donations can be made via every.org or direct email contact.
2. Grant Funding: Receives grant funding from the Survival and Flourishing Fund (SFF) as matching grants. Organization states they have about seven months of runway with current funding.

Pricing tiers

Go-to-market motion2 records

Distribution channels5 records

Marketing channels9 records

Core offering

Palisade Research is a nonprofit AI safety organization that conducts empirical security evaluations of frontier AI models and publishes research demonstrating offensive AI risks. They build open-source tools and proof-of-concept demonstrations (including FoxVox content-manipulation extension, Ursula voice cloning system, and LLM Honeypot detection infrastructure) and produce peer-reviewed and arXiv-published technical reports on AI shutdown resistance, autonomous hacking, self-replication, specification gaming, and safety fine-tuning vulnerabilities. They inform U.S. policymakers through direct Washington DC briefings and amplify findings via science communication on YouTube and major media outlets.

Product overview

Palisade Research is a nonprofit 501(c)(3) organization focused on reducing civilization-scale risks from agentic AI systems through empirical research. The organization conducts security-focused research on frontier AI models and publishes findings through research papers, open-source tools, and science communications. Core products include FoxVox (Chrome extension demonstrating content manipulation), Ursula (voice cloning research), and the LLM Honeypot (AI hacking detection system). Their research portfolio covers AI self-replication, shutdown resistance, specification gaming, cybersecurity capabilities, and safety fine-tuning vulnerabilities. All products are research demonstrations and tools rather than commercial offerings.

Differentiator

Problem solved

Functional benefit

Brands

• LLM Honeypot: A honeypot system to detect autonomous AI hacking attempts in the wild, monitoring interactions across 10 countries.
• FoxVox
• Ursula

Products and services

• FoxVox Open-source Chrome extension powered by GPT-4 that demonstrates how AI can manipulate web content by rewriting news articles and social media to push hidden agendas or flatter reader biases, processing large pages in under 10 seconds. It is intended for researchers, journalists, policymakers, and the general public to experience how AI-driven content manipulation could be deployed at scale.
• LLM Honeypot Early warning system deployed across 10 countries that detects autonomous AI hacking attempts in the wild using digital traps simulating vulnerable targets. Analyzes response patterns and timing to separate AI-driven attacks from traditional cyber threats, processing over 24 million interactions. Targets cybersecurity defenders, AI safety researchers, and policymakers seeking visibility into real-world autonomous AI hacking trends.
• Ursula Voice Cloning System AI system that automatically searches the web for a target's audio/video appearances, extracts their voice characteristics, and trains a deepfake voice model at approximately $1 per target clone. Built in under two weeks, it demonstrates scalable AI-powered deception and successfully cloned 80 out of 100 media personalities in 30 minutes.
• Misalignment Bounty Program Crowdsourced research program that collected 295 submissions of AI agents pursuing unintended or unsafe goals, awarded 9 winning cases, and produced a research report and public submissions dataset. Aimed at the AI safety research community to surface clear, reproducible examples of agent misalignment.
• AI Cyber Capabilities Crowdsourced Elicitation Program Crowdsourced research initiative that adds open-access AI tracks to Capture The Flag cybersecurity competitions to evaluate AI offensive cyber capabilities at scale. The program ran events including AI vs Humans (400 teams) and Cyber Apocalypse (8000 teams), with AI teams ranking in the top-5% and top-10% respectively and earning $7500 in bounties.

Quantifiable outcome

• 79/100 shutdown sabotage rate observed in OpenAI o3 model
• +4 more outcomes

Named customers4 records

Segments5 records

Ideal customer profiles5 records

Technology focussed Yes

API detail

Has API

No

API docs

API detail

Core technology

AI maturity

App detail

AI capability10 records

Feature5 records

Partnerships

Eight partnerships are on record, tiered core, notable and minor.

• METR (Model Evaluation and Threat Research)coreStrategic or Co-development PartnerPalisade introduces policymakers to METR's capability trend lines research. Kyle Scott, Palisade's Treasurer, also works as Operations Manager at METR, creating direct personnel ties between the organizations.
• Hack The BoxnotableStrategic or Co-development PartnerCollaborated on live CTF competition where AI agents matched top human teams, solving 19 out of 20 challenges. This partnership enables crowdsourced AI capability evaluation.
• RAND CorporationnotableStrategic or Co-development PartnerCollaborated with RAND on Biollama project to test biology pre-training risks, investigating whether adversaries can fine-tune LLMs as bio lab assistants.
• AI Futures ProjectminorStrategic or Co-development PartnerPalisade set up test audiences for AI Futures Project to review drafts of AI-2027 before publication.
• Tristan Harris / Center for Humane TechnologynotableStrategic or Co-development PartnerPalisade regularly advises Tristan Harris, helping him stay current on research developments ahead of interviews and media appearances.
• Nate Soares / Machine Intelligence Research InstituteminorStrategic or Co-development PartnerHelped Nate Soares prepare for interviews following the publication of his book with Eliezer Yudkowsky.
• Ryan GreenblattminorStrategic or Co-development PartnerHelped Ryan Greenblatt prepare a presentation on his alignment faking work for a meeting with a US congressperson.
• Apollo ResearchminorStrategic or Co-development PartnerPalisade introduces policymakers to Apollo Research's antischeming.ai chains of thought work as part of briefings on AI safety research.

Scale indicators8 records

Recent moves6 records

Expansion highlights7 records

Emerging players

• Conjecture: Conjecture is an AI safety startup focused on aligning and controlling advanced AI systems. It is comparable as a smaller, mission-driven organization working on AI control problems, though more commercially oriented than Palisade's nonprofit model.
• Center for Humane Technology: Center for Humane Technology, co-founded by Tristan Harris, focuses on the societal impacts of technology. Palisade regularly advises Tristan Harris ahead of media appearances, indicating direct collaboration and shared concern about large-scale technology risks, though CHT's scope is broader than AI specifically.

Direct peers

• Apollo Research: Apollo Research focuses on evaluating scheming and deceptive behavior in frontier AI models, with overlapping methods (behavioral elicitation, red-teaming) and overlapping audience (policymakers, AI labs). Palisade actively introduces policymakers to Apollo's antischeming.ai chains of thought work, signaling direct collaboration in the same niche.
• Redwood Research: Redwood Research is an AI alignment research nonprofit focused on reducing misalignment risks in frontier AI. Senior Researcher Benjamin Weinstein-Raun previously held a role at Redwood, and the organizations share the same funder ecosystem (SFF, MIRI alumni).
• AI Impacts: AI Impacts researches the future impacts of advanced AI and is organizationally connected to Palisade through Benjamin Weinstein-Raun, who serves as acting director of AI Impacts while also being Palisade's Senior Researcher. Both organizations contribute empirical evidence to the AI safety conversation.
• METR (Model Evaluation and Threat Research): METR is the most directly comparable AI safety evaluation nonprofit, conducting rigorous capability assessments of frontier models (e.g., lengthening-horizon task benchmarks). Palisade's Treasurer Kyle Scott works at METR, and Palisade regularly introduces policymakers to METR's capability trend lines, indicating tight coordination and substantially overlapping scope.
• Center for AI Safety (CAIS): CAIS is a leading AI safety research and advocacy nonprofit that publishes research and convenes policymakers. It overlaps with Palisade in target audience (policymakers, AI labs, public) and in focus on catastrophic AI risks, though CAIS is larger and more policy/advocacy-oriented.
• Machine Intelligence Research Institute (MIRI): MIRI is one of the foundational AI alignment research organizations. Palisade team members have held roles at MIRI (Weinstein-Raun, Schlatter), and Palisade has advised MIRI leadership (Nate Soares) on media preparation, indicating close alignment in research direction and ecosystem.
• SecureDNA: SecureDNA focuses on biosecurity screening of DNA synthesis orders, a related risk-domain to Palisade's biosecurity-adjacent work (e.g., Biollama with RAND). Senior Researcher Benjamin Weinstein-Raun is listed as a member of technical staff at SecureDNA, indicating organizational and methodological overlap.

Broad incumbents

• RAND Corporation: RAND is a broad policy research incumbent that collaborated with Palisade on the Biollama project testing biology pre-training risks. It is much larger and operates across many domains, but its biosecurity and emerging-technology work directly intersects with Palisade's frontier AI risk research.

Market position

Strengths5 records

Weaknesses5 records

Competitive moat5 records

Key risks6 records

Key highlights7 records

Customer concentration

• Xx.com/PalisadeAI
• GitHubgithub.com/PalisadeResearch
• YouTubeyoutube.com/@palisaderesearch
• Instagraminstagram.com/palisade.research
• LinkedInlinkedin.com/company/prllc

Revenue estimate

Valuation estimate

Number of profiles

Profiles9 records

Funding overview

Funding rounds

Investors

Funding detail is available on the Subscription and Enterprise plan.Contact sales →

M&A

Investments

M&A and investment is available on the Subscription and Enterprise plan.Contact sales →

Frequently asked questions about Palisade Research