Unmoderated by Design: How Hugging Face Enables NCII
aiforensics.org
By investigating Hugging Face's Spaces, a platform designed for sharing open-source AI demos, we found a hub for tools capable of generating non-consensual intimate imagery, as well as for comparing, benchmarking, and optimizing them, making image-based sexual abuse easier to scale. In fact, 73% of monitored requests were sexual in nature.

Our investigation found that Hugging Face, the top platform for sharing open-source AI models, isn’t just hosting tools that generate non-consensual intimate imagery. It’s helping users compare, benchmark and optimize them, making image-based sexual abuse easier to scale.

By investigating the platform’s most widely used Hugging Face Spaces and creating our own, we found:

  • 73% of monitored requests were sexual in nature.
  • Of those, 83% sought to undress or otherwise sexualize a person. 95% targeted women.
  • 6.7% targeted a minor. If fulfilled, these requests would constitute child sexual abuse material (CSAM).
  • Just 3% of audited Spaces showed any evidence of output moderation.

Hugging Face’s own policies prohibit non-consensual intimate imagery. Yet we found virtually no safeguards preventing these tools from being used for exactly that purpose.