DUAW: Data-free Universal Adversarial Watermark against Stable Diffusion Customization

Authors: Xiaoyu Ye, Hao Huang, Jiaqi An, Yongtao Wang

What

This paper introduces DUAW, a data-free universal adversarial watermark designed to protect copyrighted images from being used for unauthorized customization of Stable Diffusion models.

Why

The paper addresses the growing concern of copyright infringement facilitated by AI art customization tools. It offers a practical solution to protect intellectual property in the rapidly evolving field of AI-generated content.

How

The authors develop DUAW by training it on synthetic images generated using a Large Language Model (LLM) and a pre-trained SD model. This data-free approach ensures confidentiality of the copyrighted images. The watermark disrupts the variational autoencoder (VAE) within SD models during customization, leading to distorted outputs when the customized model is used for generation.

Result

Experimental results demonstrate that DUAW effectively distorts images generated by customized SD models trained on watermarked images. This distortion is noticeable to human observers and detectable by a simple classifier, achieving high protection success rates. DUAW also exhibits strong transferability across different SD versions and VAE variants.

LF

The paper acknowledges the potential impact of image interference techniques on DUAW’s robustness, although its effectiveness remains high. Future work could focus on enhancing robustness against more sophisticated interference methods and exploring DUAW’s applicability to other diffusion-based models.

Abstract

Stable Diffusion (SD) customization approaches enable users to personalize SD model outputs, greatly enhancing the flexibility and diversity of AI art. However, they also allow individuals to plagiarize specific styles or subjects from copyrighted images, which raises significant concerns about potential copyright infringement. To address this issue, we propose an invisible data-free universal adversarial watermark (DUAW), aiming to protect a myriad of copyrighted images from different customization approaches across various versions of SD models. First, DUAW is designed to disrupt the variational autoencoder during SD customization. Second, DUAW operates in a data-free context, where it is trained on synthetic images produced by a Large Language Model (LLM) and a pretrained SD model. This approach circumvents the necessity of directly handling copyrighted images, thereby preserving their confidentiality. Once crafted, DUAW can be imperceptibly integrated into massive copyrighted images, serving as a protective measure by inducing significant distortions in the images generated by customized SD models. Experimental results demonstrate that DUAW can effectively distort the outputs of fine-tuned SD models, rendering them discernible to both human observers and a simple classifier.