Could It Be Generated? Towards Practical Analysis of Memorization in Text-To-Image Diffusion Models

Authors: Zhe Ma, Xuhong Zhang, Qingming Li, Tianyu Du, Wenzhi Chen, Zonghui Wang, Shouling Ji

What

This paper presents a practical method for analyzing memorization in text-to-image diffusion models, focusing on identifying and quantifying the extent to which specific images are memorized.

Why

This paper addresses the risk of copyright infringement and privacy violation posed by memorization in text-to-image diffusion models trained on massive datasets. It offers a practical tool for model developers to assess and mitigate these risks, contributing to responsible AI development.

How

The authors define three conditions for memorization: similarity, existence, and probability. They propose using the model’s prediction error as a measure of image replication (similarity). To find prompts that trigger memorization (existence), they develop a prompt inversion algorithm with regularization to ensure realistic token embeddings. Lastly, they measure the extent of memorization (probability) by comparing the prediction error distribution of the target image under the inverted prompt with that of a safe, unconditional diffusion model.

Result

The paper demonstrates that the model’s prediction error effectively identifies image replication. The proposed prompt inversion method can successfully trigger memorization for a significant portion of known memorized images. Moreover, the analysis reveals that unconditional diffusion models are generally safe from memorization, validating their use as a baseline for measuring memorization in conditional models.

LF

The authors acknowledge two limitations. First, their hard prompt inversion algorithm, although outperforming existing methods, is not entirely foolproof, especially for images requiring multiple key tokens. Second, the analysis focuses on text-to-image models, with further research needed for other conditional diffusion models. Future work could focus on improving hard prompt inversion and expanding the analysis to different types of conditional diffusion models.

Abstract

The past few years have witnessed substantial advancement in text-guided image generation powered by diffusion models. However, it was shown that text-to-image diffusion models are vulnerable to training image memorization, raising concerns on copyright infringement and privacy invasion. In this work, we perform practical analysis of memorization in text-to-image diffusion models. Targeting a set of images to protect, we conduct quantitive analysis on them without need to collect any prompts. Specifically, we first formally define the memorization of image and identify three necessary conditions of memorization, respectively similarity, existence and probability. We then reveal the correlation between the model’s prediction error and image replication. Based on the correlation, we propose to utilize inversion techniques to verify the safety of target images against memorization and measure the extent to which they are memorized. Model developers can utilize our analysis method to discover memorized images or reliably claim safety against memorization. Extensive experiments on the Stable Diffusion, a popular open-source text-to-image diffusion model, demonstrate the effectiveness of our analysis method.