Diffusion-Based User Encoder Beats LLMs

Researchers have unveiled FINDERS, a diffusion‑based framework that redefines how user data is encoded, surpassing traditional LLM‑based models across diverse applications.

Core Innovations of FINDERS

  • Leverages iterative denoising of diffusion language models to capture multi‑granularity user behavior.
  • Introduces a router‑equipped architecture with denoise, factorize, route, and compress stages.
  • Employs learnable meta‑slots to separate cross‑modal semantics from modality‑specific factors.
  • Uses BehaviorQA, a new corpus with explicit behavioral rationales, for grounded supervision.
  • Dynamic routing prioritizes scenario‑relevant slots via an input‑level residual pathway.

From Static Encoders to Adaptive Diffusion

Traditional LLM encoders serialize heterogeneous logs into static vectors, limiting bidirectional flow and cross‑modal interaction. FINDERS unlocks the denoising process of diffusion models, enabling iterative reconstruction that preserves richer signals. The method builds on recent advances in diffusion language modeling, extending their use beyond text generation to user representation learning.

Implications for Industry and Research

With consistent gains in ten real‑world scenarios, FINDERS demonstrates that diffusion models can serve as powerful, scenario‑adaptive user encoders. This opens avenues for more nuanced recommendation systems, fraud detection, and personalized content delivery. Future work may integrate larger multimodal datasets and explore real‑time inference optimizations.

Leave a Reply

Your email address will not be published. Required fields are marked *