Researchers Pedro Curvo, Jan-Willem van de Meent, and others from the University of Amsterdam have published a paper titled ‘Follow the Mean: Reference-Guided Flow Matching’ (arXiv: 2605.10302), presenting a cost-effective new approach for controlling pre-trained generative models. The core insight of the paper is that under the framework of deterministic interpolants, the velocity field governing the generation trajectory is uniquely determined by the endpoint mean — simply shifting this mean during sampling instantly alters the entire trajectory. Users provide a set of reference images exhibiting desired attributes; the model computes their empirical mean to adjust the generation direction accordingly. This method requires no retraining, additional reward networks, or classifier guidance, and works effectively across color, style, identity features, and structural patterns.
Two specific implementations are proposed: first, training-free Reference Mean Guidance (RMG), which modifies a frozen model using a closed-form formula; second, a semi-parametric variant involving learning a residual refiner, allowing the reference set to be updated dynamically during inference. Demonstration cases cover style transfer, attribute editing, and anatomical structure correction, while the code has been released on GitHub. Co-author van de Meent described it as ‘so simple it’s almost embarrassing, yet far surpasses expectations.’
arXiv | HuggingFace | GitHub