Abstract
We propose
Prime-EVC, a fast and expressive emotional voice conversion (EVC) framework built on conditional flow matching (CFM) that converts emotion in a single ordinary-differential-equation step. Our key idea is to reallocate the emotional conversion between the conditioning and the sampler: a FiLM-fused speaker-emotion style, an emotional variance adaptor, and direct emotion and spectro-temporal supervision assemble an emotion-rich conditioning upstream, rendering the residual conversion left to the CFM decoder attainable within one Euler step. Conditioned on this representation and the fused style, a Diffusion-Transformer CFM decoder recovers the target mel, vocoded by BigVGAN. Experiments on the ESD corpus demonstrate that Prime-EVC attains the best conversion quality (UTMOS 3.37, EECS 0.87) at RTF 0.038, outperforming the strong diffusion-based DurFlex-EVC at both its 4- and 100-step settings while running over 3.5× faster. Below we compare Prime-EVC (one step) against the
DurFlex-EVC (100 steps),
Emovox,
Seq2seq-EVC,
StarGAN-EVC, and
Textless-EVC baselines, with the parallel-text target recording as reference.