VENUS: Decoupling the Samples in MARS Benefits Small-Batch Training

AdamW has long been a standard optimizer for neural network training, with recent alternatives including matrix-aware methods such as Muon and variance-reduced methods such as MARS, which scales the variance-reduction correction term. We show that the unclipped, estimator-level formulation of MARS is exactly a convex combination of momentum SGD (MSGD) and momentum variance reduction (MVR). When both branches use the same minibatch, the three mixing parameters reduce to two effective MARS parameters; thus, varying these parameters independently does not expand the estimator family. Independent sampling, however, does. We introduce VENUS, which evaluates the two branches on independent minibatches and introduces one additional effective degree of freedom. We establish convergence guarantees for updates defined by a linear minimization oracle (LMO), where the additional mixing weight enters through an explicit variance factor minimized by balanced weighting. Experiments on GPT-2-style language model pretraining show that, under matched gradient budgets, exact VENUS achieves lower final validation loss than exact MARS in a small-batch regime.