- Dust is a new algorithm that applies random perturbations to each token's neural activation values and estimates gradients by measuring and averaging how much these perturbations reduce the loss for each token.
- Traditional evolutionary strategies require separate forward passes for each weight, but Dust independently applies perturbations at the token level and evaluates thousands of perturbations in a single forward pass, greatly improving computational efficiency.
- In experiments by the Q Labs research team, starting from learning over 100 million tokens, Dust improved efficiency by 1000 to 1 times compared to EGGROLL, a traditional evolutionary strategy in weight space, with this efficiency gain becoming more pronounced as the model size increased.
- Moreover, gradients estimated by evolutionary strategies up to a scale of 10 hundred million tokens closely matched backpropagation results, although currently the computational efficiency is insufficient to fully replace backpropagation.
The idea of creating virtual entities on a token-by-token basis to perform parallel evaluation, by inserting random noise in the activation stage and compensating for the loss changes per token to estimate the gradient, is impressive. However, it seems necessary to specifically investigate how these random transformations in the activation space affect loss calculations per layer and the stability of model training.