The idea is that cleverness of intellect isn't anything mysterious. Humans do astounding feats just by applying relatively simple reasoning iteratively. Requiring artificial neural networks to do it all one-shot, from the top of the head is probably the reason why they require billions of parameters to show even a small bit of cleverness. Chain of thought is obvious solution. But in converting internal reasoning to output tokens some information is lost. Chain of thought in latent space is the natural next step. Thus recurrent networks.
I'm not familiar with flow matching, but I don't think it has any iterative processing in a sense of chain of thought or recurrence (despite arriving at the solution gradually).
It's iterative in a sense of solving differential equation iteratively. While recurrent networks are iterative in sense of putting a for loop around a bunch of if-s.
I wonder why they go with recurrent rather than something like latent flow-matching?
The idea is that cleverness of intellect isn't anything mysterious. Humans do astounding feats just by applying relatively simple reasoning iteratively. Requiring artificial neural networks to do it all one-shot, from the top of the head is probably the reason why they require billions of parameters to show even a small bit of cleverness. Chain of thought is obvious solution. But in converting internal reasoning to output tokens some information is lost. Chain of thought in latent space is the natural next step. Thus recurrent networks.
I'm not familiar with flow matching, but I don't think it has any iterative processing in a sense of chain of thought or recurrence (despite arriving at the solution gradually).
Flow matching is iterative in the sense that it predicts a dv(t)/dt at each step as it integrates toward x_0.
It's iterative in a sense of solving differential equation iteratively. While recurrent networks are iterative in sense of putting a for loop around a bunch of if-s.