Curious learners
Using curiosity as an intermediate goal to speed up learning and achieve generalization in agents.
Watch a toddler for an hour. Nobody is paying them. They pull things off shelves, drop them, and watch what happens. Then they do it again. There is no score and nothing to win. And yet this is the fastest learning any of us will ever do. By four they have a working model of physics, of language, and of other people. Whatever is driving that, it is not reward maximization.
Our agents do not work like this. Drop one into an environment where reward is rare and it flails. Almost everything it does produces no feedback, so there is nothing to learn from. The usual fix is to shape the reward by hand until the agent stumbles into the behavior we wanted. That works. It also produces an agent that is good at exactly one thing, because the only pressure on it was that one number. The toddler is not good at one thing.
So the bet is to let the agent pay itself. Give it a reason to poke at the parts of the world it does not understand yet. If that works you get two things instead of one. You learn faster when the real reward is rare, and you end up with a model of the world that was never bent around a specific task. The second one is what I actually care about. A representation built by curiosity is not tuned to any reward function, which is exactly why it might survive contact with the next problem. That is the interesting claim, and it is the one worth testing. If there is another paradigm shift in machine learning, I would bet on something in this direction.