Imagine something like that one episode in Ultimate Spiderman where they all had to move in with Peter because the helicarrier blew up. That could be a reason why they all had that sleepover at Matt's. And then it turned into them semi-permanently moving in.