He could accidentally wind up too close to a time portal some scientist or witch opens up and keeps portal-hopping to get home? And the spell or device is using Matt as an anchor, which is why it keeps dumping Foggy out near meaningful events in Matt's life?