It was Peter who suggested it, but Matt didn't shoot down the idea, so that's why he feels like it's his fault. Between the two of them, Matt feels like pretty much everything is his fault, since he's older/more experienced/taken more punishments.