The thing about Matt deciding to kill Fisk because of Mrs Cardenas' death is that it was definitely a "bad day", and it wasn't one that he really came back from through his own will. The only thing that stopped him carrying out the murder was a) finding Nobu instead of Fisk and b) being too beat-up to do anything when Fisk did appear. By the time the final showdown happened, it's arguable that Matt had the chance to kill Fisk and decided not to, but he wasn't quite driven by his emotions/desire for revenge as much as wanting justice for Fisk's crimes - not to mention, there wouldn't have been much point killing Fisk, with cops on the way and court proceedings on the horizon.

So that's why Frank's line hit hard for me. We know Matt is an emotional guy with a temper, and not many qualms about hurting bad people. Maybe by that point in the series, Matt will have made a far more solid decision to never kill, but it'll always be true that a "bad day" once did nearly bring him to cross that line. And who knows what the next "bad day" will do, if it's bad enough? Matt's driven by a sense of justice and faith in the law; what happens if he is ever in a dark enough place that he can't believe in those things any more?

As viewers we know/hope nothing will ever bring Matt to that point like it did with Frank, but the characters themselves don't 'know' that. That's one reason why it's so interesting to watch.