Matt does meet the Avengers. But he's not happy with any of them and is still holding a grudge since none of them stuck around to help clean up after Hell's Kitchen got trashed by the Chitauri.

+ if additional damage was caused by Ultron, sending Matt's blood pressure through the roof
++ if Matt comes down on the Avengers for abandoning people in the clean-up then Foggy and Karen stand behind him nodding
+++ "Kick their asses baby, I got your glasses."