Frank is gay and the only person he ever told was Maria (they weren't married for romance but because they loved each other and wanted to raise kids together). Frank has tried everything to rid himself of his perversion, but nothing worked. It stopped mattering for a while after his family died but eventually (mostly thanks to Daredevil) he starts having his disgusting thoughts again. His self hatred is enough to distract his punishing, and cause Matt to be concerned