"They don't want me in the White House," Harmon said.
But Harmon said he wouldn't be opposed to visiting President Obama.
"That would be dope," he said about such a meeting.
"Obama, man, come holler at me man," he said.
"We love you over here."
The Patriots visited the White House in 2017 after their previous Super Bowl win, and a few players, including quarterback Tom Brady, did not attend.
Other professional athletes have turned down similar invitations from President Trump, citing political disagreements. The NBA's Golden State Warriors opted out altogether of a White House meeting in January and instead visited President in Obama following their national championship victory.
Harmon and the rest of the Patriots defeated the Los Angeles Rams by a score of 13-3.
Watch the full interview here.