Speaking during an on-the-record interview Wednesday, Rasmussen said, "I don't think it's arguable it's more difficult when the environment is contaminated by mutual suspicion."
"If you're increasing the amount of suspicion and distress on these communities, it places more challenges in our way," he said.
Asked "does the environment today make your job more difficult?" -- with reference to comments by the President and administration policies such as the travel ban -- Rasmussen answered "yes."
Trump's past and recent tweets and statements concerning Muslims and the travel ban played a central role during a federal appeals court hearing last week on the legality of travel ban 3.0.
Lawyers and judges discussed whether the tweets should be taken into consideration as the court examines whether the ban on travel to the US by residents of certain nations was illegally motivated by anti-Muslim animus. Special attention was paid to the President's November 29 retweets of three inflammatory videos from a British far right account rife with anti-Muslim content.
"What do we do," Judge James A. Wynn asked, when we have "multiple instances" when the President has tweeted about Muslims "before the election, during the election and just a week or so ago?"
"Do we just ignore reality?" he asked.
The White House has previously defended several of Trump's controversial comments.
Last month, White House press secretary Sarah Sanders claimed that Trump's decision to retweet anti-Muslim content from a leader of a far-right British political group served to "elevate the conversation."
Trump had sparked criticism from Britain's highest officials when he retweeted the three messages from Jayda Fransen, the deputy leader of the group, Britain First.
Trump's support for the messages, which contained videos depicting purported Muslims assaulting people and smashing a statue of the Virgin Mary, drew condemnation in the UK, including from Prime Minister Theresa May.