In recent years a series of racist incidents have taken place in Sweden, and preparedness at the local level has not been good enough, according to Ullenhag.
"We have a big challenge in terms of xenophobia. We need to be able to handle it locally," he told Swedish Radio.
Cities and towns must be more pro-active, he said.
"Local preparedness should include everything from better accommodation facilities for newly-arrived refugees to dealing with students who receive racist comments at school. We need to take it all very seriously," Ullenhag said.