Foreign Minister Carl Bildt was in Brussels and took part in the meeting where foreign ministers from the political bloc levied travel bans and asset freezes against 21 people in Russia and Crimea.
"The EU does not recognise the illegal 'referendum' and its outcome," the EU Council wrote in a statement.
The Union also left the possibility of adding more measures open when government leaders meet again later in the week. The full list of names of those slapped with travel bans and asset freezes will be published on Monday evening.
"EU repeats call for Russia to immediately withdraw its forces in Ukraine back to pre-crisis numbers and garrisons," Bildt said on Twitter.
"We will never accept somebody tearing off pieces from countries by force. That is fundamentally unacceptable in every way and will never be accepted or recognised," the foreign minister added in a later interview.
The U.S. also announced a series of sanctions against Moscow in conjunction with the EU response. While the American list includes more prominent names than the European, neither includes president Vladimir Putin, despite his influence over the process.
"We've always had the policy of wanting to keep channels open. We had the exact discussion after the Georgian war, including foreign ministers, that we normally keep it out of this," Carl Bildt said to news agency TT.
According to the foreign minister, the sanctions should be seen as a "strong signal", rather than a punishment of responsible parties.