A fight over fish stocks between the European Union and neighboring islands will come to a head this week, when Brussels decides whether to impose trade sanctions on the Faroe Islands and start a similar process for penalizing Iceland.

On Wednesday, EU member states will vote on whether to ban imports of herring and mackerel from the Faroe Islands, after the tiny archipelago nation unilaterally permitted its fishermen to dramatically increase their annual catch this year. By the end of July, the European Commission, the EU’s executive arm, will also decide whether to take the first step toward similar sanctions on Iceland’s mackerel exports.

The actions would represent the first time that the EU has imposed sanctions on a nation outside the bloc that doesn’t comply with international fishing quotas.

Sanctions could deal a blow to the economy of the Faroe Islands, a nation of less than 50,000 people that is formally part of Denmark but has autonomy over domestic policies and isn’t part of the EU. A ban on herring exports would also affect Faroese mackerel exportssince the two are often caught in the same netsand prevent Faroese herring trawlers from unloading in EU ports.

The EU claims that the rate at which Iceland and the Faroe Islands are taking herring and mackerel out of the sea will cause a collapse in stocks, just as the 28-country bloc is overhauling its fisheries policy in the hope of reviving regional fish populations.

Yet imposing trade sanctions on the Faroe Islands and Iceland would have diplomatic costs for the EU. Relations with Iceland are already at a low, with the country’s accession bid to join the bloc stalled after EU-skeptic parties won national elections this spring. And the Faroe Islands are officially part of Denmark, so sanctions would lead to the EU effectively penalizing one of its own.

But the commissionbacked by the U.K., Ireland, France and Spainsays sanctions are necessary after the Faroese government more than tripled the country’s herring quota this year. The Faroese government is allowing its fishermen to catch 105,230 metric tons of herring in 2013, up from 31,000 tons in 2012. This means the small nation would be responsible for 17% of all the herring fished out of the northeastern region of the Atlantic Ocean, though international agreements only allow them 5%, according to the European Commission.

The Faroe Islands “are out of control,” said Stewart Harper, spokesman for the National Federation of Fishermen’s Organizations in the U.K. At a time when European fishermen are reducing their catch, the quotas of the Faroe Islands are “irresponsible,” he said.

Denmark formally opposes the sanctions, which would force it to boycott products from a part of its own country. “We favor a negotiated solution,” the Danish minister of fisheries, Mette Gjerskov, has said.

Denmark, which handles the foreign policy of the Faroe Islands but has no power over its domestic policies, fears that a ban could hurt the islands’ economy. “It is a small community that is highly dependent on its fishing activities,” Ms. Gjerskov said.

Fish products account for 95% of Faroese exports and more than half of its gross domestic product. Around a quarter of all herring caught by Faroese fishermen, as well as half of all mackerel, is currently exported to the EU, according to the Faroese delegation to the EU.

If the ban is approved, the Faroe Islands would sell herring and mackerel currently exported to the EU to other world markets, said the islands’ fisheries minister, Jacob Vestergaard.

Others question whether the impact of a ban can be overcome so easily. The sanctions would also prevent exports of fishing vessels, fishing gear and supplies from the EU to the Faroe Islands.

No longer being able to unload in traditional EU ports would pose logistical problems for Faroese trawlers, said Kate Sanderson, head of the Faroe Islands’ mission to the EU.

The Faroe Islands dispute the EU’s claim that its quotas are unsustainable, saying mackerel and herring stocks have moved in larger numbers into Faroese waters.

“The old herring and mackerel quotas did not at all reflect the biological reality in northern waters and were totally unreasonable and unfair,” said Mr. Vestergaard.

Under United Nations regulations, the Faroe Islands have to negotiate the size of their herring and mackerel catches with the EU, Iceland, Russia and Norway. This is the first year the Faroe Islands have ignored international law for herring quotas, but both the Faroe Islands and Iceland have unilaterally set quotas on mackerel since 2009, upsetting other European fishing nations such as the U.K., Ireland, France and Spain.

These so-called mackerel wars prompted the EU to create a law last year that allows it to impose trade sanctions on neighboring nations that set quotas with which it doesn’t agree. Sanctions on Faroese herring would represent the first use of this law.

Denmark and others hope sanctions can be avoided. “We are working hard to bring the parties back to the negotiating table so that we can find a solution that everyone can be happy with,” said Ms. Gjerskov.

2013 Dow Jones & Company, Inc.