How Turkey and the US fell out – and why Russia will benefit

Relations between Turkey and the United States hit a new low on Sunday 8 October. The US State Department suspended "all non-immigrant visa services at all US diplomatic facilities in Turkey".

The Turkish government retaliated – citing concerns about the "commitment of the government of the United States to the security of the Turkish Mission facilities and personnel" – by "suspend[ing] all non-immigrant visa service[s] at all Turkish diplomatic facilities in the US."

The accession to power of now-Turkish President Recep Tayyip Erdogan and his Justice and Development Party (AK Party) in 2003 was met with some caution after decades of official secularism in Turkey. There were implicit tensions between the West and an insular, Islamist-oriented governing party, though both the Islamism of the AK Party and the personality of Erdogan have been overplayed in explaining Turkey.

This is convenient for the West, helping obscure the extent to which its own mishandling of Turkey set in place the confrontational dynamic that now risks boiling over. There are many things that have led to this point, but it is worth dwelling on three.

First, the EU's signal failure to come to terms with Turkey. The EU opened serious accession talks with Turkey in 2005; that process is clearly dead and has been for some time. The EU was engaging a Turkish government earnestly – and self-interestedly – invested in a process of reform to undo the consequences of the gruesome military junta that seized power in 1980 and never truly left the scene, emplacing a dauntingly authoritarian constitution on the country and intervening again in 1997 to dismiss a civilian government that defied its instructions.

Instead of taking advantage of this situation, the EU strung out the Turks and foreclosed the possibility of EU membership long before Germany's declaration last month that it favoured a formal end to the accession talks. The Eurocrats congratulate themselves on a righteous sanction against a country "descending into authoritarianism", but the truth is that Brussels' conduct propelled, rather than reacted to, the dark turn in Ankara.

Second, Syria. In the years before the rebellion in Syria, Erdogan had cultivated good relations with Bashar al-Assad as part of Turkey's good-neighbours policy. Into the summer of 2011, Turkey tried to retain this course. Only once Turkey found that Assad was unengageable – that he insisted on massacring unarmed demonstrators and using rape as a tactic of war, pushing tens of thousands of refugees into Turkey, among other things – did Ankara make the decision to topple Assad's criminal regime.

A year later, Assad brought down a Turkish jet in international waters. Rather than a collective Nato response, Turkey was told she was on her own. In 2014, the US finally did intervene in Syria – against the Islamic State (IS), having told Assad in advance through Iran that his regime was off-limits. The Turks' rebel allies inside Syria were, by definition, anti-Assad, and as such not what the US was looking for.

The US instead deputised the Kurdistan Workers' Party (PKK), which retains cordial relations with (and a considerable dependency upon) the pro-Assad coalition, as its anti-IS ground force – directly supporting and empowering a movement that has been and continues to be the premier internal security threat to the Turkish government.

Turkey took her own counter-terrorism action in Syria, not that she gained any credit for it from the West at a popular or governmental level, and after shooting down a Russian warplane in late 2015, the Turks were likewise left in the lurch, thereafter ensnared in the Astana process that seeks to settle Syria politically on terms dictated by Moscow-enabled military operations.

Third, and most immediately, Turkey's reaction to the violent attempted coup on 15 July 2016, in which 249 people were killed. Despite the US having clearly, immediately rejected the attempted putsch, many Turks feel the US was neither swift nor sincere enough – that Washington waited until it saw which way things were going.

Then has come the contest over Fethullah Gulen, the Pennsylvania-based Islamist cleric and former ally of AK Party, whom the Turkish government says was behind the coup. While there seems little doubt Gulenists were involved in the attempted coup, it remains mysterious exactly where the orders were coming from.

Turkey has maintained its position, however, and demanded Gulen's extradition. The US has not complied. Explanations to the Turks that in the US this is a legal process, not a political one the government can influence, have not cut much ice. On the other side, Ankara's use of the "Fethullah Terrorist Organization" (FETO) to cast a wide net in its post-coup purge of the bureaucracy has increased Western suspicion about the Gulenist explanation. And it is here that the proximate cause for the latest crisis in US-Turkey relations has its origin.

In the dragnet following the coup attempt, Turkey has picked up a dozen Americans on various coup and terrorism-related charges. The case that has taken on the most prominence is Andrew Brunson, an American pastor who has lived in Turkey since 1993. Brunson was rounded up on 7 October 2016 and charged with membership in a terrorist organization.

In February, seventy-eight members of Congress wrote an open letter to President Erdogan, asking for Brunson's release, noting the lack of evidence against him, the denial of regular counsel, and warning, too, of the "significant strain" this detention was putting on the "robust bilateral relationship" between the US and Turkey. In August, Brunson was hit with four further charges, including espionage and seeking the overthrow of the government.

This situation was bad enough, but soon Erdogan – as is his knack – made it considerably worse. In a speech on 28 September, Erdogan said: "[T]hey say, 'give us the pastor [Brunson]'. You have another pastor in your hands [Gulen]. Give us that pastor and we will do what we can in the judiciary to give you this one."

The unmistakeable scent of hostage-blackmail attended this proposition, an intolerable violation of international norms one expects from the outlaw government in Iran, not a Nato member. Then came the final straw: the arrest, on 4 October, of Metin Topuz, a staffer at the US Consulate in Istanbul.

The Turkish government insists that Topuz has been in regular contact with numerous "FETO" operatives, including some of the leading members of the coup conspiracy, and had "acted as a liaison" to Gulen himself. As such, Topuz has broken the laws against terrorism.

But there are indications that Ankara's pursuit of Topuz is related to his role in securing the arrest of Iranian-Turkish gold trader and sanctions-buster Reza Zarrab, which in turn led to an indictment being issued in the US for Zafer Caglayan, a former Turkish economy minister and a close Erdogan ally. It was likely to this that the US Ambassador in Turkey referred when denouncing the arrest of Topuz as an act of "vengeance", rather than justice.

Anti-Westernism has been increasingly emphasised as an instrument of domestic government policy in Turkey, with the West identified as colluding with the two primary state foes: FETO and PKK.

This thread in Turkish discourse has formed a noticeable part of the campaign against NGOs dealing with Syria and the arrest of foreign human rights organisation representatives, and appeared in quite spectacular fashion during the referendum in April to transform Turkey's presidency into an executive office, a vote that, whether or not it was free on the day, was by no means fair.

Unfortunately there does not seem to be a way out of this. Erdogan will escalate; he knows no other way. (Indeed in the last hours an arrest warrant has been issued for another US Consulate official.) More fundamentally, while there have been misunderstandings, the bedrock sources of conflict between the two sides are very real.

The NGO crackdown, for example, is designed in part to thwart the provision of aid to the PKK statelet in Syria that Turkey sees a mortal threat to its territorial integrity and the US sees as a counterterrorism partner. There is no way to finesse this issue. The US went with the PKK into Raqqa, ignoring the pleas of Turkey to work with her to accomplish a durable settlement in the aftermath of the caliphate.

Instead, it has left Ankara with every interest in undermining "Rojava" and having plenty of local resentment to work with. Ankara can (correctly) claim to have been made a prisoner of the Russian-directed Astana process it now enables in no small part because the US helped limit her options in Syria, yet this does not change the divergence of the trajectories Turkey and the US are now on in Syria. Nor does the shabby treatment of Turkey by her allies explain everything.

Turkey had a large enough role in limiting her own options in Syria – choosing some deeply problematic Islamist allies, most obviously Ahrar al-Sham, which paved the way for the co-optation of the northern insurgency by al-Qaeda-linked jihadists, a situation Turkey is having to deal with directly – and the taking of hostages and purchasing missile systems from the Russians are not within the range of acceptable Nato behaviour no matter the conditions. Turkey cannot reset relations with the US by responding in kind to the trespasses against her.

This situation is tragic for both sides. Turkey sits on one of the most strategic strips of territory on earth and from the Middle East to Central Asia there is much Turkey can do to aid Western interests. Turkey came into Nato in 1952 to protect itself from the Russians. In turn, Turkey guarded Europe's southern flank – and even sent troops to repel Moscow-orchestrated aggression on the Korean Peninsula.

The menace in the East is back on the agenda, and it is Russia that gains most from Turkey's drift out of the Western orbit.