Why we all get burnt in the bonfire of the bond markets

When Bill Clinton was in the White House, such was the power of the mighty bond markets in taming his ambitious spending plans that his adviser James Carville notoriously quipped: "I used to think if there was reincarnation, I wanted to come back as the president or the pope or a .400 baseball hitter. But now I want to come back as the bond market. You can intimidate everybody."

Silvio Berlusconi, the erstwhile Teflon Don, who bowed out last week after a battering from the bond markets, would probably agree. Over the past 18 months, as one eurozone leader after another has been toppled and finance ministers have announced drastic spending cuts and public sector sackings, the shadowy figure of the bond trader has been landed with much of the blame.

Suddenly, we're all transfixed by daily movements in "yields" – in effect, the interest rate governments pay to borrow from financial markets. Reassuring the bond investors has become the overriding aim of politicians from Paris to Athens. When Italian yields surged through 7%, it was the last straw for the once-unassailable Berlusconi, demonstrating the extraordinary power of the buyers and sellers in sovereign debt markets.

Governments that want to borrow from the financial markets hold auctions of bonds – IOUs that carry a fixed interest rate over a certain period, typically 10 years. The actual return on the investment – the "yield" – will depend on how much investors are willing to pay for each batch of bonds, relative to the face value of the bond.

But there's a healthy so-called "secondary market" in bonds, as investors bet on their future value, and adjust their portfolios by buying and selling the debt they hold. It's in this secondary market that the Bank of England's recession-busting quantitative easing takes place: it offers to purchase bonds – to the tune of £275bn once its current buying spree is over – from investors such as pension funds in the hope that the money will flow out into the economy. And it's also in this secondary market that the European Central Bank has been intervening in recent weeks to try to bring down the yields on Spanish and Italian bonds and contain the crisis in the two countries.

Hard facts on this secondary bond market are hard to come by since – unlike shares, which are traded mainly through the major exchanges – much bond trading takes place "over the counter", which is financiers' talk for "between themselves". Many banks and interdealer brokers have platforms for trading bonds. The sums are extraordinary: one platform, MTS, which calls itself "Europe's premier electronic fixed-income trading market," claims average daily volume of over €85bn.

For traders in the once-staid European debt markets, the past few months have been a rollercoaster ride. "You come into the office and you don't know if you are going to see Berlusconi saying this, Merkel announcing that and so on, so the risk is very high. So even if you are right, a headline that is unverified can push you offside," says one.

In these nervous times, there has also been a lack of liquidity: in other words, a relatively small number of trades each day, causing volatility and what one analyst calls a "feeding frenzy" when an investor does stick their neck out. "You look at what's going on and it's just fear."

Most bond traders feel sensitive enough about their new-found power not to talk on the record; yet they maintain they have the public's financial interests at heart. "Do I want my pension fund lending to Italy at 3% when it should be more?" says one. "I don't want them to piss my hard-earned against the wall."

And they are dismissive of the idea that a single investor now could play the role of George Soros, who was seen as precipitating the pound's humiliating plunge out of the European exchange rate mechanism by shorting sterling on Black Wednesday in 1992. "Italy's bond market is the third-largest in the world, at €1.6tn. What hedge fund has enough cash to take out a €1.6tn bond market?" asks Louise Cooper at BGC Partners.

Investors in bonds have rarely had a good press. When Tom Wolfe wanted to portray the ultimate amoral money man in his novel Bonfire of the Vanities, he made the philandering Sherman McCoy a bond trader – a "master of the universe". Financial writer Michael Lewis, in Liar's Poker, his account of a stint as a cub bond salesman at Salomon Brothers in the 1980s, sketched his colleagues as fearless, larger than life, and motivated entirely by money and status: the "big swinging dicks" of Wall Street.

But one bond market expert insists traders have been far from gleeful as they watched the crisis on the continent unfold. "People are aware of the job cuts going on around them and of the money clients are losing, and that's putting a dampener on things," he says.

It wasn't always like this. During the period now wistfully known as the Great Moderation – the period of steady growth and low inflation in much of the developed world that ended in 2007 – government bond markets were a haven of calm, untouched by the "irrational exuberance" of, say, the dotcom bubble.

Yet there was a quiet boom going on even in the staid sovereign bond markets: the money that was flooding out of countries such as China in search of a safe haven washed far beyond "risk free" US treasury bonds, driving down interest rates on borrowing for many countries that would once have been seen as much less of a sure bet.

That process was especially pronounced in the eurozone, where the formation of the single currency, amid a heady rush of supra-national solidarity, gave investors a reason to diversify out of ultra-safe German debt. Despite the so-called "no bailout clause," which was meant to prevent profligate eurozone countries free-riding on their thriftier cousins, traders nonetheless assumed that Italian or Greek debt was all but equivalent to rock-solid German "Bunds". "In the first part of the last decade, from say 2001 to 2006, the European bond markets seemed the most boring place in the world," says Simon Derrick, currency strategist at BNY Mellon.

One reading of the sparring between the markets and Europe's rattled politicians over the past 18 months has been as a painful test of the assumption that, when it comes to the crunch, eurozone members won't let each other go bust. But the announcement that Greece would seek an agreement with its private sector creditors aimed at achieving a 50% writedown of its debts exploded the myth that a eurozone member couldn't default. And when George Papandreou's ill-fated proposal for a referendum revealed that both he, and the Germans and French, were thinking the unthinkable about a smaller eurozone, suddenly the idea of government bonds as the ultimate safe haven looked shaky.

"The investment case for government bonds is essentially gone in the sense that people invest in government bonds because they provide risk diversification relative to equities and credit, and that isn't the case any more," says one insider. "The majority of government bonds, with the exception of Bunds, have become positively correlated with equities and credit because there's a euro-exit-risk premium you have to price, and credit-risk premium you have to price."

As the crisis has rolled on, bond traders themselves – often seen by other trading desks as brainier and more subtle – have seen their activities scrutinised more intensely. But the point they make over and over is that the ultimate owners of much of this mountain of debt are banks, which hold it as capital, or pension funds, which appreciate the regular "coupon", as the interest payment on a bond is known. So in many cases, the Sherman McCoys who hold the fortunes of a generation of harassed eurocrats in their hands are looking after our pensions, as well as their own bonuses.