The world's energy system seems to have come unhinged. Oil is trading at record high prices because demand keeps rising even as supplies become unreliable. Oil exporters from Iran to Russia and Venezuela are using their petrocash to pursue agendas that undercut western security and interests. Supplies of natural gas also seem less secure than ever.

The root cause of these troubles is dysfunctional energy politics. The countries with the strongest incentives to cut their vulnerability to volatile energy markets - notably America - are unable to act because influential politicians view all serious policies as politically radioactive. Efforts to boost supply have little leverage because the most attractive geological riches are found mainly in countries where state-owned companies control the resources and outsiders have little clout. Thus, the current energy debates are generating a volcano of proposals that have no positive impact on tight markets.

Yet these structural barriers to serious policy remain hidden because the debate labours under the meaningless umbrella of "energy security". Proper policy on oil and gas must start with the distinct uses for these fuels - each requiring its own political strategy.

The effort on oil must focus on transportation. Vehicles and aircraft work best with liquid fuels that can store large quantities of energy in a compact space and flow easily through pipes to engines. Searching for a better substitute is worthwhile, but the effort faces an uphill battle. With today's technologies, no other energy liquid can reliably beat petroleum. Liquids can be made from coal, as South Africa and China are doing. But that approach is costly and has unattractive environmental implications. Brazil and the US have focused on ethanol, which they distill from sugar or grain from crops. However, those programmes, which account for less than 0.5 per cent of the world's energy liquids, have a negligible impact on the oil market. Yet, America is redoubling its ethanol effort because it is politically unbeatable to reward corn growers and grain handlers who are a formidable force in US politics. Indeed, requirements for ethanol in America have created a more rigid fuel supply system that actually raises the price of oil products, although ethanol's backers originally claimed they would cut energy costs. That same political force also blocks imports of cheaper Brazilian ethanol. In principle, a better approach is so-called "cellulosic ethanol", which promises lower costs as it converts whole plants into ethanol rather than just the grain. But like most messiahs, its attraction lies in the future. So far, nobody has made the system work at the scale of a commercial refinery.

The best way to temper oil demand today is by lifting efficiency. Even this economic winner is politically difficult to implement. The US, which consumes one-quarter of the world's oil, has not changed fuel efficiency standards for new cars in 16 years. Every big economy - even China's - has stricter fuel economy rules than America's. Political gridlock has stymied even modest proposals to allow trading of efficiency credits. A trading scheme is politically inconvenient as it could force US carmakers (which make generally inefficient cars) to buy valuable credits from foreign brands. No politican wants to multiply Detroit's problems.

Even better ideas - such as a stiffer petrol tax - stay stuck on opinion pages of newspapers and in academic journals. Despite what is increasingly termed today's "energy crisis", these ideas barely cross the lips of politicians who want to remain viable among the thicket of anti-tax conservatives and pro-Detroit lobbyists.

The approaches needed for natural gas are quite different. In western Europe, which has long depended on imported gas from Russia, Algeria and a few smaller suppliers, the vulnerabilities are particularly stark. In principle, though, gas dependencies are easier to manage than oil because gas has rivals for each of its major uses. In electric power generation, countries must preserve diversity - ensuring, for example, that advanced coal and nuclear technologies remain viable. While "diversity" is motherhood in energy policy, in reality it requires difficult choices. In continental Europe, for example, policy-­makers have not seriously confronted the conflict between the need for diversity while, at the same time, opening the power sector to morecompetition. Historically, companies in competitive power markets have invested heavily in gas because gas plants are smaller and require less capital than coal or nuclear plants.

Gas suppliers who dream of extending their powers forget that it is harder to corner gas markets when users have a choice. Algeria learnt that lesson in 1981 when it left a key pipeline empty in a pricing dispute with Italy - extracting a better price at the time but losing billions of dollars for the future by destroying its reputation as a reliable supplier.

That lesson should be sobering for Russia today. In December, Gazprom, Russia's giant state gas company, cut deliveries to Ukraine, which then siphoned supplies that flow on to Europe. The company rattled its pipes again last month - threatening retaliation if Europe dared try to wean itself from Russia's gas. While Gazprom's management must pander to Russian nationalism (where pipe-rattling is welcome), the company's long-term viability rests on its reliability as a supplier to lucrative west European markets. Similarly, the recent decision by Evo Morales, Bolivia's president, to nationalise his country's gas fields will give him a boost domestically and might generate some instant extra revenue, but it will also encourage his customers in Brazil and Argentina to look elsewhere for energy.

"Resource nationalism" is back in vogue. But for gas suppliers in particular, it usually ends badly - not least because the infrastructure is costly to build and buyers can afford to be choosy. Gas users can further subdue Russia's rattling by multiplying sources of supply. A robust market for liquefied natural gas will help.

The tendency for gridlock in energy politics means that policymakers must focus where tough decisions matter most, such as efficiency in the use of oil and diversity in the application of gas. Yet, prospects for serious policy are poor - not least because the US, which should be a leader, is the most hamstrung. Luckily, the markets are responding on their own - albeit slowly and patchily. Costly oil is encouraging conservation and new supplies; LNG is accelerating, and gas buyers are more wary of Russian gas than they were a decade ago when Russia was seen as a reliable supplier. If the political structure remains dysfunctional on matters of energy, then the best second is perhaps no policy at all.

Since the Iron Curtain came crashing down, American and Russian diplomats have been searching for a special relationship between their countries to replace Cold War animosity.

Security matters have not yielded much. On issues such as the expansion of Nato, stabilising Yugoslavia and the war in Chechnya, the two have sought each other's tolerance more than co-operation. Nor have the two nations developed much economic interaction, as a result of Russia's weak institutions and faltering economy. Thus, by default, "energy" has become the new special topic in Russian-American relations.

This enthusiasm is misplaced, however. A collapse of oil prices in the aftermath of an invasion of Iraq may soon lay bare the countries' divergent interests. Russia needs high oil prices to keep its economy afloat, whereas US policy would be largely unaffected by falling energy costs. Moreover, cheerleaders of a new Russian-American oil partnership fail to understand that there is not much the two can do to influence the global energy market or even investment in Russia's oil sector. The focus on oil has also eclipsed another area in which US and Russian common interests could run deeper: nuclear power. Joint efforts to develop new technologies for generating nuclear power and managing nuclear waste could result in a huge payoff for both countries. These issues, which are the keys to keeping nuclear power viable, are formally on the Russian-American political agenda, but little has been done to tap the potential for co -operation. Given Russia's scientific talent and the urgent need to reinvigorate nuclear non-proliferation programmes, a relatively minor commitment of diplomatic and financial resources could deliver significant long-term benefits to the United States.

On the surface, energy co-operation seems a wise choice. Russia is rich in hydrocarbons and the US wants them. Oil and gas account for two-fifths of Russian exports. Last year, Russia reclaimed its status, last held in the late 1980s, as the world's top oil producer. Its oil output this year is expected to top eight million barrels per day and is on track to rise further. Russian oil firms also made their first shipments to US markets last year - some symbolically purchased as part of US efforts to augment its strategic petroleum reserve. In addition, four Russian oil companies are preparing a new, large port in Murmansk as part of a plan to supply more than 10 per cent of total US oil imports within a decade.

Meanwhile, the US remains the world's largest consumer and importer of oil. This year, it will import about 60 per cent of the oil it burns, and the US Energy Information Administration expects foreign dependence will rise to about 70 per cent by 2010, and continue inching upwards thereafter. Although the US economy is much less sensitive to fluctuations in oil prices than it was three decades ago, diversification and stability in world oil markets are a constant worry.

War jitters and political divisions cast a long shadow over the Persian Gulf, source of one-quarter of the world's oil. In Nigeria, the largest African oil exporter, sectarian violence periodically not only interrupts oil operations but also sent Miss World contestants packing last year. A scheme by Latin America's top producer, Venezuela, to pump up its share of world production helped trigger a collapse in world oil prices in the late 1990s and ushered in the leftist government of President Hugo Chavez. Last year, labour strikes aimed at unseating Mr Chavez shut Venezuela's ports and helped raise prices to more than US$ 30 (HK$ 234) a barrel. Next to these players, Russia is a paragon of stability.

The aftermath of a war in Iraq would probably provide a first test for the shallow new Russian-American partnership. Most attention on Russian interests in Iraq has focused on two issues: Iraq's lingering Soviet-era debt, variously measured at US$ 7 billion to US$ 12 billion, and the dominant position of Russian companies in controlling leases for several Iraqi oilfields. Both are red herrings. No company that has signed lease deals with Saddam Hussein's government could believe those rights are secure. Russia's top oil company, Lukoil, knew that when it met Iraqi opposition leaders in an attempt to hedge its bets for possible regime change. (Saddam's discovery of those contacts proved the point: he cancelled, then later reinstated, Lukoil's interests in the massive Western Kurna field.)

Russian officials have pressed the US to guarantee the existing contracts, but officials have wisely demurred. There would be no faster way to confirm Arab suspicions that regime change is merely a cover for taking control of Iraq's oil than by awarding the jewels before a new government is known and seated.

Of course, the impact of a war on world oil supply and price is hard to predict. A long war and a tortuous rebuilding process could deprive the market of Iraqi crude oil (about two million barrels a day, last year). Damage to nearby fields in Kuwait and Saudi Arabia could make oil even more scarce. And already tight inventories and continued troubles in Venezuela could deliver a "perfect storm" of soaring oil prices.

The most plausible scenario, however, is bad news for Russia: a brief war, quickly followed by increased Iraqi exports, along with a clear policy of releasing oil from America's reserves to deter speculators. A more lasting Russian-American energy agenda would focus on subjects beyond the current, fleeting common interest in oil. To find an area in which dialogue can truly make a difference, Russia and the US should look to the subject that occupied much of their effort in the 1990s, but that both sides neglected too quickly: nuclear power.

With the end of the Cold War, the two nations created a multi-billion-dollar programme to sequester Russia's prodigious quantities of fissile material and nuclear technology. The goal was to prevent these "loose nukes" from falling into the hands of terrorists or hostile states.

The Co-operative Threat Reduction programme also included funds to employ Russian scientists through joint research projects and academic exchanges.

Inevitably, it has failed to meet all its goals. In a country where central control has broken down and scientific salaries have evaporated, it is difficult to halt the departure of every nuclear resource. Nor is it surprising that US appropriators have failed to deliver the billions of dollars promised for the collective endeavour. Other priorities have constantly intervened, and Russia's uneven record in complying with arms control agreements has made appropriation of funds a perpetual congressional battle. Various good ideas for reinvigorating the programme have gone without funding and bureaucratic attention - even in the post-September 11 political environment, in which practically any idea for fighting terrorism can get money.

Russia has opened nuclear waste encapsulation and storage facilities near Krasnoyarsk, raising the possibility of creating an international storage site for nuclear waste. This topic has long been taboo, but it is an essential issue to raise if the global nuclear power industry is to move beyond the inefficiencies of small-scale nuclear waste management.

Russia should also be brought into worldwide efforts to design new nuclear reactors. The global nuclear research community, under US leadership, has outlined comprehensive and implementable plans for the next generation of fission reactors. The Russian nuclear programme is one of the world's leaders in handling the materials necessary for new reactor designs. Yet Russia is not currently a member of the US government-led Generation IV International Forum, one of the main vehicles for international co-operation on fission reactors and their fuel cycles. Top US priorities must include integrating Russia into that effort, endorsing Russia's relationships with other key nuclear innovators (such as Japan), and delivering on the promise made at last summer's G8 meeting of leaders of the world's biggest economies - to help Russia secure its nuclear materials.

For opponents of nuclear power, no plan will be acceptable. But the emerging recognition that global warming is a real threat demands that nations develop serious, environmentally friendly energy alternatives. Of all the major options available today, only nuclear power and hydroelectricity offer usable energy with essentially zero emissions of greenhouse gases.

Neither government should be naive about the sustainability of this endeavour. Russia is not an ideal partner because its borders have been a sieve for nuclear know-how and because its nuclear managers are suspected of abetting the outflow. Thus, plans for nuclear waste storage, for example, must ensure that they render the waste a minimal threat for proliferation. The US must also be more mindful of Russian sensitivity to co-operation on matters that, to date, have been military secrets.

Another difficult issue that both nations must confront is Russia's relationship with Iran. A perennial thorn in ties, Russia's nuclear co -operation with officials in Tehran owes much not just to Iranian money but to the complex relationship between the two countries over drilling and export routes for Caspian oil. This link to Iran cannot be wished away, as it is rooted in Russia's very geography. Any sustainable nuclear partnership between the US and Russia must develop a political strategy to handle this reality.

The world, including the US, needs the option of viable nuclear power. Yet Russia's talented scientists and nuclear resources sit idle, ready for action.