Brent North Sea crude for delivery in May slumped $3.09 to $120.89 a barrel in late afternoon London trade.
New York's main contract, light sweet crude for delivery in May, sank $3.89 to $106.03.
"Investor focus has finally shifted from the supply side to concerns over potential damage to demand, as highlighted by the IEA monthly report earlier today, and amid talk of Saudi Arabia reducing production," VTB Capital analyst Andrey Kryuchenkov told AFP.
He added that the market was also weighed down by profit-taking and the softer US dollar.
The Paris-based IEA warned Tuesday that "there are real risks that a sustained $100 dollars a barrel-plus price environment will prove incompatible with the currently expected pace of economic recovery."
The Paris-based agency said in its monthly report that "global oil demand growth has shown signs of slowing in recent months in the face of sharply higher prices."
Following six months of forecast increases, the IEA kept its oil demand prediction for the rest of the year unchanged, saying it should reach 89.4 million barrels per day (bpd), 1.6 percent up on 2010.
World oil production dropped by 70,000 bpd to 88.3 million bpd in March, due to a 70 percent drop in production in Libya, where rebels in the east of the country are fighting Moamer Kadhafi's regime, backed by NATO air strikes.
"The loss of Libyan production and the 25-30 percent jump in oil prices since the crisis began in mid-February has so far drawn a limited response from fellow OPEC members," the IEA said.
Traders are particularly concerned that rocketing prices could undermine the delicate recovery in the United States, which is the world's biggest economy and the largest oil consuming nation.
Surging oil prices have sparked fears of a return to the record levels above $147 a barrel seen in 2008.
The International Monetary Fund warned Monday that high oil prices were a key risk to solid global economic recovery.
"The key downside risk to growth relates to the potential for oil prices to surprise further on the upside because of supply disruptions," the IMF warned.
In a separate report on Tuesday, the Organisation of Petroleum Exporting Countries (OPEC) raised Tuesday its growth forecast for world oil demand in 2011, expecting little import from recent events in Japan and Libya.
World oil demand will now grow by 1.39 million barrels per day (bpd), or 1.61 percent, to 87.94 million bpd, the cartel said in its latest monthly report.
In its previous report, OPEC put this year's demand at 87.74 million bpd.
Oil prices began the week on the front foot, striking 2.5-year peaks on Monday on the back of the weak dollar and concerns over spreading unrest in the Middle East and North Africa.
However, prices then tumbled before Monday's close, losing almost $3, as traders fretted about the impact of recent price rises on the US economy.
What the stars mean:
★ Poor ★ ★ Promising ★★★ Good ★★★★ Very good ★★★★★ Exceptional