The discussion has gained new urgency ahead of an Asia Pacific Economic Cooperation (APEC) summit in Japan next month, at which the 21 members will again talk about the tricky issue of opening up their markets to each other.
Although APEC has made little headway for years in creating an envisioned free trade zone in the dynamic Pacific Rim region, a US-backed initiative that started five years ago with a group of small economies is gaining momentum.
If the government in Tokyo doesn't act fast to come on board, "Japan will be left out as an orphan in the world," the chairman of the Japan Business Federation, or Keidanren, Hiromasa Yonekura, warned this week.
The free trade grouping, called the Trans-Pacific Partnership (TPP), so far has just four signed-up members who have agreed to drop most tariffs and other trade barriers against each other -- Brunei, Chile, New Zealand and Singapore.
However, the world's largest economy, the United States, is now in talks to join the group, as are Australia, Malaysia, Peru and Vietnam.
This week, Japanese newspapers reported with alarm that China -- the world's top exporter which months ago overtook Japan as the global number two economy -- was for the first time considering entering the talks.
Japanese trade officials now fear that "Washington might join hands with Beijing over Tokyo's head," as the Yomiuri Shimbun daily put it.
Japan's centre-left Premier Naoto Kan -- who just agreed in principle on a free-trade deal with India, Japan's 10th such pact -- has voiced enthusiasm for liberalising trade to bolster the nation's ailing export-led economy.
He warned that Japan "is lagging behind in signing free trade and economic partnership agreements compared with countries such as South Korea," which is now Japan's top Asian rival in the crucial auto and electronics sectors.
However, Kan faces opposition within his own party, where many worry about the impact freer trade would have on Japan's highly protected agricultural sector -- and therefore on powerful rural electoral constituencies.
The Kan government has been at war with itself over the issue.
A Cabinet Office survey this week made the point for joining the TPP, arguing it would boost Japan's economy by as much as 0.65 percent.
The trade ministry, similarly, warned that failing to sign up would lead to a 1.53 percent drop in gross domestic product (GDP), as the key automobile, electronics and machinery industries would lose market share.
The agriculture ministry, on the other hand, argued for continued protection, warning that a flood of cheap agricultural imports would hit Japanese local farmers hard and result in a 1.6 percent drop in GDP.
Japan, citing food-security and cultural reasons, has long protected its now highly inefficient rice farmers against imports of cheaper grain from big producers such as the United States, Australia and Vietnam.
Farmers' groups have stepped up their lobbying against Japan joining the TPP, seen as a building block for a wider Free Trade Area of the Asia-Pacific.
But others argue that Japan must bite the bullet or be left out in the cold.
"Unless we seriously think about opening up our country, Japan's competitiveness will diminish," warned Foreign Minister Seiji Maehara.
South Korea has already sealed a free-trade pact with the European Union, signed a free trade accord months ago with India, and has been negotiating with Washington.
In a sign of Japan's declining competitiveness, the share of South Korean-made electronics in the European Union grew to nearly 10 percent last year from five percent a decade ago, while Japan's share halved from 20 to 10 percent.
"If Japan fails to join the TPP," said Masayoshi Honma, professor of economics at Tokyo University, "its economy will be left behind the robust growth of Asia and the rest of the world."
What the stars mean:
★ Poor ★ ★ Promising ★★★ Good ★★★★ Very good ★★★★★ Exceptional