Taiwan's foreign ministry said it hoped the deal would help spur tourism between the Asian neighbours.
Under the agreement, restrictions on the number of carriers providing regular passenger services on the profitable routes -- currently two from each side -- will be lifted.
Taiwanese air carriers will be allowed to offer unlimited flights to any Japanese destination except for Tokyo.
The main airlines to be affected by the deal are Japan Airlines (JAL), All Nippon Airways (ANA) and Taiwan's China Airlines and EVA Airways.
A total of 923,000 Japanese visited Taiwan in the nine months to September, a rise of nearly 50 per cent over the previous year.
During the nine-month period, the number of Taiwanese tourists travelling to Japan fell by 21 per cent to 845,000 in the wake of the March 11 earthquake and tsunami, said Taiwan's Tourism Bureau.
Trade between Taiwan and Japan totalled $70 billion in 2010.
What the stars mean:
★ Poor ★ ★ Promising ★★★ Good ★★★★ Very good ★★★★★ Exceptional