The greenback surged to a one-month high in the morning, providing buying momentum for Japanese stocks, but gave up the gains as the day wore on.
Business was also subdued by concerns about the future of Donald Trump's economic agenda as he struggles to push through a controversial health bill.
Expectations for another increase in US borrowing costs were fanned after the head of the New York Fed, Bill Dudley, suggested policymakers could widen their parameters when deciding the course for monetary policy.
And San Francisco Fed president John Williams warned that the world's top economy could overheat if rates were not lifted at the right time.
They follow Yellen's increasingly hawkish tone and the bank's plan to wind down its bond-buying programme to suck excess cash out of financial markets.
Greg McKenna, chief market strategist at AxiTrader, said in a note that "Dudley expanded the Fed's mandate in a way that suggests he sees the Fed with a broader remit than just inflation and unemployment".
Yellen is due to speak in London later Tuesday before heading to a central bankers' conference in Portugal, with her remarks pored over for clues about the Fed's plans.
TOSHIBA SINKS
The dollar jumped to ¥112.08 at one point Tuesday - its highest level since mid-May - but soon retreated below its late New York levels. However it is still above the ¥111.31 in Tokyo earlier on Monday.
Japanese equities rose, with the Nikkei ending 0.4 per cent higher.
Toshiba fell 1.7 per cent after a report in the respected Nikkei business daily said the firm was set to sign a deal offloading its prized memory chip business to a consortium of US, South Korean and state-backed Japanese investors.
The conglomerate is desperately trying to raise money after suffering massive losses on its US nuclear operations that have raised doubts about its future.
Among other markets Hong Kong ended down 0.1 per cent, Shanghai closed 0.2 per cent up and Sydney fell 0.1 per cent.
Singapore put on 0.3 per cent and Seoul was 0.1 per cent higher, Wellington edged up and Manila rallied 0.8 per cent.
In early European trade London was flat, while Frankfurt and Paris each slipped 0.3 per cent.
The pound barely moved after Prime Minister Theresa May's ruling Conservative party signed a deal with Northern Ireland's right-wing Democratic Unionists to prop up the government.
The move to maintain its shaky hold on power comes after May lost her majority in this month's election, while ministers embark on tough Brexit talks with the European Union.
On oil markets both main contracts extended gains after last week's battering to 10-month lows.
- Key figures around 0820 GMT -
Tokyo - Nikkei 225: UP 0.4 per cent at 20,225.09 (close)
Hong Kong - Hang Seng: DOWN 0.1 per cent at 25,839.99 (close)
Shanghai - Composite: UP 0.2 per cent at 3,191.20 (close)
London - FTSE 100: FLAT at 7,447.27
Euro/dollar: UP at US$1.1194 from US$1.1181 at 2050 GMT
Pound/dollar: UP at US$1.2727 from US$1.2721
Dollar/yen: DOWN at ¥111.70 from ¥111.89
Oil - West Texas Intermediate: UP 33 cents at US$43.71 per barrel
Oil - Brent North Sea: UP 35 cents at US$46.18 per barrel
New York - Dow: UP 0.1 per cent at 21,409.55 (close)
What the stars mean:
★ Poor ★ ★ Promising ★★★ Good ★★★★ Very good ★★★★★ Exceptional