ATLANTA (AP) - Graeme Lloyd escaped a jam in the ninth and was the winner and John Wetteland got two outs for a save.
The win ensured New York would return to Yankee Stadium this weekend.
Tonight, in what will be the last game ever at Atlanta-Fulton County Stadium, Braves ace John Smoltz will start against New York's Andy Pettitte in Game 5.
The biggest comeback in World Series history was the Philadelphia Athletics' rally from an 8-0 deficit in a 10-8 win in Game 4 in 1929. There were two other comebacks from six-run deficits, Brooklyn in 1956 against the Yankees and Toronto in 1993 against Philadelphia.
Just six outs from a win, Cox made the rare move of bringing in closer Mark Wohlers to start the eighth. Moments later, Atlanta's 6-3 lead was gone.
Hayes led off with a dribbler down the third-base line that the Braves let roll, but it stayed fair for a single. Darryl Strawberry followed with a single and Mariano Duncan grounded into a force play that could have been a double play except for a bobble by defensive replacement Rafael Belliard at shortstop.
Up stepped Leyritz, in the game only because starting catcher Joe Girardi had left earlier for a pinch hitter. Leyritz lofted a high fly that carried over the left field wall, and his teammates rushed to greet him as he scored the tying run.
Wohlers, who had pitched 7 1/3 scoreless innings in the postseason before running into problems, found more trouble in the ninth. Two-out singles by Cecil Fielder, Hayes and Strawberry - another roller down the third-base line - loaded the bases. Duncan hit a low liner that rookie right fielder Jermaine Dye caught as he staggered forward, tumbling to the ground after the grab.
The Braves had their chance in the ninth against Mariano Rivera after Mark Lemke singled with one out and Chipper Jones walked. Left-hander Lloyd relieved to face left-handed Fred McGriff and got him to ground into a double play. Lloyd, the pitcher the Yankees wanted to send back to Milwaukee after learning he had an injured arm following the trade in August, has pitched 4 2/3 scoreless innings in the postseason.