Goldschmidt hit an 0-1 pitch for his seventh homer of the year after Martin Prado led off the third with a single. Goldschmidt went 7 for 15 in a three-game-series.
Arizona took two of three from the team that entered Wednesday an NL-best 22-12.
Arroyo (3-2) allowed an unearned run and five hits in 7 1-3 innings, his third straight outing holding an opponent to no more than two runs. Addison Reed got his 10th save in 11 chances despite giving up a run in the ninth.
Wily Peralta (4-2) allowed 11 hits, tying his career high, and two runs in six innings. Peralta managed to escape trouble all afternoon - except when Goldschmidt came to the plate in the third.
The 6-foot-3 first baseman loves hitting in Miller Park, where he has four homers and 12 RBIs in eight career games. His drive to left-center caromed back on to the field after bouncing off the facade above the wall about 370 feet from the plate.
Milwaukee closed within a run in the ninth after Scooter Gennett doubled down the left-field line and scored on Lyle Overbay's sacrifice fly. Reed struck out Mark Reynolds swinging on a high fastball to end the game.
Arizona stranded 13 runners, including leaving men at third three times through the seventh. But the right-hander, in his 15th year in the majors, kept the free-swinging Brewers at bay well into the afternoon, making Jonathan Lucroy flail on an 86 mph sinker to start the seventh with a strikeout. He ended the inning by striking out Reynolds on a slider.
After Milwaukee put two on with one out in the eighth, pinch-hitter Rickie Weeks lined a single off Brad Ziegler that loaded the bases for Carlos Gomez - who hit the next pitch into an inning-ending 6-4-3 double play that elicited groans from hometown fans.
Arroyo also helped himselfat the plate in the eighth when he singled to center and hustled to third on a wild pitch that Lucroy couldn't immediately find from behind the plate. Arroyo scored on an error by first baseman Lyle Overbay.
NOTES: Arroyo is among eight active pitchers to have appeared in a game at Milwaukee's old home, County Stadium. Miller Park opened in 2001. ... Brewers 3B Aramis Ramirez got a day off. The veteran is hitting .089 (5 for 35) since April 17. ... Both teams get a day off Thursday. Brandon McCarthy (1-5) starts for the Diamondbacks on Friday when they open a three-game interleague series in Chicago against the White Sox. Brewers ace Yovani Gallardo (2-1) opens a three-game series against the visiting New York Yankees.