Officers recovered about three dozen shell casings at the scene, Milwaukee Police Chief Edward Flynn said, describing the incident as "a spontaneous feud" that sparked a shootout.
"There was a significant exchange of gunfire," Flynn said. "There was great deal of violence."
No suspects were in custody and no weapons were recovered, the Milwaukee Journal Sentinel reported. But Flynn said police were seeking a warrant Thursday evening to search both houses for weapons.
Flynn said the gunfight stemmed from a disturbance Wednesday night between the occupants of two households. On Thursday, an occupant of one house approached the other house and opened fire, the chief said. The house that was shot into had a 3-week-old baby and other children inside, he said.
People inside the targeted house also had guns and returned fire, setting off the shootout, he said.
An off-duty officer heard the shots on his way to work police detail at the Milwaukee Brewers game and called for help. He and about a half-dozen police squads converged on the scene.
The woman who was critically hurt suffered a wound to an artery in her thigh, which one officer tended to - applying his tourniquet to her leg to stop the bleeding. The woman was then taken to a hospital.
The others wounded were two women, ages 27 and 20, and two men, ages 25 and 32, Flynn said. Their conditions were not immediately available.