Young French striker Gueye tapped home the winner seven minutes from the end after Jack Rodwell's second-half opener had been cancelled out by Kosta Barbarouses.
New signing Jermaine Beckford made his first start in an Everton shirt and was almost gifted a first goal when he just failed to reach an under-hit backpass.
Fellow striker Louis Saha then warmed the gloves of former Norwich goalkeeper Michael Theoklitos before ending a goalless first-half with a disappointingly tame shot after his bursting run.
Rodwell opened the scoring four minutes after the re-start with a delightful curling shot from outside the penalty area.
The visitors looked the most likely to add a second with Seamus Coleman firing just over before Victor Anichebe, who came on for Rodwell, was also off target.
The Roar found an unlikely equaliser after 66 minutes when Barbarouses fired home from a tight angle after a quickly-taken free-kick.
But 20-year-old substitute Gueye made sure Everton left Australia with a perfect record as he tapped in Jose Baxter's cross from close range after 83 minutes.