The French Open champion was beaten 6-7 (5-7) 6-3 6-4 in one hour 56 minutes in her first match since a fourth-round defeat by Alison Riske at Wimbledon.

The Australian, 23, forced a first set tie-break from 5-3 down but Kenin took the second with three breaks of serve.

World number 29 Kenin then broke twice to race 4-1 ahead in the deciding set before seeing out a surprise victory.

The 20-year-old will play either Dayana Yastremska - who beat British number one Johanna Konta on Monday - or Victoria Azarenka in the third round.

Also on Tuesday, Spain's Carla Suarez Navarro defeated Venus Williams 6-4 6-2 to set up a second round match against Anett Kontaveit.

Estonian Kontaveit, seeded 19th, recovered from a set down to beat Russia's Maria Sharapova 4-6 6-3 6-4.

Meanwhile, Sharapova's 22-year-old compatriot Daria Kasatkina overcame German world number 13 Angelique Kerber to reach the second round, winning 0-6 6-2 6-4.

Japan's world number two Naomi Osaka, chasing Barty's world number one ranking in Toronto, plays German Tatjana Maria on Wednesday.