No need to beg China for PM invite: Downer