I read the posts about being level 53 before you could do this part of the quest, but I thought I'd try being only a level 46.
I came up behind Lheao using sneak. I hailed him and he gave this response "Lheao jumps at your voice and begins laughing at his skittishness. He says, 'Ahh, hello there. Not many people know of this place, so visitors are quite rare. Hehehe. It's a very beautiful place indeed, so, ummm, please don't go telling everyone you know about it. Heheh. Thanks, friend.' " I then gave him the robe and Danl's Reference and got the Celestial Fists Book with no issue. Maybe they have changed it.