I have to say... I have never ridden this bike, as I am working on it for a friend. I am putting it back together tonight and am going to take it out for a ride in the AM so I can have first hand knowledge of the issue... As to what I know the bike shifts good except for down from 5th when hot...?
I think your right on that the issue is in the shift drum family. When I find what the issue is I will post it for all, however in the meantime any clue helps!