Ah, looks like you've seen ep6, which I haven't yet.
Fuck, Joffrey must be up to some seriously nasty tricks to make you say that - hard to think what could make him sink any lower in one's estimation after the oh-god-I-can't-watch scene with Ros and the other girl in the second season. I'll have to watch tonight and find out.
Varys is a great character, isn't he? I've actually had him pegged as a 'good' guy (to the extent that the ASOFAI universe has any) from season 1 - not good in any kind of epic, heroic way, as he says himself, but more in the sense of a sort of Benthamite utility. I genuinely believe him when he says that he serves "the Realm" and that what he wants most of all is peace - he just thinks that if one person has to die, or even if ten or a hundred have to die, it's worth if it can prevent a huge war in which thousands die. Like when Ned was in the dungeon and Varys said he could spring him out of there, but didn't - I take that to mean Varys thought that by having Ned humiliated (made to take the black) but ultimately released unharmed, he could avoid an all-out war between the Starks and the Lannisters. Which, if true, would mean even he misunderestimated Joffrey as a power-drunk adolescent shithead rather than a true psychopath of Caligula-esque proportions. Or maybe he just thought it would be easier to broker an allegiance between the Starks and one of the surviving Baratheons in order to get rid of Joffrey.
Then again, again, isn't he conspiring with that beardy bloke Illyrio to bring about a Targeryen restoration? He doesn't seem the ideological sort so it's hard to believe he's doing it out of a fanatical devotion to House Targeryen, so maybe he thinks peace will be easier to achieve with Dany on the throne? But then, last surviving Targeryen hell-bent on revenge + Dothraki + dragons doesn't really sound like the optimal recipe for peace in our time...and Varys didn't have much of a problem with Robert's orders to have Dany bumped off (in fact wasn't it his idea?), but maybe he knew Ser Jorah would foil the attempt, thus inciting Drogo into war against Robert...
I've probably spent far too long thinking about this.