I've had the same weird feeling whenever people say that too. Something just doesn't seem to match up with what the Bible has to say about sin. But I think that the sketchiness of the statement has to do with the context it's used in, or perhaps more specifically, what's being proved.
For instance, the main way I've heard it that kind of makes me want to disagree is when people are talking about judging whether or not someone is a Christian (and just cause I used the word "judging", I don't mean "passing judgement"). It goes something like this: "Since all sins are equal in God's eyes, then a a little pride is just as bad as killing someone, or doing drugs, etc. So you can't say that someone's not a Christian because they live in a habitual sin of incontinence, because you probably live in a habitual sin of pride". The logic sounds good, but the problem is that it seems to contradict nearly every epistle written.
Anyways, that's my two cents. It's a very good question, though--I still don't really have a satisfactory answer. One thing I do know: that God is holy and we are ALL sinful, and thus damned. But God, while we were dead, sent His Son to die for our sins and save us from God's wrath. And that is enough for me. (But don't get me wrong, I LOVE theological nitpicking. It's probably my favorite pastime)
I feel like sin is sin, because it separates us from God; however, whenever people say that "all sin is the same in God's eyes" it makes it sound like God is an unaware, ignorant being. Obviously, that's not what these people mean :). But God created the order of the world, and with that made natural consequences for certain sins, some worse than others. If that makes sense. In any case - I don't think that all sin is the same in God's eyes, but at the same time, any sin separates us.