"Thought crime" comes from the idea that even a person's thoughts can be illegal or immoral. The idea was originated in George Orwell's novel 1984, where Thought Police utilize a variety of psychological and surveillance techniques to discover people who are capable of even contemplating a challenge to authority. However, unlike religion, thought crime as seen by Orwell is more behavioral. It was suggested that though you could fake it perhaps for a while, eventually the Thought Police would come because of some slip up that tipped your hand. Religion (especially the idea of sin in the Abrahamic ones) is literally the idea that thinking it is instantly a sin as God will be able to tell (as he is all-knowing.)
Some apologists consider the act of merely thinking of breaking a commandment to be an actual sin against that commandment. Hatred, for example, is considered a sin against thou shall not kill. Another example (an idea that Kirk and Ray use quite often) is that if you have lust for someone, that you are committing adultery. Yet another perfect example, again pulling from the ten commandments is the idea of "Thou shalt not covet thy neighbour's house, thou shalt not covet thy neighbour's wife, nor his manservant, nor his maidservant, nor his ox, nor his ass, nor any thing that is thy neighbour's." The entire idea of coveting anything would be a thought crime.