I used to be an atheist
Many apologists, such as Alister Mcgrath, Kirk Cameron and Lee Strobel claim that they were once atheists, before converting to their religion. The argument is that since they were once like you, the arguments that convinced them will likely convince you as well, and you should follow the same path that they took.
This is an instance of the argument from authority, where the authority happens to be the person making the argument. As with other lines of reasoning, the important thing to do is to examine the arguments themselves and see whether they're sound.
In particular, ask which arguments and what evidence convinced the former atheist. Kirk Cameron, for instance claims that he was an atheist until he asked himself, "What if I'm wrong?" He may have been convinced by Pascal's wager, but that doesn't mean that you should be. Similarly, Antony Flew famously softened his atheist stance based on intelligent design arguments based on topics outside of his field of expertise.
Consider, too, that the ex-atheist may mean something else by "atheist" than you do. Many Christians, in particular, seem to believe that everyone believes in God, though some people rebel against him; they may think that "atheist" describes such a person.
Conversion stories are as popular among theists as deconversion stories among atheists. The person may have started out as a lukewarm theist before undergoing conversion; but through retelling, the story has grown into "I used to be an atheist until I was saved." Thus, it may also be worth asking the person why they were an atheist to begin with. If their experience doesn't seem to match yours, the argument that "I used to be where you are now; you should follow my path" probably doesn't apply.
Not so much a counter-argument, but maybe useful as an example of how meaningless the "I used to be an atheist" "argument" is: imagine being challenged with, "I used to be the opposite sex I am now, and I'm better off since my sex-change, therefore you should undergo a sex-change too." Even if we assume that the first part was true, that doesn't mean that it applies to anyone else.
Being an atheist doesn't imply that the person is necessarily correct about his or her reasons for being an atheist. Frequently, such people who enter atheism for bad reasons will often leave atheism for bad reasons. One of the most common examples of this is deciding to be an atheist because one is angry with their god. It's not a rational reason for disbelieving. Some go so far as to assume everyone who is an atheist has the same reasons for disbelieving, and walk away with a straw man understanding of atheists in general.
This argument has a Biblical precedent, in the story of Doubting Thomas. Thomas is someone who was originally skeptical to some aspects of Christianity (in particular, the claim that Jesus had risen from the dead). After Jesus quells his skepticism, he admonishes Thomas:
"Because thou hast seen me, thou hast believed: blessed are they that have not seen, and yet have believed." (John 20:29 )
In effect Jesus chides Thomas for seeking evidence rather than having faith in what he was told. Thus, Doubting Thomas is one long example of someone who used to be an atheist, and is meant to be a lesson in learning from the experiences of others.
Not only is this a fallacy as described above, but it is doubly hard to apply as evidence, since it is unknown how much of the story was simply invented.