Absence of evidence is not evidence of absence
"Absence of evidence is not evidence of absence" is a rebuttal to an argument of the form "there is no evidence for X, therefore X does not exist." This objection may be valid or invalid, depending on circumstances.
For example, imagine an explorer who lands on an island. seeing no buildings or other signs of human habitation, he concludes that the island is uninhabited. This is an unwarranted conclusion: there might be inhabitants on the other side of the island. In this case, absence of evidence of habitation is not evidence that there are no inhabitants.
But if, a few weeks later, the explorer has explored the entire island and has still not found any signs of habitation, he is justified in believing that the island is uninhabited. In this case, inhabitants should have left signs somewhere; if there are no signs where there should have been some, then it is likely that there are no inhabitants.
The two aspects of this phrase might therefore be summarized as "when presented with some proposition P, we should initially keep an open mind. But if a diligent search fails to produce evidence that P is true, then we can rationally conclude that P is false."
Absence of evidence for God
The question of whether God exists is considered very important to many people, and has been for millenia. Thousands of people have assiduously searched for evidence that a god exists. Nonetheless, there is still no good evidence for a god. Many atheists thus feel justified in not believing that any gods exist.