Mathias Anderson, who founded the New Wine Ministries allegedly began the relationship with the girl, who is now just 14-years old after meeting her and her mom at a women's shelter run by the church.
Her mom had suspected something was going on for a while but she initially refused to press charges after reporting the pastor to cops in October 2010.
But recently she found e-mail messages as well as several calls between the pair listed on her daughter's cell phone records.
This was after the family moved to Murrieta, GA when Anderson, 41 told them to leave the congregation without providing a reason.
But the mom has since told cops that his wife told her it was because of something that happened between Anderson and her daughter.
Quizzed by officers, the daughter allegedly reluctantly admitted to cops that the pair had been in a relationship, after he began counseling her in 2009.
According to the New Wine Ministries website, Anderson, a former Marine, is married with six children.
He denies the unlawful sex and lewd acts with a child and is being held on $1million bail.