From the church, go around to the back, and the torch trail will pick up again.
That said, you have actually managed to find the end of the swamp by yourself, or at least the close vicinity of it. Right after the potions shop I believe, there should be a huge tree trunk with an entrance down to the goblin lair. The falling down thing is a trap, and not it, but it shouldn't be too far away.