Even though Joseph was thrown in prison, God was with him and blessed him. The enemy left Joseph alone because God was with him.
Genesis 39:20-21 Then Joseph’s master took him and put him into the prison, a place where the king’s prisoners were confined. And he was there in the prison. But the LORD was with Joseph and showed him mercy, and He gave him favor in the sight of the keeper of the prison.
No matter the circumstance, God will give favor to those who love and serve Him.