It's working fine now.

I'm just using the CollisionOutlines instead of Physics2D