Go ahead and apply to poly if you're sure there's a course
which fits your intended career. But don't go to poly just because
you hate GP.
Being great in your English helps GP, but GP isn't English, and
there are many students every year who are still not
great in their English, eg. China nationals, but because they
have figured out the requirements of GP, they still manage to
score As and Bs.
Worst case scenario, you just only need to pass GP (quite easily
done, as opposed to scoring A), and excellent grades in all your
H2s will still allow you to enter many good NUS, NTU and SMU
courses.