An update:
I visited the "Cavenagh Road" car park that SBS9818A mentioned.
Yes, there is such a building there, but I have yet to find it any
reference to it being used as a bus terminal.
I just found out that, in 1972, service 45 used Kallang Park
(currently known as Stadium Boulevard) as a terminal. So that
should be the terminal building behind KFC. As for the terminal
building at Stadium Road, near slip road to Nicoll Highway, I have
no idea yet.