Can only change server group in default channel

I can only change server group when I am in the default channel or the channel it is located under. I have one channel called Lobby, the default channel is located under this called spawn, there are 2 other sub-channels under lobby called AFK room and Support. I can set the server group when I am either in Lobby or in Spawn but not in any of the other rooms, neither in other channels or sub-channels.

Can only change server group in default channel