The Twentieth Century Society said the future of the building was “uncertain” because of Sadiq Khan’s plans to move the GLA to east London.