You can change or delete the name of a specific group. There should always be a default group where contacts who are no longer part of a group will go.