You will need the Limited Write Financial permission to perform this action.
Click on "Contributions > Envelopes" in the left hand navigation menu. In the table that appears, click on "Unassign" in the drop down menu at the end of the row containing the envelope you want to remove. A confirmation message will appear. Click on "Yes" to finish.
Contributions added using the envelope number will not be deleted but will stay associated with the person.