You will need the Limited Write Financial permission to perform this action.
Navigate to the business you want to delete and select "Delete" from the "Actions" menu in the upper right.
You can also delete businesses from the business listing screen by clicking on "Contacts" the "Businesses tab" and selecting "Delete" from the drop down menu at the end of the row of the business you want to delete.
A confirmation message will appear. Click on "Yes" to finish deleting the business.
Transactions linked to this business will not be delete, but will instead be marked as anonymous.