You must have the Write Background Checks permission to perform this action.
Background checks can only be deleted while they are in the "Invited" status. To delete, navigate to People > Background Checks from the left hand menu.
Then locate the background check you want to delete and the choose "Delete" from the menu at the end of the row.
Once the background check status moves to "Pending" or "Complete", it can no longer be deleted. Instead, you can archive them. That will remove them from the list by default (can be brought back updating the table filters).
To archive, it's the same steps as for deleting, but you'll see "Archive" in the drop down menu instead.
Note that you cannot delete a safeguard the background check is connected to. You will get an error message if you try to.