You can delete whole folders or items within a folder. To delete a folder or an embed file, click the three vertical dots on the Files & Media page.

Click "Delete"

To delete a file within a folder, open the folder then click the three dots on the file you wish to remove and click "Delete"