The time may come to clean up. Here are steps I have taken

To delete the account for one or more users

  1. Sign in to Office 365 with your work or school account.
  2. Go to the Office 365 admin center.
  3. Go to Users > Active Users.
  4. Choose the names of the users that you want to delete, and then select DELETE Delete.
  5. In the confirmation box, select Yes.

Done; not so fast.  The deleted users is not fully gone yet. It takes 30 days after you have deleted the user for it to purge from Office 365.  However there is a way to do this faster

To delete, deleted users in Office 365

Connect to Exchange Online using the Windows Azure Powershell module.

To connect you enter the following cmdlet’s:

This will prompt you for your credentials and stores them within $msolcred.

Next we enter to connect using the stored credentials

Now that you are connected you can issue the following command to list deleted users

Display deleted user

To remove the deleted user

If you had multiple users, this method would work to remove all deleted users recycle bin