You can not delete users who were created - they are marked inactive. The reason for it is referential integrity. If a user was created and later this person created a record or updated it - a history of it is kept in the database along with this user ID.
The only way to delete a record with that user ID would be to reassign all records of this user to someone else, however logically this makes very little sens. If Joe Doe was hired for one day only and created a note under XYZ company, then even if was fired the very first day and this was his only entry in epesi, the fact is that this was his note and if you want to trust the system it should stay this way.
Technically it is possible of course. You can write a module that scans every module for records created by this user. Then substitute new user ID. Finally you will delete this user from the database.
We have no plans of writing such a module though, sorry.