Hi,
The idea of Tasks (To Do) is to remind about something to be done, and this does not have to be done at a particular date and time. You have an option to set the deadline, if the task should be completed by a certain date. In that case the task will show on the calendar indicating the deadline.
For tasks that should be done at a particular date and time you should use Meeting.
On the other hand if the tasks requires a phone call you should use Phone Call module. This is a version of While You Were Out - and when recording it the date and time will default to today and current time (when a phone call was received - for example Joe Doe called and wants you or another employee to return a phone call). You can change the date and time to indicate when this person is expecting a call.

I hope this explains the basic use of Meetings, Phone Calls and Tasks.
For tasks that require more complex status you should use Projects/Tickets.
We don't plan to change the basic structure of these module anytime soon. However if you wish you can use Administrator module to add new field under tasks to record the time. However this would require some changes to the code to display on a calendar.
Please let us now if a Meeting instead of Task will serve the scenario you are describing.