What you need to do is create a different permission type in Administrator->CommonData->Contacts->Access
Then set your user to the new Access which is striaght forward I hope (Administrator->Manage Users->edit->Login Panel
Now that the user has special access, you need to specify what kind of access that permission has.
Go to Administrator->Record Browser -> Permissions and see the current examples
Once you looked at them, you want to create/modify them so it does what you need to do.
Sorry my answer is a bit vague but it give you the idea how to set it up.