We imported over 10,000 contacts. If you are getting error 500 you need to inspect php error log. Also good place to check is mysql error log and apache (or whatever HTTP server you are using) error log.
For troubleshooting steps see this post: viewtopic.php?f=6&t=496
Try to increase limits in php and mysql.
As for import procedure see this post: viewtopic.php?f=21&t=815