Liran,
Exactly. In general it is just for grouping companies or contacts. However some modules are designed to use groups to filter selection of records. For example Projects module allows creating projects only for Companies assigned to Customer group.
By the same token we can lock certain modules or even individual fields, or hide them from view based on Contacts group. While this is better done utilizing permissions controlled by phpGACL (Admin->Users), sometimes it is necessary to let an employee control this filtering without giving her/him admin rights.
Ultimately it is up to a developer if filtering by group will be used. Here is a sample code:
// Filter criteria for Company Name
public static function projects_company_crits(){
return array('group'=>array('customer'));