The function of Groups is to help categorize a users's functional working roles within your setup. While that this sounds fairly similar to departments, there is a distinct difference between the two, not only in semantics but in functional purposes as well. The main differences between groups and departments are:
...
b. Groups are usually agnostic from an organizational structure viewpoint. Lets say, for instance, that an organization has 10 departments and each department has a secretary. In this case, Secretary would be a group in itself and the members of the Secretary group would all belong to different departments.
In short, use departments when you are trying to represent a user's placement within an organizational chart, and use groups to represent the various functions a user might carry out.
To manage groups, access go to Setup Users > Setup Groups. Note that each group created can be tied to a specific organization, although not necessarily so. As mentioned earlier, a user can belong to multiple groups and, unless restricted, a group can have members that spans span several organizations.