‘WP Team Manager’ uses ‘team_managers’ based capabilities set, like ‘edit_team_managers’, ‘edit_others_team_managers’, just replace ‘posts’ to ‘team_managers’ to get the full list of capabilities you may need to get the full access to this custom post type.

If content is created by some other user – members are added to the team by one user, and you wish to restrict access for the other user, you should give to him the access 1st (‘edit_others_team_managers’ capability in this case) and only then to restrict those access for special groups/categories for him.