Pay attention, that 1st, user should have edit access to the event, in order to see it. If event was created by other user, this user should be able ‘edit_others_events’ or similar capability. In other case such event will be hidden from user, even if you allow valid category ID for him.