You'll need to modify the view and change either the "groups" attribute or the "invisible" attribute on the button element you're trying to hide conditionally. If you have access to Studio, you can open Studio on the Form view for transfers and then click the Validate button. From there, click the "conditional" beside the "Invisible" checkbox in the left pane. You will want to add to the existing domain to make the button invisible when the user is in the Inventory Clerk group. You can add something like this at the beginning of the existing domain:

['user.groups_id', 'in', your_group_id]

You can achieve the same result by making a small module that will modify the button with view inheritance. Also note that there are actually 2 validate buttons in the view that are shown under different conditions. You can see all the hidden buttons by clicking "Show Invisible Elements" in the View tab of the left sidebar in Studio.