Thank you so much Tomamaso - it works (I struggled the whole evening yesterday )

For the reference of others, here was the solution I used to manage access

Requirements

1. Create a list of risk units (i.e departments) and users can be added to the risk units
2. No constraint on the risk units users can be added to. Some can be added to one, while others could be added to multiple
3. Users should only be able to view the risks associated with the risk units they were added to

General Solution

Create a risk unit ('risk.unit') master table

Create a risk unit user lines ('risk.unit.lines') table which references the risk unit master table

Add users to the risk unit lines table

To apply domain force to restrict visibility of departments (security.xml):  

 [( 'department_id.department_lines.user_id' ,'=' , user.id )]

To apply domain to restrict the options in drop down

<fieldname="department_id"domain="[( 'department_lines.user_id' ,'=' , uid )]"/>

Enhancements 

• Create a default department / risk unit option so that will be sorted on top in the case of users being assigned multiple departments / risk unit
• Create child units for example a department like risk management may have subunits like liquidity risk team 
• Have a way to present a list with tree view at the start of the app of available departments and users select the department, which is then applied throughout the rest of the user journey

have you tried uid instead of 'user.id' ?