My manager made me go home on annual leave on a Sunday because the unit was quiet. Can I be forced to do this?

No. The Public Health System Nurses’ and Midwives’ (State) Award 2023, Clause 30 (vii)(c) provides that:

The employer should give each employee, where practicable, three months’ notice of the date upon which he or she will enter upon leave and in any event, such notice must not be less than 28 days

Staff cannot be directed to utilise annual leave to meet the needs of the employer unless there are agreed plans in place in accordance with the Leave Matters Policy Directive which relate to excess annual leave. If you are not currently being managed for excess annual leave, and there is no action plan in place to reduce your leave accrual balance, you cannot be forced to take leave.