Split-Shift Floating

I am currently on an assignment where I am sometimes asked to switch units mid-shift (4 hours on one unit, 8 hours on another). Sometimes, I am asked to do this not out of an emergent need, but rather because an 8-hour shift staff nurse or per-diem nurse picking up a shift prefers not to work on a certain unit (and this facility has been short-staffed frequently as of late).

The split is done between a step-down unit, and a telemetry unit where the patient ratio can go up to 8 patients at night-- and that is the floor where nurses do not want to pick up shifts, so I have been asked to float there to pick up 7 patients and then receive an admission, after taking care of step-down patients for the first 4 hours.

I do not feel comfortable doing this again, as it is simply too much, and there is too high of a risk that something will be missed. I cannot properly assess and care for 8 telemetry patients (one of whom is always an admit) when I am always getting there late due to completing duties in step-down (and I refuse to cut corners, as this facility does audits). This floating mid-shift is never done by staff (I am on 12 hour shifts and they are on 8 hour shifts), and as there are no other travelers on my night shift, no one else is doing it.

My contract does have a float policy, but it does not specify mid-shift floating. I am planning to refuse the next time I am asked to float mid-shift to care for 8 patients because a staff nurse or per diem nurse coming in for their shift does not want to work that unit, and I am even prepared to be sent home for refusing (and also I don't think they would do that due to short staffing), as I feel strongly that it is that unsafe with those ratios. Have any experienced travelers encountered a similar situation? Any advice would be appreciated!(and please not just 'suck it up', as that is what I have been doing, and it is just unsafe to continue.)