As always, Chas has the greatest answers. If you have a union, this should also be addressed in the contract.
Our hosp nursing P&P states we can float to "like" areas, e.g., NICU to peds or post partum, med-surg to other "floor" areas like ortho or neuro, PICU, PIMU, peds floor to NICU (they work in the less acute, convalescing area, with a staff buddy, and are carefuly oriented), etc.
If, when floated, we recieve an assignment with which we do not feel safe, we do try to negotiate; if that doesn't work, there is an assignment despite objection form we can fill out that goe to management and to the union.
We can refuse an assignment, but if an assignment is accepted, you can't leave, that's considered abandonment. It would probably be better NOT to accept a dangerous assignment than to put a pt's life and your license in jeopardy.
BE SURE to write a letter stating your side of the story for placement in you personnel file.
I think you did the right thing. When I was a new grad, I looked at floating like some sort of adventure, or a learning experience. I would even consider how it would be to work there; if they were mean to the float... I asked a gazillion questions, they were probably glad to see me go!
Even now, when the census gets low in the NICU, I try to keep it in my mind that it could be my turn to float today, so that they don't surprise me. I keep a good attitude; I know I'm not a "plug and play" nurse, but I usually know nurses on those other floors because they float to us. That makes it easier, at least there are friendly faces.