In many systems there are units, med/surg floors in particular, that are perpetually understaffed or difficult to keep nurses in.
Often managers of said units will pull favors when they find out someone is trying to go to another unit to keep them from getting hired there.
Most systems (for profit or not) really look at money closely. If they hire you onto a specialty area without experience (unfortunately prior med/surg time does little to prepare you for areas like L&D, NICU, OR, ED, Peds, et cetera) they are essentially hiring a new grad. This is part of the reason some systems lump their new grads and new to specialty programs together. Then they also have to hire and train someone to your floor as well.
I don't like that you are discouraged from talking to managers from other units. This seems pretty toxic and I'd be looking for a way out. Our nurses can easily email any of our unit directors or managers, and set up a face to face meeting with them.
I think you have to weigh how much you want to stay in your system, and how much you want to change specialties. It seems to me that your chance of getting into the units you want is probably higher at an outside hospital, but I don't know if your tenure in your system gives you benefits that aren't worth leaving (401K match, PTO rate, et cetera).
Ordinarily I would suggest trying to reach out to managers to shadow or cross train, but that doesn't seem to be an option in this case.