No advice...just wanted to say that's it's definitely 'doable'. I did it and it worked out very well. Of course, I was very concerned because I was hearing the same advice you've been given...."You have to put in at least a year in Med Surge to get your floor experience." But I had worked odd hours and weekends, holidays and evenings for a long time before I became a nurse and I was fed up with it. I wanted regular hours with weekends off so I took this job. I've loved it! I would highly recommend Endoscopy work.
There were times when I didn't have the experience and wisdom of nurses that have worked on floors but I just had to suck it up and ask experienced nurses what they thought about the situation...then I'd learn from them. Now, I feel pretty confident with what I'm doing and I don't regret a thing. I have to say that I'm also middle-aged and don't plan on going back into the hospital...ever. If I was younger and didn't know where my career might take me, I might have wanted to gain that floor experience as a new nurse so that I wouldn't be shutting any doors later. But I'm perfectly happy spending the rest of my work life right where I am now. Good luck with your decision.