How many times have you shadowed for a full 12 hour shift? I think that's something you need to do to understand what you'd be getting into. Obviously there are areas of nursing that require less physical labor than others, but do new nurses where you live and work get into those areas without a year or two of med-surg? Can your body tolerate a year or two of med-surg? If your goal is to avoid that type of nursing entirely, do you have the connections to do so?
Have you considered any other healthcare positions? I had no idea before nursing what else was out there. Respiratory therapist, imaging tech, occupational therapy, etc. There are many ways to work in healthcare if that's what you want. If you haven't investigated any of those areas, maybe check them out.
If you are reconsidering, ask your school about deferring your enrollment until next year. If you decide to go, don't be afraid to say that it isn't for you and drop out. If you find clinicals are too labor-intensive for you, just know that working the floor will be much more challenging.