I was a new grad in 2011 landed that hospital job on a busy neuro/trauma unit, was quite overwhelmed, shifts where I would cry, didn't feel like I knew what I was doing and after only 4 months I left for a job in a speciality practice, because I said the same thing, floor nursing just isn't for me, I hate it! My nurse manager and director said "your doing yourself a dis-service, you need to give it and yourself a chance" Hey, I figured I'm 40+ years old, I know what I want, I don't need to be miserable, etc. so I left. 7 months later, I feel like I did do myself a disservice by not giving myself a chance, at least to stick it out that 1st year, I feel I would have gained valuable experience to put it all together, if you know what I mean, experience that I am not gaining in a speciality office. Now working in a primary care office, you will see a lot more, but I can tell you there will be tons of calls, parents and families to deal with, filling prescriptions, setting appointments, etc.
While I know it don't care for floor nursing, I also know I probably should have stuck it for a little a while. Think long and hard about it, see if you can't shadow for a day to see what it's really like, talk to a trusted friend, talk to other NP's to get their perspective, maybe they wish they had been a floor nurse, etc.
Good luck with whatever you decide, only you know what is best for you.