Absolutely! I recently accepted a full-time 7p-7a position on a Med/Surg floor that I thought would be a great, long term thing. I have several years of Med/Surg experience, so the problem wasn't that I was a new nurse and felt like I was drowning or anything, the problem was the entire culture of the unit. A manager who was so sweet and kind that the employees would have walked across broken glass for her but got very little respect, day shift nurses who had been on the unit long enough that, between feeling like they owned the unit and our sweet but pushover boss, would spend their shift sitting at the nurses' station laughing and joking, often loudly discussing off color topics for the patients and their families to hear and in between complaining that they didn't have enough staff, a poorly laid out floor plan where the kitchen, the med room, and the supply room were nowhere near any of the patient rooms, etc., etc., etc.
I think the final straw for me was the night that I came in to work and the day shift nurse, who was sitting on her rear end at the nurses' station joking loudly about the dirty parts of a movie she had seen recently, told me with a straight face that I had a new admission waiting for me that had arrived on the floor almost two hours earlier but she hadn't "had time" to get to her yet. I still had to get report, but I checked on the new admission anyway, and she was having difficulty breathing...alone, in a private room at the farthest point possible from the nurses' station, and didn't know how to use her call light. I...was...FURIOUS!!! Talk about patient neglect! The patient ended up being okay, but I still told the manager what had happened after I resigned from the position...after only four months. I felt guilty quitting so soon, but I knew that I could not work in that kind of environment. I have since been hired by the hospital system where I worked previously for a number of years and, not only am I loving it, but my job in the internal agency pays $11/hour more than I was making at that awful place.
Sometimes you have to take a leap of faith when your heart is telling you it's time. You are NOT a failure for leaving that position, you are doing what is right for you. Nursing seems to attract people who feel guilty acknowledging our own needs and too often bite the bullet, even when we know that we aren't happy or doing what we want to be doing. I used to subscribe to that philosophy, but not anymore. Life is short, and no matter where you work in nursing you are helping people, so you also need to help yourself by doing work that makes you feel happy and fulfilled.
Best wishes to you.