Yes, the stress will lessen with time, and yes, the vast majority of new nurses feel this way.
I consider myself fortunate to have started as a new grad in an ER in which it was said often, it will take you a year just to know what you're doing, 2-3 to be really competent, and 5 to feel like you can handle anything that comes in the door. I wish that sentiment was more widely repeated -- it's distressing to see new grads come in with the presumption that their learning curve will be short and that they should be comfortably, fully competent within the first year. It's just not realistic.
I hope that you've had a multi-faceted orientation (both classroom and on the unit) which provided a good foundation. The fact that you seem to indicate that your ER is not overrun with new grads is actually a very good thing, even if it leaves you feeling a little lonely. (p.s. ... fellow new nurses with the same gripes you have now might feel comforting ... but they can't actually do anything to help you get to the next level -- so don't write off us more experienced nurses
Unless there are specific aspects of the ER you know you don't like ... and you can realistically say that you're sure you won't encounter the same things in another setting ... stick around for a while and see how you feel in another 6 months.
Good luck to you!