I for one will tell you that Psych. nurses are real nurses...and very much respected. I worked in psychiatric nursing for 8 years. The one benefit my experience as a psych nurse has given me is a deeper patience and understanding when dealing with difficult patients that a lot of my coworkers actually envy. Never ever feel down on yourself for choosing this career field. People require nursing care for their mental problems and their physical problems. Also, when I was in the military I found out that the psych nurse is one of the most respected nurses in the group. During wartime or wartime training someone (health care providers...officers and enlisted, and soldiers) will either suffer the effects of stress, anxiety, or depression, and maybe they just need someone who can listen and help them work through their problems.
On a side note, the only reason I got out of psych. was I started suffering from burn out (which can happen in any clinical setting), and I needed a change.
If you want to feel better look into taking some classes to help you enhance your physical assessment skills (even in psych, this is important). With your ER background and psych together, you will be a valuable commodity. Early on when I started working in psych, I made it a point to attend as many inservices on the med-surg units as I could. Also, I took ACLS,...which helped because I was the only ACLS certified nurse on my shift on the floor where I worked. I found out they needed at least 1 ACLS certified nurse per shift, so they reimbursed me for my class. It also came in handy too, because as you know codes do occur on psych. floors too.