OR traveler here. Perhaps you just have itchy feet! Travel relieves those symptoms, and makes the OR much more challenging learning the preferences of new surgeons, anesthesia, staff; patient flow, different services and new procedures, and charting software. Not easy, and in many assignments, it takes a couple weeks for minimal functionality, far more time than.
But back to switching specialties, it sure doesn't sound like you are in a teaching hospital (which is an optimal first staff job no matter how sure you are of a lessor job being right). Small hospitals makes switching specialties internally very difficult. Regardless of your current position, you should stay a minimum of 2 years, particularly in your first job. Anything less will make you look bad to any prospective employer (also a minimum to travel).
If you really have your heart in a more clinical specialty, put some energy into it! Take a critical care class. Read a few books. Shadow some nurses as someone else suggested.Your performance at interviews will get much better showing enthusiasm, giving reasons, and showing that you are actively working towards a career in a new specialty.
You may be then able to answer some clinical questions - here is one I failed at an interview as a new grad (simple in hindsight): You are taking care of infant with an a-line and notice blood dripping on the floor. What is your first action?
Consider applying at a lot of teaching hospitals and be prepared to move out of state to achieve your goal.