If you had test anxiety, you would know it. There's a difference between someone who can't recall information or gets frustrated during tests vs someone who completely shuts down once the test is in front of them. There are many different levels of test anxiety, but you have to determine if you are having true anxiety or just nervousness. Do your emotions stop your ability to think? You said that you feel confident after taking tests, so I would think that you don't have test anxiety. I'm sure you have nervousness, but that's a good thing. It sounds more like a disconnect between how you are answering the questions. See if you are using the right tactics and strategies for exams.
If you are feeling nervousness, just do stuff to relax... Breathing, relaxing, even talking to yourself can help. One thing I always do during a test is to allow myself a quick break. If you are taking a 50 question exam, put your pencil down 2-3 times during the test and breath a couple of times or rub your eyes. Bring everything back down. If people are finishing the exam before you, don't let your pride push you into taking the test faster. People always do that. Students start handing the test in and then it turns into a race against yourself. It doesn't matter if you are the last person to hand in a test and it is not testament of how smart you are. There's always a person who brags about how they finished it in 20 minutes. Many times, they aren't the highest score. Go at your own pace.
Flip through the test an examine the format of the test. It takes one minute. Scan the questions and assess what you up against. See if there is a question that you want to skip and do last. In algebra, there always seemed to be one question that had way to many steps and I would spend a lot of time on it. Then, I would feel under the gun to finish the exam. If I skipped it and did it last, I knew that I had the rest of the time to do one question. Then, I had time to really think. Exams are all about strategy.
And like everyone else said, talk to your instructor and find out what you missing. Always make appointments for office hours and go over the exam with the instructor. Everything will click when you have hindsight. Then, find out why you answered it wrong and fix it for the next test.