I feel you, math has always been my stumbling block too, especially algebra. I just had to review it for the GRE, and all I can say is practice practice practice. Math is one of those subjects where you can know all the rules, but still have trouble applying them, or make small mistakes that will completely change the answer, and the only way to rectify it is to just do it so much it becomes second nature.
One thing that makes me stumble on math is knowing HOW to approach a problem, or how to start. If I can see the first few steps, it clicks and I can easily go from there, but just seeing number soup that I'm supposed to simplify or solve is intimidating. Certain types of problems will appear over and over, so learn to recognize what they look like, and the strategies for solving them.
Sorry, it's difficult to give more specific advice without being face to face with pencil and paper, but if you have any specific questions, feel free to message me.