It is TBI, to begin with. It is totally helpless human being, unable even to scratch his nose. It is pretty much hopeless, and it is to the end of his life, and he might or might not realize it. And he can speak sailor's language even before that.
I think you understand it all already.
Only one more prospective I can offer from my own days of being bedbound is that those learned, stereotypical cliche phrases, including "it is not necessary to speak like that with me" you used, "I understand that it is difficult" (which, unless you had experienced something like that, is a lie), "sorry about that", etc. can literally drive your patient crazy. Imagine, for example, people intentionally causing you pain and discomfort every time they come close enough and every single time you hear "oh, sorry about that" in the same casual not-at-all-sorry tone. I was absolutely there and understood what they did, which is questionable with your patient. But after several days I asked my nurse either to just do it silently or say something else for a change. She was shocked because she perceived "just telling something pleasant" as her "iamjustdoingmyjob" stereotype, a task of a sort, and just had no idea what else to say, but we figured it out. Later, when I was working with similar category of patients, they confirmed my impression.
So, try to be a little bit more creative with your responses while remaining professional. Take this guy as a person he is now. Do not criticise, do not confront, do not rationalize. Ask him about ways he prefer things to be as long as it is possible and within your scope of practice. Allow him as much control as possible. If he has a soft spot like favorite show, candy, etc., do go extra mile to get him that thing. If he blurts, ask in normal soft tone what exactly he wants and let him know that you won't do it till he asks without cussing.
It is a tough job to care for such patient. (((Hugs and prayers to you))))