How bad of an error do you think this is?

I work on a medical ward, we are the covid ward so are currently taking a mixture of patients.

My hospital has decided to close the covid surgical ward so now we are taking a lot of surgical patients. 

I came on night shift to a newly admitted  patient with a tibial fracture. When I met him his cast had been cut open completely. 

The ortho doctor entry was the only documentation since admission and the plan just stated: check INR, NBM. 

The nurse I took handover from said the team had come and cut the cast open during the day and then left.

 To me the cast looked a mess and I assumed was doing nothing to support the leg as it was wide open. I assumed the doctor had just cut it and not bothered to completely remove it. So I removed it completely. 

When I changed the patient I used sliding sheets and didn't move his lower body. I supported the leg either side with pillows.

I contacted the ortho doctor on call overnight to say the day team had removed the cast/he had no support what action should I take.

The on call said no action overnight just keep ankle elevated.

The next morning the ortho team returned and were angry the cast had been removed. They wrote in the notes that they had given specific instruction to rebandage the cast. But there was no instruction to rebandage or even that they had cut open the cast and the day nurse did not hand anything over to me. 

I realise now I made a massive error removing the cast, I should have sought advice before touching it. The general rule is if you don't understand, don't act.

But I also feel that the surgical team assumed that a medical ward know how to handle a surgical patient when actually they need to document clear plans.

They also didn't ensure the correct bloods were ordered for theatre or that the patient had fluids as NBM, I chased these things overnight.

I'm not making  excuses but I'm on a medical ward with end of life patients, a deteriorating patient and dementia patients. 

Because of this I have a bit of a 'let's get on with this/get it sorted' mindset (the exact reason I think medical and surgical patients shouldn't mix.

Anyway I incident reported myself and the patient was moved to a surgical ward the next night. When I handed over to the surgical nurse they were equally angry with me saying I should have rung the surgical ward for advice.

I realise I made a mistake, how bad do you think this error was? Do you think it showed worrying poor judgement?