First, I try to remember that, oftentimes, difficult family members are dealing with feelings of helplessness, fear or anger which has nothing to do with me, but with the situation, especially if the patient is suddenly ill or injured. Often times, they will take out those feelings on us.
Most of the time, actively listening to family members and repeating what they are trying to say, goes a long way to defusing a situation. If they feel like they are being acknowledged and their concerns have been/ are being addressed, it does a lot to calm and reassure them, easing fears.
If they are feeling helpless, I like to listen to them, validate their feelings, and give them small tasks to do. In my opinion, if family is going to stay at the bedside, they are going to work. If that is the expectation, then it helps weed out the freeloaders. Help me turn, help me boost, help meet the patients needs, as appropriate. Be clear what they can and cannot do. Turning off alarms is not acceptable! Oftentimes, giving the person things to do gives them a feeling of contribution and like they are helping to care, a sense of purpose. This calms them enough to stop being a pain in your behind.
For the angry, they may be dealing with grieving, feelings of guilt if they had anything to do with the pt's condition, or other complex emotional issues. For them, I can listen actively, acknowledge their anger by saying, "It sounds like you are feeling angry," and validate the feelings. Often times they don't know that they are angry, or don't understand why. Then it's time to call one of our awesome social workers!
If none of these tactics work, it may be that the family member needs to be medicated and that is outside my scope of practice.
At this point, good boundaries, clearly verbalizing expectations (it is not acceptable to have a party in the room, you may not call the staff names, you may not hit the staff,) and consequences, (you will be escorted outside by policemen and taken to jail, you will not be able to visit the patient anymore,) can work wonders. A written contract has been used where I work, as well, with the especially difficult visitors, usually family.
Hope this helps.