I think you can survive better on per diem, why? you can dictate your days of availability, weekly. After I did HH FT for 2.5 yrs, dropped to per diem and it was never the same. I can choose to work as many days as I want or not at all.
Charting is always the big downside, the worse is management, schedulers and whoever surrogates that are sent your way. As I always say, to many chefs in the kitchen or to many hands in the pot. Same thing.
Do not ever feel that you have to chart in the office, they have to do the same thing to others and not just you, otherwise, they are discriminating. You may argue, as long as charting is done (on time or delayed...mostly) but the work itself would intimidate your supervisor. Use it accordingly and if it fails to make the change, I guess you have to find your bliss. Good luck!