Depending on the shift and the facility, sometimes.
At my facility, through the week most of our kids go to school or workshop through the week. On day shift, once you have everyone up and bathed and dressed, you sometimes have a little time to spend with those who are still home.
Similarly, on 2nd shift, when we come it at 2, not everyone is home from school/workshop yet, so once your showers are done you might have a little time before dinner.
On 3rd shift, on nights we are fully staffed, I might have a little time to chat with a few of the kids who are awake. I will usually do my books (charting) in one of the rooms and just sit by one of the residents while I do them.
Geriatric facilities are a little different, though, since most of them are home all day. An obvious answer is that you can do those things while you care for them. Chat with them while washing, dressing, bathing them, feeding, etc.
A lot has to do with time management, but in most cases, unless you're working home health or something similar, you won't have large chunks of time for just sitting and chatting, unfortunately.
I do often go in on my days off and spend an hour or two just visiting with some of them...I might read a book, hold some of the little ones, and just spend a little time with a few residents at a time. It doesn't take that much time out of my day off, and it makes me feel better about the rushed days.