I say go for it. You will be awesome at it. I am also on the spectrum and I have some learning disablities as well and I'm gay. However, I am in Critical Care and I absolutely love it. When I was in clinical rotation for OB/PEDs one of the sites we went to the kids were practically jumping all over me, wanting me to pick them up and play with them. Also most peds they have a patient or guardian there with them. The most I ever got was are you a boy or a girl.
Also think about where you want to be in Peds. I know a friend thats in the NICU, I know another that was Pediatric Oncology. I honestly don't think you're going to have much of a problem especially if its your passion.