The downside is that you have to live in a town the population of your block, with the nearest grocery store maybe 30 miles down the interstate, 6+ hours away from the nearest airport, maybe you have to drive 50+ miles to see a dentist or a general practitioner (who is often really a nurse practitioner who has her own clinic and has a traveling GP who visits once a week) and if you need any specialty care, you are sent 200-300 miles away, to Denver, Wichita, or Topeka.
Your kids may ride the bus one hour or more each way to school, and even then, they may only have 50 kids in their entire graduating class. This is an advantage in some ways, but in teacher recruitment and class offerings it is definitely a disadvantage. You might have the same teacher teaching three or more subjects, you don't have the class offerings larger districts have, and academically speaking some of the smaller, rural districts are behind their larger counterparts, particularly where AP and college prep classes come to mind. You'll drive sometimes 4 hours to play a football game. Many smaller districts have fewer sports offerings--only the major ones, no tennis, golf, swimming, some places nothing but football/volley ball, basket ball, and track. Few if any extracurricular clubs or intramural sports or activities. You aren't going to find fencing, archery, film club, or chess club in small town Kansas.
I went to a central Kansas school in a smallish town where my graduating class was around 200, my cousins went to schools where their graduating class was less than 50;. I didn't have to drive far for school, but for junior high and high school my cousins drove almost 45 minutes (better than taking the bus, the ride was almost 1 1/2 hours each way). I was jealous, because they had their own cars and were driving to school at I'm thinking 14?
None of us (myself, my cousins, my friends from school) felt even close to prepared when we went to college. We were successful in our college careers, but we floundered for a while with the academic expectations, and were jealous of the people who grew up in Wichita, Topeka, Lawrence and KC who had good schools where they could take Latin, play in the kickball league of their school, and do after school activities like film club or science olympics.
While there are some fantastic advantages for families in the very small towns of rural northwest Kansas, there are also some significant disadvantages. Most of those towns are shrinking in size (hence the ability to buy cheap, cheap homes) but they are shrinking for a reason: jobs, schools and opportunities.