Most CNM programs are Master's programs; some are DNP (doctor of nursing practice) programs but it seems to me most schools
are still offering the MSN in Midwifery (or equivalent). So yes, you do need to get a nursing degree and then midwifery is a Master's level degree at least. (if being a CNM is your goal).
I can tell you the two paths I am considering, as I, too, have a BA already (though not in nursing). One option: take an accelerated BSN program or traditional BSN program, then apply to graduate school for nurse-midwifery, or option two: apply to an accelerated BSN-to-MSN program for students with a non-nursing bachelor's degree. There are other options--if you are planning to live and work in New York or a tiny handful of other states you could apply to school to be a CM (certified midwife) instead of a CNM (certified nurse midwife), so essentially you get the midwifery part but not the nursing education, but CMs are only licensed in a few states, whereas CNMs can work legally in all states. There is also the CPM route (certified professional midwife), but laws for their practice varies state by state, and almost without exception they only can work out of hospital, usually in a solo or small home birth practice.