I take notes in markdown as well and had to deal with this problem as well. There were numerous plugins that gave this functionality that the author implemented but I didn't feel like having an extra plugin just for viewing markdown headlines (especially since I don't use LSP's during software development)

I ended up just using gO command. I made some extra changes so that I could make the gO buffer take up half of the height of the window with C+w+=.

There's also [[ shortcut that will jump you to the parent heading, so you can then search inside the gO buffer for it.

I'm not completely happy with this but it will suffice for now. I will need to automate this in the future, though.