A recurring thing you may notice when browsing publishers' guidelines: they all say something like "know what we publish" or "read several back-issues" or "be familiar with our magazine," blah blah blah.
Okay, it's not really blah blah blah. It really IS important to know the markets. But you can't subscribe to every magazine or read 80 issues of every magazine at the library. There are so many markets out there to "get to know" and it can be overwhelming!
So what do you do?
1) Whenever possible, really do look at an actual copy of a magazine you hope to submit to. Most libraries have a decent selection.
2) Study the websites. If you dig around a publication's website long enough, you are bound to find writers' guidelines (try links that say "about us" or "contact us" or "FAQ"), sample articles, sometimes even theme lists.
3) Use resources such as the Children's Writers and Illustrator's Market (or Writer's Market for those who don't write for kids) and online resources. Some good online resources are: KidMagWriters (lots of goodies for children's writers, including publishers' guidelines and special reports) and Write4Kids.
4) Take your time. You won't get to know all the dozens of potential markets for your work overnight. But the good news is that once you get to know them, it's pretty easy to stay up-to-date.