To enjoy hiking with children, you need to maintain a high degree of flexibility.
When taking breaks, you may need to rest longer than you planned. You may need to eat lunch earlier or later than scheduled. You may find kids aren't interested in seeing what you think is exciting but instead are entirely taken in by something else that utterly bores you.
You may never reach the end of the trail. But that's all right; "bailing" is okay. When hiking with kids, the point is the journey, not the destination.
Read more about day hiking with children in my Hikes with Tykes guidebooks.