What should I do to make a week out of Williamsburg, VA?

I know about Colonial Williamsburg, Busch Gardens, and Water Country USA. How many days should I plan for each place I visit in the area? Should we swing by Virginia Beach while we are in the area? Keep in mind, I have a 2 year old and a 4 year old.