My new puppy is having his first jab at 9 weeks but will that mean I can't walk him outside until 13 weeks?

My vet has said the first jab should be at 9 weeks as antibodies from his mother's milk may interfere with inoculations given before this age. I'm concerned that this will delay him being allowed to go for walks until 13 weeks (2 weeks after his second vaccination). He is a chocolate lab and has loads of energy and is really confident. I am taking him out in my arms to help get him used to traffic noise but he'll be pretty heavy by 13 weeks of age and I'm worried he needs more stimulation both mentally and physically.