Best answer as chosen by user cawahe: Aperture is a Ratio
It has nothing to do with the diameter per se, but with the focal length of the lens, and also what size sensor the lens is designed to cover.
A wide aperture longer focal length lens needs to be physically wider; that's why there are few long lenses with apertures bigger than f/2.8, because a larger aperture lens would become too large and even more expensive than they already cost. For example, the Nikon 300mm f/2.8 is $5900.
A lens also needs to be wider to cover a full frame DSLR sensor compared to cameras with smaller sensors.