We're having the same problem
Hey, it's funny how similar our problems are. My XBox 360 is doing the exact same thing. It can't be the router, because the router connects me quite quickly to the internet from my laptop. It can't be the cable because it's the cable that's right out of the box from Microsoft and it works connecting my computer to the router. It has to be the XBox.
I'm having a friend who supposedly had a similar problem come over today and see what he can do with it. If all fails, I'm either taking it to Geek Squad or sending it back. I hope it's something that can be fixed.
Let me know if you have anymore luck.