Things to try.
1. Does a wired connection behave the same? That's a simple experiment, which shows if it's your wireless setup or the connection to the ISP.
2. Try another channel for the wireless connection.
3. Change your wireless protection to WPA2, if it isn't that yet.
4. Change your wireless password to prevent intruders.