One of three things is likely preventing you from removing the drive: First, is if you have a windows explorer windows open that is pointed to that drive and have its contents on the screen. Second, your antivirus is in the process of scanning the device or has left a file/folder locked as active. Close the windows explorer (not internet explorer) windows and wait a minute or so and try again. Third: you have a program installed like a photo transfer program that is locking up the drive. I have one that installs with my Olympus camera and one with my Lumix camera. Pain in the rear they are, really. Close the programs like that that are sitting in your taskbar. Worst case scenario, close all windows and wait a minute and see. Last resort: If you shutdown Windows, it should force whatever process is using it to complete and the system should shut down normally. If not, there is a remote chance that you have a virus that is cataloging the files on the thumbdrive so it can phone home to its master for identity theft reasons. Scan your computer with antivirus.

Also, regarding your thrown away thumbdrives: While unplugging a drive before using "remove safely" can corrupt the data, the thumbdrive itself is often just fine. You just can't access that data anymore. You can re-insert the thumbdrive and reformat it like any other disk by opening My Computer, right-clicking on that drive, and selecting "Format". Be warned, it will definitely toast any data you have on that drive, and
if you accidentally do this to a hard drive, well, things won't be
pretty. You may want to try a long format (uncheck quick format) even though it will minutely decrease the lifespan of the drive (most thumbdrives have a finite lifespan based on the number of writes to each cell). The reasoning for this is that if a particular cell of the memory on the thumbdrive was damaged by unplugging it, the long format should catch it. If you can right-click it because the drive doesn't show up in "My computer" anymore, you may have to try re-partitioning it through the Computer Management-->Disk Management control panel in Administrative Tools (or right click "computer" and select manage). If you are uncomfortable doing this, have a geek friend try. If the drive still is not accessible, then recycle it. It's actually illegal in many states to throw electronics away now as there is a fair amount of lead in the solder of electronics.