Storage forum: External USB laptop hard drive storage file corruption

Hi Bob - no it's NTFS although I have tried FAT32.

I've also had the same problem on a Dell Dimension desktop so I don't think it's a question of power.

I don't have a spare cable so I'll try and track one down - it's one of those miniUSB to USB cables with a second USB connector (I think for the purpose you have just mentioned - power).

I have no understanding of what exists or remains on a disk when re-formatted or the partition is removed. Is this where low-level formatting comes in - would it be worth trying?

I've read about some USB enclosures generating problems however normally they also generate error messages. My gut feeling is enclosure - cheap and probably made by a kid in asia - anything possible!

Thanks Bob-both that and trying another cable will take some time as I do not readily have these to hand. So, points taken and not excluded - just deferred until possible!

In the meantime, any other knowledge or advice is welcome.

...major internet downtime here!

Re: acoustic. I have already used the Seagate Tools. They didn't offer that option. Incidentally all the drive self-tests came up clear of any errors or irregularities.

Re: files. Why would it be that certain files fail? For instance the AVG Free installer always corrupts whereas some other apps are fine. Out of hundreds of photos in numerous folders, it seems a certain few from a certain couple of folders corrupt (although not always the same photos, from the same folders).

They are fine until they reach external drive.

If it was not for the fact that AVG is freshly downloaded, I would be questioning the source file integrity.

Bob, I will still be trying your suggetions when I can however I cannot help but feel I also need to be asking what might make these certain files susceptible to corruption whilst others not?

Bob, I will follow your advice and try to find a quality cable.

Very many thanks for your insight.

It may not be in the immediate future however I shall post the outcome here.

a. Use the USB PCI card connection.
b. Get SPINRITE and run it on said drive.
c. Run a compare after the copy.

Bob

Hi,

Well, the motherboard of my laptop was replaced, but the data corruption still occurs .....grrrr.

I found only one combination when this awfully annoying problem disappear:
- format the external drive to NTFS
- convert the NTFS filesystem to FAT32 (I used Powerquest Partitionmagic 8); this strange thing was suggested by the manufacturer of my laptop....
- turn off he write cache.

That's all folks.

Bye

I have kind of let this go but strangely I stay subscribed to this topic out of some perverse desire for I don't know what.

I'd like to be following Bob's no doubt very wise referral to power and power cables however I don't have the luxury of quality/brand cables where I am - a dream to one day return to a land with....

I try to replicate some of the routines here where fellow sufferers have moments of hope followed by stark realisation that it ain't so.

All I can offer to maybe further perplex any braincells still assigned to this quirk of external storage is this. In fact this is a repetition of observations I have made before but need to talk about them again because it is a pattern, of sorts. I also add that I don't know about file structures and data or tags that might be embedded in photos, music etc.

I generally backed up music, images and application installers to the external drive - I have many of each in all kinds of formats - mainly itunes AAC, zip, exe, jpg. The photos are the most easy to notice because, well, they are photos. It is not always the same photos however it is always photos from the same two or three folders of photos from several years ago. It is also the same certain application installers (as for the music I do not know). The trouble is, as have been mentioned, it is not easy to trace. A music file will be added to an iTunes library without problem and it's properties reveal all is well - it is only when it is played that it skips short of the end (although the error/corruption is only a few seconds long). This proved a right pain when I restored my library from backup - there was no easy way to identify which were corrupt but to wait until they were played! I did eventually figured out a way to find them using dbpoweramp.

Anyway, I just find it hard to believe this is purely power (more my lack of understanding Bob rather than doubting you). I just find it too coincidental that the same groups of photos will always corrupt - not always the same photos in those groups but always photos from those groups. I think the same applications corrupt. Music I don't know. It also does not follow that larger files corrupt.

Well there we go. Quite therapeutic!

*Sigh*

Not to sound ungratefull, but if I hear "Its yer data cable" or "power problems will corrupt everything" again I may just go insane - I think the HUNDREDS of results you will find in google with regard to this unsolved mystery, combined with the 3 different casings, 5 different drive brands and over 12 different cables, I've used on more than 35 Pc's, Laptops & Macs - Without AND WITH external power supply Supplied Directly to the External casing (Such as Vantec witch allows this) MULTIPLE Power supply's rated From Spec, to over 5Amp output, AND inline with the data cable, and STILL This problem persists for myself and at least 7 other people I know who have these drives (Varying brands & cases) - ALL SUMMED UP totally and Conclusively DESTROYS POWER & FAULTY CABLE superstition...

**IT IS NOT THE POWER**

I have proven this repeatedly, as a 2.5" drive CANNOT DRAW 5Amp - and even with a 5Amp power external - the error still occurs.

**IT IS NOT YOUR CABLE**

The mere fact that People think almost every USB cable that has ever shipped with a casing is faulty is silly enough in itself. Beyond this I have tested MULTIPLE cases with MULTIPLE cables Some of them VERY expensive with all manner of shielding and other such fabulous innovations. ALL CABLES FROM THE CHEAPEST TO THE MOST FANCY PROVIDE THE EXACT SAME CORRUPTION.

Fact is almost 2 years of testing & field research seem to prove that most modern USB integrated chipsets are simply FAULTY by original reference design ...

So far I have NEVER seen this corruption on an Intel system PRE-LGA - OR on an AMD system Socket A or earlier...

It ONLY seems to persist in NEWER motherboard chipsets - and again I too have noticed that some files are more prone to corruption than others - almost as though random arrangements of data themselves affect the USB controller (I know it sounds silly).

In batches of over 200 files the same 3 or 4 files corrupt again & again even when copying them in different orders - where as the rest remain unaffected.

The ONLY 99% cure for this problem I have found is to get a dedicated PCI/USB card - fit it to your pc, and run the drive off of that. this DOES NOT FIX ALL CORRUPTION - but does reduce it SIGNIFICANTLY.

Furthermore this fix reduces almost all corruption irrelevant of power or cable as the add on card STILL only provides 500ma (0.5A) as is spec standard. Further proving my point.

Not much help to laptop owners ....

Sadly - in all my exploration I have not found a perminent way to remedy this situation ...

Sorry if I sound too grumpy now, but I've been battling this with MUCH frustration for a LONG time now and sometimes it seems people post comments without even reading the posts in the first place.

Oh-Well..

Maybe the AM3 / or Core4 or whatever Intel & AMD have lined up, will see this problem addressed, as I'm SURE by now they are also aware of this... As sure as I am by now that this IS a hardware fault with the USB chipsets.

Time will tell...

Yes - 99% is NOT good enough :\

I also use a number of external 3.5" drives - on most of the PC's I've tested my 2.5" drive and So far I've yet to experience any corruption on those ...

I generally prefer to use Vantec external cases - although I do have a few no name brands in the field - mostly using SeaGate HDD's, and to date I have no complaints about those at all...

Actually typing this post gives me an idea - I may just try use an IDE -> 2.5" adapter IN a 3.5" case to see if the problem persists .. will update as soon as I've had a chance....

Again all the talking heads immedietly flap about it being that the 3.5" case is self powered, but i still don't think this is ALWAYSE the case (pardon the pun).

The thing is that most of my 3.5" ext's are resident at a site and don't move around - and the 2.5" was SUPPOSED to be a nice way to move large data - as you don't have to mess around forever with external power (a royal pain in the ass).

Looks like I'm buggered tho.. I also have not yet tested the "Pre-Fab" 1.8" pocket drives that are leaking onto the market - but their poor capacity and relative high cost make them sort of redundant... very soon a 16gb flash drive may be a cheaper option...

So the short answer ... 3.5" is a very safe option as far as I'm concerned , I know a good number of people using them and I've been using them for over 5 years...

Will try a few 2.5" drives in a 3.5" case as soon as I can - just out of interest.

Hi Allcom,
Did you try 2.5 drive in 3.5 USB case already?
Is 3.5 drive in 3.5 USB case works some other way with "FAULTY" USB chipset then 2.5 drive in 2.5 USB case? How then to explain this difference? Then 2.5 USB case is the problem.
I'm using 7zip to archive and then test backup. Almost every time it's corrupted (~6 GB file).

Thank you
Gennady

Well put. I too have spent the last 10 or 15 years or so battling USB drives and data corruption - usually massive data corruption, and/or the drives won't even format. (As long as USB drives have existed.)

This has spanned at least:

- A dozen computer and notebooks.

- A couple dozen versions of Windows, Mac OS X, and Linux (Ubuntu usually).

- All of the major USB chipsets controller (and some cheapo ones).

- All of the major Firewire chipsets.

- All of the major controller chipsets (including the brand that many people swear is the only one that works...am spacing on the name at the moment but if memory serves it is the same or similar to the name of an ivy league university).

- 32-bit OSes, 64-bit OSes.

- Bare installs, bare+patched installs, and fully configured installs.

- Commercial name-brand computers, home-built and rigorously tested computers.

- 256mb RAM, 512mb RAM, 1gb RAM, 2gb RAM, 4gb RAM, 8gb RAM, 16gb RAM (not that that should matter).

- PATA drives, SATA/bridge drives, SATA drives. SATA 1, 2, 3.

- Thoroughly tested bare drives + enclosures, name-brand drive+"permanent" enclosures, drop-in docks, bare USB/SATA connectors+power.

- Bus-powered 2.5" drives, externally-powered 2.5" and 3.5" drives.

- New cables, old cables.

- Thick cables, thin cables.

- Cables with an RFI magnet, and (usually) without.

- Short cables, slightly longer cables.

- Pentium II, III, 4, Core2 Duo, Xenon, Core i5 systems.

- ASUS, Gigabyte, some other big brand, and OEM motherboards.

- FAT32, NTFS with default cluster size, NTFS with 64k cluster size, Ext2, Ext3, Ext4.


*** ALMOST ALWAYS *** with data corruption or just as often, with outright volume failure.

The only exceptions I've ever found to this, were with two...JUST TWO...name-brand commercial 2.5" drive+"permanent" enclosure systems. Those were the only ones that ever worked. One is working right now. It works on EVERY system, every time, with any USB chipset, any USB port, any OS, any USB cable. And bus-powered. It's an NTFS volume with default cluster size. It has never failed me. (I can't remember what the previous one was but it eventually died...but had a long and productive service life.)

The one out of only two that ever worked and is working now, is an old 2.5" Seagate FreeAgent Go. (And I hate Seagate. In my 20 TB server, and in my 30 years as a computer power-user, Seagates have failed far and away more frequently than other brands [with the exception of the IBM DeskStar debacle which yes I owned TEN of the bad ones; including their so-called "enterprise" versions - actually those have failed at a slightly higher rate and earlier on average than their equivalent desktop models, at least in my objective experience.)

When I try to use my own drives in external enclosures, I try to buy high-end 5400 RPM drives (for less heat buildup), and test them rigorously on a SATA port before moving to USB enclosure. As for the enclosure, I only buy the best and research the chipset when buying to make sure I get whatever is most highly recommended at the time. And they always fail. (Even though the drive may continue to be perfectly useful and yield a long life when connected back to SATA.)

I do this every now and then, thinking "surely by now they've got the USB drive kinks worked out by now". Nope. Never.

So yeah, it drives me CRAZY when people give such trite answers as "fluctuating power" or USB cables.

I do have a fairly reliable solution for USB HDD storage now though.

Here's what I do:

1) Buy two expensive name-brand 2.5" HDD USB drives. [Neither name-brand nor expensive necessarily has anything at all to do with reliable, but just in case.]

2) Run Solaris 11 Express [formerly OpenSolaris and can still go with OpenIndianna for openness] in a VirtualBox virtual machine (with VT-x hardware assistance).

3) Map the two identical USB drives via "passthrough" raw VMDK, directly to the virtual machine (so Solaris thinks it has two extra permanent HDDs).

4) Configure the two drives as a ZFS mirror.

I should point out this is a very tricky approach. Since the ports, addresses, and physical drive numbers to USB change depending on what drive is plugged into what port, and in what order, the only way to get this to work is to launch the virtual machine via CMD or bash script. In this script, you have to delete the drives from the virtual SATA adapter, delete the raw VMDK files, recreate them pointing to the correct physical drive numbers, then reattach the new raw VMDK files to the virtual SATA adapter. All in CMD or bash script. Once you get the kinks worked out though, this approach is quite solid. Do NOT attempt to just use passthrough USB port mapping. Trust me, it either won't work or will be terribly unreliable.

Also, do stick with Solaris and not BSD or Linux FUSE for ZFS. I tried to get both BSD and ZFS-FUSE working, and neither is satisfactory for this approach in particular. Plus, Solaris 11 just has the best implimentation of ZFS, hands-down. (For example, it is easy to disable write-caching. Which you WILL want to do. With it disabled, you'll get good write performance. With it enabled by default, it will be so slow as unusable. Don't listen to the fear-mongering about disabling it. You WILL NOT lose data in the event of a power outage [or USB unplug]. The only thing that will happen is that whatever transaction being committed, will be lost. Which, is precisely the same thing that happens with any journaled filesystem, and for this setup, will happen to ZFS whether you have write caching enabled or not.)

The main benefit to ZFS is NOT mirroring. You can get that with NTFS. What it gains you - and this is invaluable for USB drives - is error checking. All data is stored with a checksum. In the case of silent corruption (frequent with USB drives), it is detected by the filesystem. If a file on one drive is corrupt, it will read it of the (hopefully not corrupt) second drive, and heal the first automagically. If it can't fix the corruption, it TELLS YOU.

To me, USB drives with important data, without ZFS is just stupid.

Just thought I'd come back to this thread and say I have had absolutely no problems with using the same 2.5in drive (as well as others), and on the same USB ports, but with a generic 2.5/3.5in/SATA Cable.

On the other hand (or shut one case and another bursts in your face), I haven't been able to connect to a 3.5in drive though although that's probably to do with the drive being corrupted. Any thoughts on that are welcome at:
http://forums.cnet.com/5208-6142_102-0.html?forumID=70&threadID=254112

>same 2.5in drive (as well as others), and on the same USB ports, but with a generic 2.5/3.5in/SATA Cable

What is it - generic 2.5/3.5in/SATA Cable? Where to get this product?
Is it connects somehow 2.5 IDE to 2.5 SATA USB2 Enclosure?

Thank you

Given the severity of the issue why are these units not back at the store or repair shop?