Computer help: How do you future proof your digitally archived documents?
- 5/1/09 10:39 AM
How do you future proof your digitally archived documents?
by Lee Koo (ADMIN) - 5/1/09 10:39 AM
Question:
Hello, over the years I have accumulated a lot of data in
various forms. I have written documents, my own and those
sent to me by others; I have photos in many different file
formats; I have videos in several formats and I have
converted tapes to digital; finally, I have a large
collection of audio recordings. You might also say that I
have a large software collection--it begins with DOS 1.0 for
the OS family and MS Word 1.0 for the word processing family.
My question relates to archiving all of this material. What
format(s) should I use for each to future proof my
collection? I may need to access documents 10 or 15 years
from now and who knows if the future word processors will be
able to open today's .doc files. Will PDF always be
universal? Photos, video, and audio all have efficient
compressed formats, but which ones will survive 50 years from
now? What do we need to do today so that future generations
are not left with a useless alphabet soup of file formats?
--Submitted by C. Philip C., M.D., J.D.
Here are some featured member answers to get you started, but
please read all the advice and suggestions that our
members have contributed to this question.
Stick to standards --Submitted by rbsjrx
http://forums.cnet.com/5208-6122_102-0.html?messageID=3031912#3031912
The Future of your Data --Submitted by waytron
http://forums.cnet.com/5208-6122_102-0.html?messageID=3032162#3032162
Several Solutions --Submitted by Flatworm
http://forums.cnet.com/5208-6122_102-0.html?messageID=3032158#3032158
File formats.. Here today gone tomorrow. --Submitted by hawk318
http://forums.cnet.com/5208-6122_102-0.html?messageID=3032212#3032212
Damn Good Question --Submitted by Artist3d
http://forums.cnet.com/5208-6122_102-0.html?messageID=3032505#3032505
Simple Solution --Submitted by friarchuck
http://forums.cnet.com/5208-6122_102-0.html?messageID=3033280#3033280
Future proof? Men plan, God laughs... --Submitted by Watzman
http://forums.cnet.com/5208-6122_102-0.html?messageID=3033493#3033493
Hard to be certain, but some formats safer than others. --Submitted by BigGuns149
http://forums.cnet.com/5208-6122_102-0.html?messageID=3033629#3033629
If you have any additional recommendation or advice for Philip, please click on the reply link and post away. Please be as detailed as possible in your answer to him. Thanks!
First off . . .
by Coryphaeus - 4/24/09 5:46 PM In reply to: How do you future proof your digitally archived documents? by Lee Koo (ADMIN)
No one can tell us what file formats will be available in any future. All we can do is save what we have. For longevity, CDs, DVDs, and Blu-ray are the current media of choice. I say this because even magnetic data such as tape and hard drives have a shelf life. And tape can break, hard drives can crash, and the sun may rise in the west.
What storage device is in the future? One little item is the four quantum states of a special isotope of I believe cesium (don't quote me on the material). It exhibits four states at the molecular level. Storage of unheard of capacity because each molecule can store four bits.
Save what you have to optical media. In any future, if a file format changes, create new files in the new format. If it's available.
I know you're concerned, but you've asked question no one can answer with any reliability.
The question was about data formats, not medai
by rbsjrx - 4/24/09 7:59 PM In reply to: First off . . . by CoryphaeusAll of the points about storage media are well taken, but they don't address the original question. I have document files that, over the years, have migrated from Apple II 143k floppies to Zip disks to Windows and Linux hard drives. Keeping them is simply a matter of transferring them as the older format becomes obsolescent or the media start to evince the effects of oxidation, moisture, etc. However, some of these old files are pretty much useless because the software to make sense of them is no longer available. This is what the original questioner asked about.
Agree on Proprietary vs. Standards
by Hforman - 5/1/09 10:27 PM In reply to: First off . . . by CoryphaeusThere are issues with either proprietary solutions and standards. The fact is, BOTH change over time. Microsoft is itching to do away with anything written by Word 97, which my former worksite did all of their documentation in. Open Document format will probably change as well, especially since many types of data storage (formats) will need to be tweaked as newer concepts and technology come about.
What can you do? Maintain your data in whatever format pleases you but you need to periodically review your data to make sure the format is current. Do a conversion now and then to a newer format. Automate the process if you can. I know we are not supposed to be talking about media here in this discussion as we went over that in great detail prior to this. We did mention that nothing lasts forever. Try to read an 8" floppy disk today! So, if you are going to move your data from one medium to another, it might be a good time to upgrade the format as well, just to be safe.
everything has a shelf life - even discs
by gbswales1 - 5/2/09 4:18 AM In reply to: First off . . . by Coryphaeusit is well known that discs of all kind also have shelf lives - just as paper photographs, films, videos and the like - I have had compact flash cards fail after a time so guess that nothing is 100% future proof. I think that the storage medium is a far bigger consideration than the file format. There will in my view always be some way to recover files created in a mainstream product even if the company has gone under - eg most word processors can read legacy versions and even those created in other software.
There simply is no such thing as future proof - the only answers are multiple copies stored in multiple location on multiple media formats. It would also be a good idea to maybe archive everything to at least one brand new hard drive which is then carefully stored without further use in an air tight temperature controlled environment.
to make doubly sure then store legacy hardware as well - computer and monitor with the data so that future generations can blow the dust off and get the old "toy" out. With netbooks and the like becoming really cheap this is not so preposterous as it sounds - dont forget to leave the instruction manual in case the concept of keyboards and 2 monitors have long since died! Mind you better explain about electricity, power plugs etc and pray that hasnt been completely replaced with some kind of energy in the air!
Future Proof? little thing called MTBF
by mikefxlee - 5/2/09 6:23 AM In reply to: everything has a shelf life - even discs by gbswales1Uh, mean time between failures. Much lower when one drinks mountain dew and is clumsy with liquids.
There is one method that has survived the test of time, but we would need to relearn much.
It's called oral tradition. Before you go off on the name calling, please look at this
http://en.wikipedia.org/wiki/Oral-Formulaic_Composition
Give it some time to sink in. I mean be the box and think outside yourself for a change.
Then go ahead and call me anything you want (except Roger).
You can make your DVDs last a thousand years, but in that time who remembers how to build a player?
a half bubble short of level
by cesareDH - 5/2/09 6:38 PM In reply to: Future Proof? little thing called MTBF by mikefxleeyou need to get you some help
Did you read the link?
by mikefxlee - 5/2/09 8:40 PM In reply to: a half bubble short of level by cesareDHIt demonstrates a method of data compression, decompression, and integrity checking. And that is not my quote. When you apply information theory, and calculate the amount of data transmitted, factoring in that it has already been transformed into a second order context aware structure - as an inherent characteristic. I'm not going to do the math again. No digital standard includes context awareness. You may see a picture of a chair but have no idea what the object is or was used for.
What the context awareness means is you could carve lines into a rock and with the context it might be 500GB of data. So, by handing down the context via oral tradition, there is no concern for formats or standards. The context inherently has that decoding information. Like a Clave rhythm is an internal time synch, this is an internal second order transformation topographical space matrix.
Such is objectively measurable with an oscilloscope and dual FFT audio measurement instruments from Tektronix. When the context factor is included, the quantity of data is very high. Do the math yourself.
As I do know what a golden ratio base number system implies when AND'd to a ternary and binary system, as well as the multiplication and addition relationships involved with the natural logarithm base e. Which should explain it mathematically.
Almost, AND is not the correct term. it's a much more obscure mathematical concept used when mixing different base number systems.
There are published algorithms to calculate such values.
worry about the storage media, not so much the file format
by luminova - 5/3/09 11:52 PM In reply to: everything has a shelf life - even discs by gbswales1I will have to agree with what gbswales1 said.
i would say, for most digital formats, there will always be some sort of backwards compatibility, or at least a special conversion software for that purpose, but if you really worry, you can always progressively update them anyways to be safe. time is on your side for file format compatibility; you can open early video/audio/document files just as easy as the newest formats (perhaps even easier, since they are more universal).
the best way to preserve something is to diversify file formats and storage media. print a copy, burn it to CD, copy it to flash drive, etc. but i think the best is to simply transfer it to contemporary storage media as they become available, to keep them accessible.
i strongly advise AGAINST using media such as CDs, flash drives, etc. these are good for securing your data against disaster, but NOT against time. those things will be obsolete and unsupported within a couple decades, if not sooner. how many of you have a record player? exactly. computers don't even COME with floppy drives anymore. it is best advised to keep transferring them to the newest forms of storage media.
Just so you're aware...
by rmazzeo - 5/3/09 5:01 AM In reply to: First off . . . by Coryphaeus...optical media has a shelf life also, the longevity of which is in dispute. Optical media can also get scratched up unless stored carefully. External hard drives are much more reliable in my estimation.
Future proof media
by thatJC - 4/24/09 7:33 PM In reply to: How do you future proof your digitally archived documents? by Lee Koo (ADMIN)
All the possible file formats are not the only problem. What media will survive for more than a few years? CDs and DVDs are not nearly as long-lasting as many people believe.
Mass-produced music CDs are stamped from an original with physical pits to diffuse the reading laser beam away from the normal mirror surface - creating the digital one's and zero's. Pretty robust.
Blank consumer CDs/DVDs use a low power laser to heat up a dye on the disk surface. These dyes can fade in even a couple of years, under some conditions. Before choosing an archive format do your homework!
Stick to standards
by rbsjrx - 4/24/09 7:52 PM In reply to: How do you future proof your digitally archived documents? by Lee Koo (ADMIN)
The answer is different for different types of documents. In general, though, use the simplest and most popular standards-based format that you can.
For text, use ASCII if possible. If you need formatted text, use PDF which can be readily converted to PostScript. If you use special fonts, be sure and archive them as well as your document. Once you get it to PostScript, it can be manipulated and printed with any number of standard tools, e.g. LaTeX, which have been around for decades already and are likely to still be around in decades to come.
For images, stick to RAW or TIFF if you can afford the storage space (RAW is better since TIFF is, technically, a proprietary Adobe format). If not, stick to free, non-proprietary compressed standards such as JPEG or PNG. Avoid GIF unless you need animation - it's technically a proprietary format, and limited to 255 colors. BMP is probably OK, but its close association with Microsoft suggests there may be better alternatives.
For video, use MPEG, which is another free, non-proprietary standard.
The same rationale applies to music, where the safest alternatives are either WAV, if you can afford the storage space, or MP3 (part of the MPEG standard) if you need compression.
This brings me to the last point - compressed storage. Data compression is a wonderful thing for archival storage, but you need to avoid niche or proprietary tools. I use RAR all the time on my Windows machines, but whenever I need something to be archived forever, I stick with ZIP which is free, standardized, and universally available. On my Linux machines, I'll use either zip or gzip.
BTW...
by rbsjrx - 4/24/09 8:13 PM In reply to: Stick to standards by rbsjrxA subsequent post compels me to add an explicit caveat. Regardless of what type of computer or what software you use, when I say to avoid proprietary formats, that especially includes _any_ Microsoft formats. Even if you use MS Office, there are options to save documents in formats other than Microsoft's native formats. Some are already on the "Save as" or "Export" menus. Others may require the use of 3rd-party software (e.g. Acrobat or one of the several free PDF writers).
Why? Microsoft has a terrible history of releasing new versions of software which have various levels of incompatibility with previous versions. Right now, I can use a Microsoft-supplied filter to provide interoperability between Word 2003 and Word 2007 documents, but will such filters be available for Word 2020 when I have a Word 97 document? Using PDF/PostScript, I can be reasonably confident I won't be stuck.
Devil's advocate
by BigGuns149 - 5/2/09 12:50 AM In reply to: BTW... by rbsjrxWhile historically I would agree with you since Microsoft has a published standard for OOXML in theory there should be viewers for it as long as there is demand for them and someone willing to write one.
That being said my gut instincts tell me that ODF has a better long term outlook, but Microsoft has a lot of money to try to ensure that their format doesn't get replaced.
RTF (Rich Text Format)
by Owyn - 5/8/09 7:35 AM In reply to: Devil's advocate by BigGuns149is another format which has a good chance of surviving the test of time.
It is unfortunately a Microsoft proprietary format but has very wide support.
http://en.wikipedia.org/wiki/Rich_Text_Format
| Forum legend: | |
 |
Locked thread |
 |
Moderator |
 |
CNET staff |
 |
Samsung staff |
 |
Norton Authorized Support team |
 |
AVG staff |
 |
Windows Outreach team |
 |
Dell staff |
 |
Intel staff |
