Darn, you're giving away my favorite *nix secrets!
|
| Author | Content |
|---|---|
| theduke459 Feb 15, 2007 6:29 PM EDT |
Next you're going to be talking about cpio! |
| dcparris Feb 15, 2007 7:04 PM EDT |
No, it's CP3O and R2D2. :-p |
| DarrenR114 Feb 16, 2007 3:18 AM EDT |
It's not R2D2 - it's R2M! (okay - it's a bit of a stretch in programming humor for a bad pun ... *I* thought it was cute) |
| dcparris Feb 16, 2007 10:43 AM EDT |
What's that saying about one good crack deserves another? |
| DarrenR114 Feb 16, 2007 10:59 AM EDT |
theduke459:
You mean something like using cpio to do remote backup of your home directory over ssh like this: >$ find ~/ -depth -print |cpio -oaV |gzip -c |ssh remote.server.address 'gunzip -c |cpio -imVd' |
| dcparris Feb 16, 2007 2:28 PM EDT |
Darren, I actually used a KDE desktop app for backup once. It created huge (3-4GB) *tar.gz files that I now cannot decompress at all. (A) I don't understand how gzip can compress, but not decompress those files. (B) I'm afraid to try that again until I can figure out how to get my data back. I have been able to extract a small handful of files, but not the whole kit & kaboodle. Do I really need 4GB of RAM? While I'm at it, I'll give it another shot now that I have debian on my primary box. It's my first GUI-less server, and currently just running Apache and CUPS. Those files were created on SUSE 10.0 (not OpenSUSE). After migrating to Ichthux, I tried extracting the files, but to no avail. I tried using the CLI to no effect - raised this issue previously here on LXer, in fact. I found no help from my LUG, either. Any ideas? |
| DarrenR114 Feb 16, 2007 5:10 PM EDT |
Don, As an immediate answer to your dilemma - I don't believe you need 4GB of RAM to handle the large tarballs. If gzip compressed the files, then gunzip should be able to unzip them. What you might try is to get an external USB drive with plenty of free space. There are adapters available that will connect an IDE drive as an external USB drive. Once you have enough target space available, try to cat the file and pipe the output through to gunzip. The '-c' parameter for gunzip says to accept input from the console (stdin): >$ cat verybig.tar.gz | gunzip -c > /media/USBdrive/verybig.tar You could even do this across different machines if you don't have enough space on the local and can't get an external drive: >$ cat verybig.tar.gz |ssh remote.machine.address 'gunzip -c > verybig.tar' I'll have to check and see if the gzip format is like the zip format. I know that the zip format puts the file handling information at the end of the file. With files of the size you're talking about, that would be a lot of data to read through before they could even begin to decompress, if gzip format is the same way as zip. Also make sure you have any of these files backed up before you mess with them. |
| Aladdin_Sane Feb 16, 2007 5:34 PM EDT |
> KDE desktop app for backup Which one? > huge (3-4GB) *tar.gz files that I now cannot decompress How many files? What error? > Do I really need 4GB of RAM? In my experience, highly unlikely. I've never heard of a practical case where the hardware requirements would be higher to undo something than to do it. > I have been able to extract a small handful of files Sounds like an experience I recently had. Still can't completely explain it. I created a large .tar.gz backup file and could not uncompress the archive locally. When I used a different system, its tar command, via NFS mount, I could extract the whole archive without error from the NFS mount, to the same NFS mount. In my case, the files were archived with tar -z under Fedora Core 5, failed to fully extract under FC6, successfully extracted as an NFS mount under Debian Sid. I'm unable to reproduce the issue now, indicating that that particular version of tar or gunzip may have had a bug. I can't recall the error I got, but it may have been something as silly as "out of memory." Yet another argument against a monocultural computing environment, I guess. |
| dcparris Feb 16, 2007 6:47 PM EDT |
Sorry, it was Konserve. I'm attempting this operation on the 50+GB partition of an 80GB HDD. Even with the 4GB backup file, there shouldn't be a space problem. iirc, that machine has something like 512MB RAM. Konserve was automatically creating backups of /home every week. I occasionally burned a file to DVD for archive purposes. All I need is one of the backups to work. I won't get to work on it again until probably Sunday evening. I'll see. |
| Aladdin_Sane Feb 18, 2007 9:18 PM EDT |
>I won't get to work on it again until probably Sunday evening. Any luck on that? My research hasn't shown anything except that bugs are induced, and fixed, in GNU tar regularly. I've not been able to reproduce the similar error I was having. Having the exact error string that you get would help us hunt down the problem. Probably not related: I did recently run across a commercially-produced BASH script that attempted to self-extract its files to /tmp (using tar or cpio I don't recall which) and failed every time with an "out of space" error. The /tmp partition in question had 100's of GB free. Lame: These type of developers need to stay in the Windows world until they get a clue. After fixing the parsing error that was causing the math problem, we got another (stupid) error, and gave up. |
| dcparris Feb 18, 2007 11:13 PM EDT |
Well, funny you should ask. I tried mounting it on my Debian GUI-less box (the only one I have that has a DVD player) using 'mount /dev/cdrom0' and /media/cdrom0. I couldn't see the file on the DVD. I brought it to work with me tonight (not normally here at this time), and it shows up on the Windoze box as a tar file. I know it exists - I was able to access it in Linux before. I assume I need to mount it differently, but not sure what options I need. I was logged in as root, since I was doing other tasks as well. Maybe someone can give me a suggestion on my mount command? |
| dcparris Feb 19, 2007 3:01 AM EDT |
O.k., I got the file. The DVD needed to be mounted via /media/cdrom0 Will report back results later. |
| dcparris Feb 19, 2007 8:58 AM EDT |
O.k. It failed. I ran 'tar -zxvf' on the file, which shows being 3.6-ish GB. Only about 20 or so files came out. tar ended without any messages at all. |
| jdixon Feb 19, 2007 9:03 AM EDT |
DC: Does a tar -tzf give you the full list of file names (-t gives you a list of files in the archive)? |
| Aladdin_Sane Feb 19, 2007 1:02 PM EDT |
>tar ended without any messages at all. Grr. >I ran 'tar -zxvf' on the file I'd suggest, given enough room on the target, that you copy the archive from the optical disk to the target storage device before attempting to extract from it. Sometimes Linux has this "thing" about crossing device, partition and/or file system boundaries. So do DOS and Windows, sometimes. jdixon's suggestion I second, as a means of helping narrow down the problem area and possibly getting a good error message from tar. |
| dcparris Feb 19, 2007 2:11 PM EDT |
> I'd suggest, given enough room on the target, that you copy the archive from the optical disk to the target storage device... That's what I did. I copied it over to /home/don/, and then ran tar. jdixon: tar -tzf did what tar -zxvf did - it lists a few files, along with their directory structure, and then stops listing anything, but seems to continue trying to extract or list files. Again, whether I try to list or extract, it gets so far, and then hangs(?). I don't know what it's doing to be honest. At some point, if finally returns to the prompt. |
| jdixon Feb 19, 2007 3:21 PM EDT |
tar -tzf did what tar -zxvf did - it lists a few files, along with their directory structure, and then stops listing anything... In that case, tar or gzip is almost certainly running into something it can't understand. Otherwise you would at least be able to get a file listing. So, the next step is to determine whether the problem is with the compression or the archive. Can you gunzip the file? Oh, there appears to be a tar repair program availalbe for the small fee of $149.99. :) |
| dcparris Feb 19, 2007 3:48 PM EDT |
Hmmm... Allegedly, the gunzip worked. I now have a file: perms, owner, 4309643776 bytes, dates, file_name.tar (I would copy/paste, but I'm doing this via PuTTY from work, and it doesn't seem to like me highlighting and trying to copy the text. Oh well.) I'll give tar another shot, just for fun. |
| dcparris Feb 19, 2007 4:14 PM EDT |
O.k., well running tar -xf gave a message about "removing leading `/' from member names". Do I need to run tar with a different option? That's what it looks like. I'll see if I can find out how Konserve actually creates the file. |
| jdixon Feb 19, 2007 4:21 PM EDT |
> "removing leading `/' from member names"... Do I need to run tar with a different option? From the tar man page: -P, --absolute-paths don't strip leading `/'s from file names Does that help? However, unless you want to write the files starting at the root of your filesystem, in which case they may overwrite your existing files, that's probably not what you want. |
| dcparris Feb 19, 2007 4:39 PM EDT |
Well, it _did_ extract the same files. So apparently, there must be some issue with the tar file. I'll read the tar man page and see what else I can find. |
| jdixon Feb 19, 2007 4:50 PM EDT |
> So apparently, there must be some issue with the tar file... Looks like it, yes. From what I understand, tar is a simple sequential archiver, so if there's an error it often loses everything beyond that error. He's something from uncovered by a "repairing a tar archive" search on Google: In many case if there is a corrupted tar file, the following command can be used in an attempt to extract the file: % cat [tar-filename] | tar -xvf - NOTE: Where "-" is the STDOUT I have no idea why that would work when a standard tar doesn't, but it's worth a try. |
| jdixon Feb 19, 2007 5:01 PM EDT |
DC: And finally, you may find this page to be of some use: http://oss.bestsolution.at/bash/repair_tar_archives.html.en |
| dcparris Feb 19, 2007 5:05 PM EDT |
I assume you used '% ' as the prompt? Anyway, it looks like it's hanging up at the usual spot. |
| jdixon Feb 19, 2007 5:14 PM EDT |
> I assume you used '% ' as the prompt? Anyway, it looks like it's hanging up at the usual spot. Well, the person quoted did. :) OK, that leaves the page above as the last option I could find. The perl script there is supposed to scan the tar archive and find good tar headers. That should enable you to pick up at the first good point after the bad section, so you'll only loose the corrupted section. Or, if you have access to a Windows machine, you could see what the demo of that tar repair program can do. See http://www.datanumen.com/atr/index.htm to download it. |
| Aladdin_Sane Feb 19, 2007 5:14 PM EDT |
I've seen the stripping leading '/' error in my own backup logs: I ignore it. Not stripping leading '/' causes risk, risk of overwriting files you actually wanted. Unless Konserve did something special in the archiving, I'd ignore this error as "Mostly harmless." The tar default is to always ignore/strip leading slashes for safety. Contrary to my sense, 'tar --help|less' spits out more info than 'man tar' in Debian based dists. Funny characters upsetting tar are possible. See -H, --format=FORMAT and --ignore-case just for some examples, there are more. When I was having this extract problem, --format did not fix it, but it did help narrow down what was not the cause: The results were slightly different with each attempted format. Another troubleshooting option is vv. As in 'tar -tzvvf archive.tar.gz'. Be sure to check 'tar --version' to be sure you are at ver 1.15.1 or better. See http://www.gnu.org/software/tar/#releases (It may be worth it to manually install 1.16.1 if all else fails.) gunzipping archive.tar.gz to archive.tar may help eliminate some complexity. More about upsetting tar with funny characters: http://www.gnu.org/software/tar/manual/html_node/Selecting-A... Besides asking the authors of Konserve what might have happened to your archive in creation you can also see who to blame at http://www.gnu.org/software/tar/manual/html_node/Authors.htm... |
| dcparris Feb 19, 2007 10:44 PM EDT |
O.k., I did successfully gunzip the tar file. So I am now dealing with 'archive.tar'. Debian Etch came with GNU Tar v.1.16. I am currently awaiting the output of the perl script. I'm tired and going to bed. Shhh. |
| dcparris Feb 20, 2007 10:58 AM EDT |
I never did see any output from the perl script. I'll keep looking. I'm also going to try an older archive to see what happens. |
| dcparris Feb 20, 2007 11:09 AM EDT |
A little good news: I have a couple DVDs of data I backed up in 2005 without tar. I just copied the pertinent directories to the DVDs. So not all is lost - just the 2006 archive, if I am unable to recover it. However the older archive may contain the info I was after to begin with. If so, then at most, I've only lost a few documents, near as I can tell. Nothing really catastrophic (thank goodness). Anyway, I really appreciate all the efforts. You guys have really tried like the dickens! Thanks! |
Posting in this forum is limited to members of the group: [ForumMods, SITEADMINS, MEMBERS.]
Becoming a member of LXer is easy and free. Join Us!