Archiving, splitting and uploading a large file

Have you had problems uploading a large file to your webserver? is your connection letting you down? If you said yes to either of these you may have considered uploading you file in parts and then re assembling them on the server. I have a terrible internet connection at work and somtimes uploading one file can take me hours. It seems to start off really well uploading at a reasonable rate then after a few mins it slows right down. What i usually end up doing is splitting it up and uploading each file one at a time and this seems to get the job done faster. If you are using a mac or linux you will have this functionality out of the box, unfortunately if you are like me and are currently using windows you will need a few extras before you try and do this. To do this on windows visit Unix Utilities and download the files and follow the instructions for installing. This will basically allow you to run any of the common unix/linux terminal programs in your msdos prompt its a great addon and is something i can’t live without on windows now. Once thats all done open up your terminal or command prompt and type the following replacing the word filename with the file you want to target.

tar -cvf archive.tar.gz filename | split -b 10m archive.tar.gz archive.tar.part

What this basically does is create a new compressed archive and then splits it into 10mb parts. If you need smaller or larger parts change the 10m to something else for example for 5mb change it to 5m. This should now have created a few files named archive.tar.partaa, archive.tar.partab, archive.tar.partac etc. The amount of them will vary depending on the original filesize and the size you set to split them into. You want to now upload each part to your web server. Once its all there and uploaded log in to your server via ssh and to the directory where the archive parts are and run the following command.

cat archive.tar.part* | (tar x)

This will take all the archive parts and combine them into one file and then extract this to the working directory. Thats it all done!