Wget for downloding Apache directory files in windows using visualWget

How to download HTTP directory with all files and sub-directories as they appear on the online files/folders list?

up vote down votefavorite

There is an online HTTP directory that I have access to. I have tried to download all sub-directories and files via wget. But, the problem is that when wget downloads sub-directories it downloads the index.html file which contains the list of files in that directory without downloading the files themselves. Is there a way to download the sub-directories and files without depth limit (as if the directory I want to download is just a folder which I want to copy to my computer).

3 Answers

up vote81down voteaccepted

Solution wget -r -np -nH --cut-dirs=3 -R index.html http://hostname/aaa/bbb/ccc/ddd/ Explanation: It will download all files and subfolders in ddd directory: recursively (-r), not going to upper directories, like ccc/… (-np), not saving files to hostname folder (-nH), but to ddd by omitting first 3 folders aaa, bbb, ccc (--cut-dirs=3) excluding index.html files (-R index.html) Reference: http://bmwieczorek.wordpress.com/2008/10/01/wget-recursively-download-all-files-from-certain-directory-listed-by-apache/

up vote13down vote

I was able to get this to work thanks to this post utilizing VisualWGet.(https://sites.google.com/site/visualwget/a-download-manager-gui-based-on-wget-for-windows) It worked great for me. The important part seems to be to check the -recursive flag (see image). Also found that the -no-parent flag is important, othewise it will try to download everything.