up vote
down votefavorite

There is an online HTTP directory that I have access to. I have tried to download all sub-directories and files via wget. But, the problem is that when wget downloads sub-directories it downloads the index.html file which contains the list of files in that directory without downloading the files themselves. Is there a way to download the sub-directories and files without depth limit (as if the directory I want to download is just a folder which I want to copy to my computer).
online HTTP directory


3 Answers



up vote81down voteaccepted
Solution
wget -r -np -nH --cut-dirs=3 -R index.html http://hostname/aaa/bbb/ccc/ddd/
Explanation:
  • It will download all files and subfolders in ddd directory:
  • recursively (-r),
  • not going to upper directories, like ccc/… (-np),
  • not saving files to hostname folder (-nH),
  • but to ddd by omitting first 3 folders aaa, bbb, ccc (--cut-dirs=3)
  • excluding index.html files (-R index.html)


up vote13down vote
I was able to get this to work thanks to this post utilizing VisualWGet.(https://sites.google.com/site/visualwget/a-download-manager-gui-based-on-wget-for-windows) It worked great for me. The important part seems to be to check the -recursive flag (see image).
Also found that the -no-parent flag is important, othewise it will try to download everything.
enter image description here enter image description here