Wget for downloding Apache directory files in windows using visualWget

How to download HTTP directory with all files and sub-directories as they appear on the online files/folders list?

up vote down votefavorite

There is an online HTTP directory that I have access to. I have tried to download all sub-directories and files via wget. But, the problem is that when wget downloads sub-directories it downloads the index.html file which contains the list of files in that directory without downloading the files themselves. Is there a way to download the sub-directories and files without depth limit (as if the directory I want to download is just a folder which I want to copy to my computer).

3 Answers

up vote81down voteaccepted

Solution wget -r -np -nH --cut-dirs=3 -R index.html http://hostname/aaa/bbb/ccc/ddd/ Explanation: It will download all files and subfolders in ddd directory: recursively (-r), not going to upper directories, like ccc/… (-np), not saving files to hostname folder (-nH), but to ddd by omitting first 3 folders aaa, bbb, ccc (--cut-dirs=3) excluding index.html files (-R index.html) Reference: http://bmwieczorek.wordpress.com/2008/10/01/wget-recursively-download-all-files-from-certain-directory-listed-by-apache/

up vote13down vote

I was able to get this to work thanks to this post utilizing VisualWGet.(https://sites.google.com/site/visualwget/a-download-manager-gui-based-on-wget-for-windows) It worked great for me. The important part seems to be to check the -recursive flag (see image). Also found that the -no-parent flag is important, othewise it will try to download everything.

APACHE : wget – recursively download all files from certain directory

Case: recursively download all the files that are in the ‘ddd’  folder for the url ‘http://hostname/aaa/bbb/ccc/ddd/’

Solution:
wget -r -np -nH –cut-dirs=3 -R index.html http://hostname/aaa/bbb/ccc/ddd/

Explanation:
It will download all files and subfolders in ddd directory:
recursively (-r),
not going to upper directories, like ccc/… (-np),
not saving files to hostname folder (-nH),
but to ddd by omitting first 3 folders aaa, bbb, ccc (–cut-dirs=3),
excluding index.html files (-R index.html)

Jdbc configurations in eclipse

 Usage of Mariadb java connector


#Download mariadb java connector mariadb-java-client-1.3.6.jar (exact name depends on the driver version) from https://downloads.mariadb.org/connector-java/
#Then include in eclipse under java build path --> libraries --> Add External Jar files and also include in tomcat web server under lib folder
#In servlet use the below code

<pre>
  //loading drivers for mysql
            Class.forName("org.mariadb.jdbc.Driver");

//creating connection with the database
          Connection  con=DriverManager.getConnection("jdbc:mysql://localhost:3306/test","dbuser","dbpasswd");
</pre>

 Usage of mysqldb java connector 


#Download mysql java connector mysql-connector-java-5.1.36-bin.jar (exact name depends on the driver version) from http://www.mysql.com/downloads/connector/j/
#Then include in eclipse under java build path --> libraries --> Add External Jar files and also include in tomcat web server under lib folder
#In servlet use the below code

<pre>
            //loading drivers for mysql
            Class.forName("com.mysql.jdbc.Driver");
 

//creating connection with the database
          Connection  con=DriverManager.getConnection("jdbc:mysql://localhost:3306/test","dbuser","dbpasswd");
</pre>