HTGET version 0.1 (alpha) (C) J Whitham 1998
partly based on snarf (C) 1995-1997 Zachary Beane
Send bug reports/helpful comments to: jwhitham@globalnet.co.uk

Contents:	Description
		Disclaimer
		Installing
		Usage
		To Do
		Credits


Description
-----------

HTGET is a file grabber that will get files from HTTP servers. 
The aim behind this program is to create a downloader that you can leave
running in the background - one that's totally reliable and can cope with
just about any problem, and won't stop downloading unless it's forced to.

It already supports resume, list files, timeouts, minimum rate timeouts
and other cool features. 

Currently it compiles on Linux. It may need substantial modification to
work on other platforms. I have got it to compile on Linux 2.0.30, using gcc.

The networking code is based on a program called snarf, 
by Zachary Beane (xach@mint.net), available from sunsite.unc.edu. See
Credits for more details.



Disclaimer
----------

This software is distributed under the GNU Public License: see the file
LICENSE included with the program for details. Essentially it means
that this software is totally free, but has no warranty of any kind.
In other words, even if it formats your hard disk or brings the Internet
to a halt, it's your fault for using it.

By using this program you agree to the terms of it's LICENSE, and understand
that you are responsible for anything you use it for, and anything it does.


Installing
----------

Untar into a suitable directory, then type "make". 
This creates the binary "htget". Copy that to a suitable place eg. /usr/bin.
Now it will all work.


Usage
-----

You can give htget urls to fetch, eg.

	htget http://www.channel4.com/index.html

will get index.html from channel4.com. You don't need to specify http://.
You can specify multiple urls at the command line.

Command line switches are used to change htget's behaviour, you can
have many on at once:

Long version	Short	Description
------------    -----   -----------

--noresume	-nr	Do not attempt to resume files.
--notimeout	-nt	Never let the connection time out.
--timeout <t>	-t <t>	Set the timeout to <t> seconds.
--redial	-r	If the connection times out, reconnect and resume
			the download instead of returning to the console.
--silent	-s	Dont output download info to stderr.
--nocheck	-nc	When resuming files, dont check the file time/dates
			match.
--list <l>	-l <l>	Download files listed in file <l>.
--logfile <l>	-f <l>	Make a log file <l>
--directory <d> -d <d>	Output files to alternate directory <d>
--noestimate	-ne	Dont estimate the download time remaining
--minrate <r>	-m <r>	Set the minimum download rate to <r> bytes/sec.
--stdout	-S	Output file to stdout. Only works with one file.
--verbose	-v	Lots of information including HTTP headers
--be <n>	-b <n>	Set the user-agent to <n>


Of all the above, the most useful is -r (redial).
This is really handy for big downloads. When you're 19Mb into a 20Mb file,
and the server times out, don't you just want to explode? Not any more.
Redial connects again and resumes the download. Now the only problem is
with servers that don't support resume :(.

minrate is also useful for servers that start off transmitting data at
a good speed, then slow down. When the rate drops below minrate for the
amount of time specified as the timeout, the connection is assumed to
have timed out. Can be combined with redial.


To Do
-----

Implement -nc (nocheck).
Allow output to go to both a log file and stderr.
Add FTP downloads.


Credits
-------

The networking part is heavily based on Zachary Beane's code, used in
snarf.. His code didn't support many of the features I've added, and I
have changed it all so much that this is effectively a completely new
program.

snarf is (C) 1995-1997 Zachary Beane

The original credits list for snarf included:
     o Greg Lewis for helping me find out what flags I
       needed to add to newer SunOSs to get this program to compile.
     o Ganesh (ganesh@cse.iitb.ernet.in) for providing me with a very nice
       snarf-specific ftp interface
     o Sean Connor for helping me patch a few little buffer overruns and
       other cosmetic things. He's incredibly nit-picky, but that's a good
       thing :).
     o The Debian people (doesn't that sound like a rock band) for including
       snarf in Debian distributions, and making more people bother me with
       bug reports (I love them)
     o msw on GIMPnet IRC for doing some beta testing


