Article delegate-en/3235 of [1-5169] on the server localhost:119
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[Reference:<10309@~/Mail/drafts>]
Newsgroups: mail-lists.delegate-en

[DeleGate-En] Re: File size issue with FTP cache
23 May 2006 09:23:27 GMT feedback@delegate.org (Yutaka Sato)
The DeleGate Project


Hi,

In message <_A3232@delegate-en.ML_> on 05/22/06(20:38:40)
you Steffen Kaiser <p44eqbdyi-5sbstgh6vwxr.ml@ml.delegate.org> wrote:
 |but only see the file with the size below 2GB.
...
 |Current remote directory is /pub/debian-iso/sarge.
 |ncftp /pub/debian-iso/sarge > ls
 |debian-31r0a-i386-netinst.iso
 |
 |# ls -Al /mnt/mirror//pub/debian-iso/sarge
 |total 9021632
 |-rw-r--r--    1 root     root     4686706688 Jun 13  2005 
 |debian-31r0a-i386-binary-1.iso
 |-rw-r--r--    1 root     root     4428851200 Jun 13  2005 
 |debian-31r0a-i386-binary-2.iso
 |-rw-r--r--    1 root     root     113541120 Jun 13  2005 
 |debian-31r0a-i386-netinst.iso

The data returned by DeleGate responding to LIST command in FTP protocol
is that of the output of "delegated -Fls -l" command.  So you can test
it with:

  % delegated -Fls -l /mnt/mirror//pub/debian-iso/sarge

I noticed that non-BSD Unixes seem not support files over 2GB in
stat()/lstat() system calles.  Thus -Fls fails in stat()/lstat() system
call failes for files over 2GB in "rary/fstat.c:ls1()", then they
are not displayed.
The solution for such Unixes seems to use stat64() instead of
stat().  I made a test version to do so and uploaded it as
"ftp://ftp.delegate.org/pub/DeleGate/tmp/delegate9.2.2-pre2a.tar.gz".
To enable stat64, you need to make it with "-DSTAT64" like this:

  make CFLAGS="-DSTAT64"

I hope it could be compiled on your Linux and your problem is solved.

Unfortunately, my ancient Linux (RedHat7.1) seems not give the standard
definition of stat64 by default (and I don't know how to enable it),
so I copied it into "include/file.h" and enabled it as this:

  make CFLAGS="-DSTAT64 -DSTAT64H"

But the definition of "struct stat64" is unlikely portalbe among Linuxes.

In message <_A3233@delegate-en.ML_> on 05/22/06(20:40:48)
you Steve Brown <ps4gabdyi-5sbstgh6vwxr.ml@ml.delegate.org> wrote:
 |> If you did it with the "cp" or "cat" command on the same host where
 |> DeleGate runs, then it shows some hint.
 |
 |I used the 'cp' command, and it successfully copied a 4.7GB image from
 |the DVD drive to the filesystem that Delegate uses for cacheing.
 |
 |> I uploaded 9.2.2-pre1 (at ftp://ftp.delegate.org/pub/DeleGate/alpha/)
 |> which includes refinements of ftp-data transfer:
 |> 
 |>  - SIGXFSZ handling
 |>  - improved abortion of data transfer (ABOR command)
 |>  - faster data relay (reduced delay by poll())
 |>  - displaying the size of huge files in 2GB < size < 4GB
 |
 |So the maximum limit has increased from 2GB to 4GB if I'm reading this
 |correctly. Would this still mean that it's still not possible to
 |transfer, say, a DVD ISO image at 4.7GB ?

Displaying files sizes over 32bits correctly is another problem.  But it
is independent of platforms and the solution is clear; respresenting file
sizes in "long long int" and print or scan it with "%lld" (or "%I64d" for
VC++ ;)

Cheers,
Yutaka
--
  D G   Yutaka Sato <y.sato@delegate.org> http://delegate.org/y.sato/
 ( - )  National Institute of Advanced Industrial Science and Technology
_<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
Do the more with the less -- B. Fuller

  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V