Article delegate-en <_A4364@delegate-en.ML_>
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[delegate-en/4364] [Reference:<_A4363@delegate-en.ML_>]
Newsgroups: mail-lists.delegate-en

[DeleGate-En] Re: delgate 9.7.7 ftp to sftp proxy and file size
01 Feb 2009 18:42:51 GMT feedback@delegate.org (Yutaka Sato)
The DeleGate Project


Hi,

In message <_A4363@delegate-en.ML_> on 02/01/09(17:35:37)
you James <pbeiqbdyi.ml@ml.delegate.org> wrote:
 |> "int" means the signed integer of 32bits with the max value 2GB-1.
 |> The limitation will be able to be enlarged by replacing "int"
 |> with "FileSize" as this:
 |>
 |>  static FileSize DATA_MAX = 16*1024*1024*1024
 |>
 |> or you can bypass the limitation just by setting the value to zero.
 |>
 |>  static int DATA_MAX = 0;

I'll modify the code to enable specifying the max. size with "datamax"
option as follows:

  MOUNT="/* sftp://server/* datamax=16g"

The modification will be like the enclosed patch in DeleGate/9.9.2-pre1.

 |> But I'm not sure if the current implementation of sftp/FTP gateway,
 |> using the sftp command as a filter process, is stable enough for
 |> such usage.
 |
 |Many thanks for the assistance. My C is very rusty, I forgot about the
 |data typing.
 |What are the main concerns around stability ? ie what should I watch out for ?

I'm not sure but it is likely that buffering into a huge temporary file
during long time can suffer some kind of disturbation caused by limitation
like disk space, session timeout, and so on.  Also the sftp command
designed for human interaction via tty, at the endpoint as a client,
can produce some kind of unknown interaction or data format which might
be bad as a proxy server.

Cheers,
Yutaka
--
  9 9   Yutaka Sato <y.sato@delegate.org> http://delegate.org/y.sato/
 ( ~ )  National Institute of Advanced Industrial Science and Technology
_<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
Do the more with the less -- B. Fuller


*** dist/src/delegate9.9.1/src/sftp.c	Fri Jan  2 12:05:14 2009
--- new/sftp.c	Mon Feb  2 03:36:28 2009
***************
*** 46,52 ****
--- 46,55 ----
  FILE *fopenTO(PCStr(path),PCStr(mode),int timeout);
  FileSize file_copy(FILE *src,FILE *dst,FILE *cache,FileSize bytes,int *binary);
  
+ /*
  static int DATA_MAX = 10*1024*1024;
+ */
+ static FileSize _DATA_MAX = 10*1024*1024;
  static char *sftp_com = "sftp -oPort=%d %s@%s";
  static int pid;
  int SFTP_WITH_SEEK = 1;
***************
*** 610,615 ****
--- 613,626 ----
  	int ncc = 0;
  	int leng;
  
+ 	FileSize DATA_MAX = _DATA_MAX;
+ 	IStr(dmax,128);
+ 	int getMountOpt1(DGC*ctx,PCStr(onam),PVStr(oval),int size);
+ 	if( getMountOpt1(MainConn(),"datamax",AVStr(dmax),sizeof(dmax)) ){
+ 		DATA_MAX = kmxatoi(dmax);
+ 		sv1log("## datamax=%s 0x%llX\n",dmax,DATA_MAX);
+ 	}
+ 
  	DEBUG("--SFTPGW start\n");
  	fc = fdopen(gw,"r");
  	tc = fdopen(gw,"w");
*** dist/src/delegate9.9.1/src/httphead.c	Fri Dec  5 13:08:18 2008
--- ./src/httphead.c	Sun Feb  1 12:02:13 2009
***************
*** 123,128 ****
--- 123,134 ----
  	leng += HTTP_echoRequestHeader(Conn,tc);
  	return leng;
  }
+ int getMountOpt1(Connection *Conn,PCStr(onam),PVStr(oval),int size){
+ 	if( Conn && MountOptions ){
+ 		return getOpt1(MountOptions,onam,BVStr(oval));
+ 	}
+ 	return 0;
+ }
  int getMountExpires(Connection *Conn,PVStr(date),int size){
  	CStr(odate,64);
  	CStr(mexpire,32);

  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V