Article delegate-en/4044 of [1-5169] on the server localhost:119
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[Reference:<_A4042@delegate-en.ML_>]
Newsgroups: mail-lists.delegate-en

[DeleGate-En] Re: NNTP-Delegate will not cache in Linux
05 Aug 2008 03:07:44 GMT feedback@delegate.org (Yutaka Sato)
The DeleGate Project


Hi,

In message <_A4042@delegate-en.ML_> on 08/05/08(04:33:55)
you Jeff <pyqhqbdyi-ii4a7x4xikdr.ml@ml.delegate.org> wrote:
 |Ok I have tried what you said I should try. Testing using the "simple
 |method" does indeed write the article to the cache on the drive. However,
 |testing with a typical NNTP client seems to not have the same result. I

What the "typical NNTP client" is?
In your log output, I saw why DeleGate does not cache with the client.
It is because your client issues "BODY" command instead of "ARTICLE".
As long as I know, usual news readers issues the "ARTICLE" command to
retrieve an article and DeleGate does cache the response.
The ARTICLE command downloades the whole message of an article including
its header and body, while the BODY command downloads only its body.
It is very strange that a NNTP client does not see the header part of an
article which contains impotant meta information as subject or
content-type and so on.

 |I'm finding different results for using a typical NNTP download client
 |versus the telnet approach. I'm not sure if that is relevant - BUT I want it
 |to work with all clients.

Maybe your "NNTP download client" did "ARTICLE" or "HEAD" already to
get the header and did cached in itself.

 |08/04 21:05:24.31 [11279] 1+0: ## C-S body <65211$4895b8f2$5b79aabb$
 |9255@news.usenext.de>^M
...
 |When I download an article using the telnet method, I get the following in
 |the logs. Why do I not see this when downloading using a client?
 |
 |08/04 21:24:01.67 [11870] 3+0/1: ## C-S ARTICLE 300000000^M
...
 |When downloading a second time using the same method, I see this line in the
 |log:
 |08/04 21:29:16.79 [12000] 4+0/5: reuse [396454 bytes][age=297]
...
 |I assume this means it was used from the cache?

Yes.  You will see a similar result of "NNTP download client" if you
use "BODY" instread of "ARTICLE" in your manual testing.

I made an enclosed patch as a workaround to cope with "BODY only client".
With the patch, when DeleGate received "BODY" request from a client, it 
send "ARTICLE" to the server, then caches the response and returns its
body part.  On the first BODY request for an article, it puts messages
into the LOGFILE as follows.

08/05 12:01:28.95 [16738] 1+0: ##BA send ARTICLE for BODY <message-id>
08/05 12:01:28.99 [16738] 1+0/2: ##BA resp <<< 220 0 <message-id> ARTICLE^M
08/05 12:01:28.99 [16738] 1+0/2: ##BA resp >>> 222 0 <message-id> BODY^M
08/05 12:01:28.99 [16738] 1+0/2: openCACHE: create ...
08/05 12:01:28.99 [16738] 1+0/2: #### article spool wrote [1215 bytes] ...

Cheers,
Yutaka
--
  9 9   Yutaka Sato <y.sato@delegate.org> http://delegate.org/y.sato/
 ( ~ )  National Institute of Advanced Industrial Science and Technology
_<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
Do the more with the less -- B. Fuller

*** dist/src/delegate9.8.4-pre3/src/nntp.c	Mon Nov  5 17:11:16 2007
--- ./src/nntp.c	Tue Aug  5 11:46:50 2008
***************
*** 3625,3630 ****
--- 3625,3632 ----
   fprintf(stderr,"####RESP %s",line);
  */
  	/* should be after rewriting of GROUP response */
+ 	if( code == 220/*ARTICLE*/ && strcaseeq(com,"BODY") ){
+ 	}else
  	if( fputs(line,tc) == EOF )
  		return -1;
  
***************
*** 3675,3680 ****
--- 3677,3695 ----
  setArticleMasks(ns->ns_rewaddr,ns->ns_host,ns->ns_port,curGroup,curAnum);
  MIME_disableTransHead = 1;
  
+ 	if( code == 220/*ARTICLE*/ && strcaseeq(com,"BODY") && cache ){
+ 		sv1log("##BA resp <<< %s",line);
+ 		setVStrElem(line,2,'2'); /* 220 ARTICLE -> 222 BODY */
+ 		strsubst(AVStr(line)," ARTICLE\r\n"," BODY\r\n");
+ 		sv1log("##BA resp >>> %s",line);
+ 		fputs(line,tc);
+ 		RFC821_skipbody(fs,cache,AVStr(line),sizeof(line));
+ 		fflush(cache);
+ 		fseek(cache,0,0);
+ 		RFC821_skipheader(cache,NULL,NULL);
+ 		decodeBODY(cache,tc,filter,NULL,NULL,enHTML);
+ 		fputs(line,tc); /* EOR */
+ 	}else
  	switch( decoding ){
  	case D_THRU: thruRESP(fs,tc); break;
  	case D_HEAD:
***************
*** 4866,4871 ****
--- 4881,4892 ----
  		}
  
  		enqRequest(serverx,curGroup,ReqBuf,com,arg,NULL);
+ 		if( strcaseeq(com,"BODY") ){
+ 			if( cachedir() && !(ns->ns_nocache & CACHE_ARTICLE) ){
+ 				sv1log("##BA send ARTICLE for BODY %s\n",arg);
+ 				sprintf(ReqBuf,"ARTICLE %s\r\n",arg);
+ 			}
+ 		}
  
  		if( 0 < nserv ){
  			CStr(hostport,MaxHostNameLen);

  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V