Article delegate-en/4055 of [1-5169] on the server localhost:119
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[Reference:<_A4049@delegate-en.ML_>]
Newsgroups: mail-lists.delegate-en

[DeleGate-En] Re: NNTP-Delegate will not cache in Linux
07 Aug 2008 09:50:16 GMT Jeff <pyqhqbdyi-c4ifwxovy25r.ml@ml.delegate.org>


Hi Yutaka,

I have been testing the last patch you sent, it seems to work acceptably but
I have noticed an interesting side effect.

First off, let me explain briefly how I am testing. My setup has an
NNTP-delegate proxy serving the content from a Newsfeed provider. My local
cache (Linux server doing the caching) connects to this main proxy.

Newsfeed provider -> Remote-Proxy -> {traffic shaper} -> Local-Cache ->
Requesting-User

The side effect that I am seeing is that the Local-Cache waits for the
article to download completely to the cache before sending it on to the
Requesting-User. So a traffic graph on the user side on the local network
will see traffic spikes. Normally, the data should be "teed" or split so it
is simultaneously written to cache and sent over the network. This seems to
not be happening. Everything does work fine though. I only noticed this when
doing traffic shaping on the traffic from Remote-Proxy to Local-Cache - in
that scenario, the effect is even more pronounced as there are 1-2 minute
gaps between these spikes. A constant traffic flow would be a little better.


Interestingly, if I connect as a user directly to the Remote-Proxy with the
shaping on, I see a constant traffic flow so the "problem" is definitely
limited to the Local-Cache..

Any thoughts as to why this is happening? It seems to only happen on the
local cache, the remote proxy (BOTH are running the same latest version of
delegate) runs normally with constant traffic flow.

Regards,
Jeff

On Tue, Aug 5, 2008 at 12:09 PM, Yutaka Sato <feedback@delegate.org> wrote:

> Hi,
>
> In message <_A4047@delegate-en.ML_> on
> 08/05/08(17:45:31)
> you Jeff <pyqhqbdyi-c4ifwxovy25r.ml@ml.delegate.org> wrote:
>  |I have tested after making the patch you suggested for the ARTICLE/BODY
>  |problem and it works perfectly. The articles are now caching and
> everything
>  |is working great. Thanks again for your time and help, I really
> appreciate
>  |it. Will it be safe to assume these changes will be incorporated in your
>  |main releases or will I need to use my patched versions from here
> onwards?
>
> The modifications will be included in the next release (9.8.4-pre4).
>
>  |Regarding the "other NNTP clients", I was referring to the software that
>  |users on my network are using to download via NNTP namely Newsleecher,
>  |Grabit, Alt.binz etc
>
> I see.
>
>  |Lastly, some users on my network require RADIUS authentication (supported
> by
>  |my feed provider). For this to work properly, I need to be able to insert
> a
>  |domain identifier so that the origin server forwards RADIUS packets
>  |correctly. Essentially, all that delegate needs to do is modify the
> AUTHINFO
>  |command as follows:
>  |AUTHINFO USER username -> AUTHINFO USER username@domain
>  |Since the @domain part is static, it can be inserted by delegate using
>  |strcat(user,"@domain") on each request
>  |
>  |I have tried to patch it myself with little success :( Perhaps it could
> be
>  |added as a parameter i.e. RADIUS-DOMAIN=@domainname - although I would
> not
>  |mind hard coding it into the source code if needed since this is a
> special
>  |one-time request.
>  |
>  |I don't want to use more of your time but if you could maybe assist me
> with
>  |some patch code as before, that would be very useful.
>
> Note that users except commercial licensee have no right to make their
> own modification to DeleGate.  (and you are not one as long as I know :).
>
> The enclosed patch is a tentative code for rewriting the NNTP command
> using a MOUNT parameter.  It is used as this:
>
>  MOUNT="command:AUTHINFO/USER*%S data:;,AUTHINFO/USER*%S@domain nocase"
>
> You will see the message about the rewriting in the LOGFILE as this:
>
>  1+0: *** command:authinfo/user%20useri =>
> data:;,AUTHINFO/USER%20useri@domain ***
>  1+0: ##CM [AUTHINFO][USER useri]->[AUTHINFO][USER useri@domain]
>
> Cheers,
> Yutaka
> --
>  9 9   Yutaka Sato <y.sato@delegate.org> http://delegate.org/y.sato/
>  ( ~ )  National Institute of Advanced Industrial Science and Technology
> _<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
> Do the more with the less -- B. Fuller
>
> *** dist/src/delegate9.8.4-pre3/src/nntp.c      Mon Nov  5 17:11:16 2007
> --- nntp.c      Tue Aug  5 18:57:00 2008
> ***************
> *** 44,49 ****
> --- 44,81 ----
>  #include "file.h"
>  #include "auth.h"
>
> + int commandMount(Connection *Conn,PVStr(req),PVStr(com),PVStr(arg)){
> +       const char *opts;
> +       IStr(xreq,1024);
> +       IStr(crlf,8);
> +       refQStr(xp,xreq);
> +       IStr(xcom,128);
> +       IStr(xarg,1024);
> +
> +       sprintf(xreq,"command:%s/%s",com,arg);
> +       if( xp = strpbrk(xreq,"\r\n") ){
> +               strcpy(crlf,xp);
> +               clearVStr(xp);
> +       }
> +
> +       strsubst(AVStr(xreq)," ","%20");
> +       opts = CTX_mount_url_to(Conn,"",com,AVStr(xreq));
> +       if( opts == 0 )
> +               return 0;
> +
> +       if( strneq(xreq,"data:;,",7) ){
> +               strsubst(AVStr(xreq),"%20"," ");
> +               Xsscanf(xreq+7,"%[^/]/%[^\r\n]",AVStr(xcom),AVStr(xarg));
> +               if( xcom[0] ){
> +                       sv1log("##CM
> [%s][%s]->[%s][%s]\n",com,arg,xcom,xarg);
> +                       strcpy(com,xcom);
> +                       strcpy(arg,xarg);
> +                       sprintf(req,"%s %s%s",xcom,xarg,crlf);
> +                       return 1;
> +               }
> +       }
> +       return 0;
> + }
>  static void RecvMssg(FILE *fs,FILE *afp,int withEOR){
>        recvMssg(fs,afp,withEOR);
>  }
> ***************
> *** 4478,4483 ****
> --- 4510,4518 ----
>                if( dp = wordScan(ReqBuf,com) )
>                        lineScan(dp,arg);
>                else    arg[0] = 0;
> +               if( 1 ){
> +
> commandMount(Conn,AVStr(ReqBuf),AVStr(com),AVStr(arg));
> +               }
>
>                if( !ImCC && nservers_active(1) == 1 )
>                if( PENALTY_SLEEP )
>


  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V