Last login: Mon Aug 3 03:54:46 1998 from polysynaptic.iq. Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 2.2.6-STABLE (SUB) #1: Sat Apr 4 01:34:14 EST 1998 You have mail. proff@suburbia:/data/home/proff[]$ cd ~www proff@suburbia:/usr/local/www[]$ ls cgi-bin data proff@suburbia:/usr/local/www[]$ cd data/nntpcache proff@suburbia:/usr/local/www/data/nntpcache[]$ ls FAQ.html index.html news.html HTML neil.jpg nntpcache.jpg proff@suburbia:/usr/local/www/data/nntpcache[]$ cat FAQ.html The official

                     _                        _
         _ __  _ __ | |_ _ __   ___ __ _  ___| |__   ___
        | '_ \| '_ \| __| '_ \ / __/ _` |/ __| '_ \ / _ \
        | | | | | | | |_| |_) | (_| (_| | (__| | | |  __/
        |_| |_|_| |_|\__| .__/ \___\__,_|\___|_| |_|\___|
                        |_|
                          __
                         / _| __ _  __ _
                        | |_ / _` |/ _` |
                        |  _| (_| | (_| |
                        |_|  \__,_|\__, |
                                      |_|

NNTPCACHE FREQUENTLY ASKED QUESTIONS AND ANSWERS

Peter Tonoli <anarchie@suburbia.net> This is the list of frequently asked questions, and the answers to those questions for nntpcache, the caching NNRP server daemon.

Index

Section 1. Introduction and General Information
    Q1.1 What is nntpcache?
    Q1.2 What operating systems does nntpcache support?
    Q1.3 Where can I find nntpcache? / Where is the latest version?
    Q1.4 Can I use nntpcache freely or do I have to license it?

Section 2. Frequently encountered problems and solutions
    Q2.1 My system chokes on all that damned mmap()ing?!. What can I do?
    Q2.2 Netscape-news kills itself when reading news from nntpcache-0.88.3!
    Q2.3 Expiring with nntpcached -e doesn't work
    Q2.4 Help! The size of our cache keeps growing! What should I do about expiration?
    Q2.5 nntpcache spits out : 'post failed From: address not in Internet syntax'. What does that mean?
    Q2.6 Why does nntpcache-update take so long to load at initialisation?

Section 3. Support and Resources
    Q3.1 Is there a nntpcache mailinglist?
    Q3.2 How do I unsubscribe from the mailinglist?
    Q3.3 Is there an archive of the nntpcache mailinglist?
    Q3.4 Is there an official nntpcache home page?
    Q3.5 Who should I contact if I can't get nntpcache to work / I've found a bug in nntpcache, what should I do?

Section 4. Configuration of nntpcache
    Q4.1 What are all those entries in the servers configuration file?
    Q4.2 How do I specify the "Xref:full" part in the "overviewFmt" line in the config file?
    Q4.3 I've entered a server but nntpcache doesn't seem to want to know about it.
    Q4.4 I want to set the nntpcache-virtual-server-size record, what are some public news-servers I can add to my servers file?

Question 1.1. What is nntpcache?

Nntpcache (efficiently) executes on the localhost pretending to be an NNRP news reading server. In fact, what it does is pass certain NNTP commands through to real (remote and possibly local) news-servers based on various pattern matching rules. nntpcache then takes the output from those servers and caches & indexes it in funky ways (much specific case magic goes into this). The next time such information is asked for, or other information which can be logically inferred from the previously collated information, it is sent directly from the cache, without consulting the remote servers. It provides multi-server group merging, and sophisticated weighted content-filtering, access control and a chrooted environment. It makes heavy use of shared memory, copy-on-write, fd caches, mmaping and more for efficiency capable of serving thousands of simultaneous users.

To sum that up in layman's terms : nntpcache is a cache, similar to the Squid object-cache, but nntpcache works on newsgroups. By acting as a cache, nntpcache transparently and efficiently saves bandwidth enough to feed a small nation. It can also merge dozens of news servers into one large "virtual" news server, and provide content filtering (censoring! censoring!). It's a miserly beast with cpu and memory due to some gee-whizz trickery.

Question 1.2. What operating systems does nntpcache support?

* BSD
FreeBSD 2.2-CURRENT + GCC2.6.3
BSDI [something]

* HP-UX
HP-UX 9.05 + HP ANSI C

If you find a OS/compiler/library tuple that nntpcache appears to compile and run under, then don't keep it to yourself, please let tell the devteam at nntpcache@nntpcache.org! (this list is.. err dated - nntpcache works on nearly all modern unixes)

Question 1.3. Where can I find nntpcache? / Where is the latest version?

ftp://ftp.nntpcache.org/pub/nntpcache/nntpcache.tar.gz

The UK, Europe and US mirror of nntpcache can be accessed at: ftp://planete.theplanet.net/pub/nntpcache/nntpcache.tar.gz

The Norwegian and European mirror is at : ftp://ftp.nethelp.no/pub/news/nntpcache-1.0.7.1.tar.gz

FreeBSD 2.2.x and above has a port/package available in the ports system in news/nntpcache

Rumors on the wire suggest a Debian package also, but don't oink on it.

Question 1.4. Can I use nntpcache freely or do I have to license?

Probably. It depends on who you are. See the LICENSING file for full details, but in a nutshell, individuals, non-military, non-profit research groups, non-profit organisations (including educational institutions) and medical or educational government departments may use nntpcache freely. Everyone else (that is commercial organisations and government) are required to license in order to fund on-going development and support.

Question 2.1. My system chokes on all that damned mmap()ing?!. What can I do?

b) get a decent OS. Ultrix mmap is totally broken!

Question 2.2. Netscape-news kills itself when reading news from nntpcache-0.88.3!

We know. Didn't get a bugs-bounty either. Geesh. nntpcache-0.90+ formats its data in a way that makes netscape happier. But it IS netscape's fault for expecting nnrpd style response messages.

Question 2.3. Expiring with nntpcached -e doesn't work

Running nntpcached -e on a system that is already running nntpcache doesn't work. To expire on a system that is already running nntpcache, send the nntpcache process the USR2 signal.

Question 2.4. Help! The size of our cache keeps growing! What should I do about expiration?

# expire if there are under this number of blocks in the cache

# partition

minBlocksFree 1000

# run expire if there are under this number of inodes in the cache

# partition

minFilesFree 1000

# don't let the history file grow larger than this (bytes) hisHighWater 60M

# when it does, trim it back to this (bytes)

hisLowWater 40M

# When we do an expire, kill articles older than this

maxArtAge 2w

# check disk space, etc for automatic expiry this often

# (but only after a client connects or disconnects)

expireCheckPeriod 5m

minBlocksFree

minFilesFree

Question 2.5. nntpcache spits out : 'post failed From: address not in Internet syntax'. What does that mean?

That error message translated into normal English means: Your attempt to post failed because the address in the "From:" field was not in the format user@host. To solve the problem, ensure you have a correct reply-to address configured in the newsreader.

Question 2.6. Why does nntpcache-update take so long to load at initialisation?

nntpcache update takes a while on master startup since it loads overview.fmt, active, newsgroups and active.times lists from the providing server(s). these lists typically run into several mega-bytes, and nntpcache isn't able to serve clients till it has its hot little hands on this vital information.

Question 3.1. Is there a nntpcache mailinglist?

nntpcache-users-request@nntpcache.org

Question 3.2. How do I unsubscribe from the mailinglist?

nntpcache-users-request@nntpcache.org

Question 3.3. Is there an archive of the nntpcache mailinglist?

ftp://ftp.nntpcache.org/pub/nntpcache/mailinglist

A HTML version of the mailinglist archive is available at: http://www.support.nl/online/nntpquery.html

Question 3.4. Is there an official nntpcache home page?

http://www.nntpcache.org/

Question 3.5. Who should I contact if I can't get nntpcache to work / I've found a bug in nntpcache, what should I do?

nntpcache@nntpcache.org

Question 4.1. What are all those entries in the servers configuration file?

* Active
the active file

* Act.tim
active.times

* Newsgrp
newsgroups

* Group
What you get if you / a client types 'group some.newsgroup'

* Xover
What you get if you / a client types 'xover low-high'

* Arts
What you get if you / a client types 'head/body/article no'

Question 4.2. How do I specify the "Xref:full" part in the "overviewFmt" line in the config file?

You shouldn't have to specify the overviewFmt thing if the remote server supports 'LIST overview.fmt'. But if you insist, just 'Xref' will do.

Question 4.3. I've entered a server but nntpcache doesn't seem to want to know about it.

For example:

news.mynet.net DEFAULT 4h 2d 2d 30m 60d 60d

news.mynet.net. DEFAULT 4h 2d 2d 30m 60d 60d

Question 4.4. I want to set the nntpcache-virtual-server-size record, what are some public news-servers I can add to my servers file?

If anyone can add to or has corrections for this list, please let me know. Some entries are commented because I haven't had time to specifically add all groups they carry or because they weren't responding.

newsgroups.intel.com. DEFAULT 24h 4d 4d 30m 60d 60d

msnews.microsoft.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.3dfx.com. DEFAULT 24h 4d 4d 30m 60d 60d

miriam.fuller.edu. DEFAULT 24h 4d 4d 30m 60d 60d

adesknews.autodesk.com. DEFAULT 24h 4d 4d 30m 60d 60d

forums.borland.com. DEFAULT 24h 4d 4d 30m 60d 60d

hope.harvard.net. DEFAULT 24h 4d 4d 30m 60d 60d

cnews.corel.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.avault.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.linkexchange.com. DEFAULT 24h 4d 4d 30m 60d 60d

#forums.macromedia.com. DEFAULT 24h 4d 4d 30m 60d 60d

#news.activeie.com. DEFAULT 24h 4d 4d 30m 60d 60d

#news.interaktiv-media.se. DEFAULT 24h 4d 4d 30m 60d 60d

publicnews.msn.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.netobjects.com. DEFAULT 24h 4d 4d 30m 60d 60d

secnews.netscape.com. DEFAULT 24h 4d 4d 30m 60d 60d

beetle.privatei.com. DEFAULT 24h 4d 4d 30m 60d 60d

massena.com. DEFAULT 24h 4d 4d 30m 60d 60d

cor.newman.upenn.edu. DEFAULT 24h 4d 4d 30m 60d 60d

forums.powersoft.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.sff.net. DEFAULT 24h 4d 4d 30m 60d 60d

nntp.net-link.net. DEFAULT 24h 4d 4d 30m 60d 60d

service.symantec.com. DEFAULT 24h 4d 4d 30m 60d 60d

#jazz.trumpet.com.au. DEFAULT 24h 4d 4d 30m 60d 60d

news.turbopower.com. DEFAULT 24h 4d 4d 30m 60d 60d

news.ksc.nasa.gov. DEFAULT 24h 4d 4d 30m 60d 60d

news.glbs.com. DEFAULT 24h 4d 4d 30m 60d 60d

%BeginGroups

# Group pattern Host

* your.main.server.here

alt.oui.support* beetle.privatei.com.

asp.* news.glbs.com.

audioweb.* news.glbs.com.

3dfx.* news.3dfx.com.

autodesk.* adesknews.autodesk.com.

avault.* news.avault.com.

borland.* forums.borland.com.

cakewalk.* hope.harvard.net.

corel.* cnews.corel.com.

crescent.* news.glbs.com.

db.* news.glbs.com.

forum.* news.avault.com.

glbs.* news.glbs.com.

intel.* newsgroups.intel.com.

le.* news.linkexchange.com.

linux.* miriam.fuller.edu.

microsoft.* msnews.microsoft.com.

msn.* publicnews.msn.com.

net-link.* nntp.net-link.net.

netobjects.* news.netobjects.com.

netscape.* secnews.netscape.com.

ntworthy* news.glbs.com.

people.* secnews.netscape.com.

pilot.* massena.com.

powerbasic.* news.glbs.com.

powersoft.* forums.powersoft.com.

sff.* news.sff.net.

sybase.* forums.powersoft.com.

symantec.* service.symantec.com.

#trumpet.* jazz.trumpet.com.au.

turbopower.* news.turbopower.com.

woll2woll* news.glbs.com.

mail.winvn news.ksc.nasa.gov.

perl.porters-gw cor.newman.upenn.edu.

Acknowledgements:

alan@manawatu.gen.nz

herbert@greathan.apana.org.au

proff@nntpcache.org

cyberw4@cyberworldcafe.com