Lists | Search | News | Contact
Tuesday 9 February, 2010 @ 06:13 AM
List view - Thread view
Subject: 6.2-Release and Clamd 0.90 with libpthread.so
From: Martin Blapp
Date: 2007-02-20 16:49:46
Hi, After upgrading to clamd 0.90 I found that clamd doesn't work well with libpthread.so.2. Using libmap.conf entries for libthr.so.2 or libc_r.so.6 worked fine. With both thread libraries clamd works as it should. The effect with libpthread is that clamd works very very slow. It's only able to scan 1-2 files in 5 seconds instead of 400-500 in the same time with libthr/libc_r. A ktrace shows the following behaviour: clamd 0.90 and libpthread.so.2 1735 clamd RET fork 0 1735 clamd CALL kse_release(0xbe7eac20) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_release(0x9ea3f10) 1735 clamd RET kse_release 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x805df4c) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9eabedc) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x805dedc) 1735 clamd CALL kse_wakeup(0x9e4670c) 1735 clamd CALL kse_release(0x9eabedc) 1735 clamd RET kse_release 0 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_wakeup(0x9e4680c) 1735 clamd RET fork 0 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9ea7f10) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbef61c10) 1735 clamd CALL kse_release(0x805dedc) 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbec30f50) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbe8fbb90) 1735 clamd CALL kse_wakeup(0x9e4670c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9ea7f40) 1735 clamd CALL kse_wakeup(0x9e4680c) 1735 clamd RET kse_release 0 1735 clamd RET kse_release 0 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_release(0x9ea7ed0) 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd CALL kse_release(0xbef61bd0) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbee50bd0) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 After ~10000 lines which show all like this, clamd actually opens and scans something, then another ~10000 lines follow ... I guess it's a bug in libpthread, but it really doesn't work fine with this lib. Or is it bad programming in clamav ? Any comments ? Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-20 17:44:20
Hi, > Probably because it's fork()ing and trying to call > non-async-signal-safe functions without doing an exec()? The strange thing is that I can't see any any fork() calls with libc_r or libthr. How's that possible ? Or do you think that its stuck at the beginning ? But why does it work then for some degree ? Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-20 17:39:45
On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi, > > After upgrading to clamd 0.90 I found that clamd > doesn't work well with libpthread.so.2. > > Using libmap.conf entries for libthr.so.2 or libc_r.so.6 > worked fine. With both thread libraries clamd works as it > should. > > The effect with libpthread is that clamd works very very slow. > It's only able to scan 1-2 files in 5 seconds instead of > 400-500 in the same time with libthr/libc_r. > > A ktrace shows the following behaviour: > > clamd 0.90 and libpthread.so.2 > > 1735 clamd RET fork 0 Probably because it's fork()ing and trying to call non-async-signal-safe functions without doing an exec()? -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-20 17:46:33
On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi, > >> Probably because it's fork()ing and trying to call >> non-async-signal-safe functions without doing an exec()? > > The strange thing is that I can't see any any fork() calls > with libc_r or libthr. How's that possible ? Or do you think > that its stuck at the beginning ? But why does it work then > for some degree ? I dunno, maybe ktrace was confused and it wasn't a fork()? -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-20 19:15:39
Hi Daniel, Any idea how I can track this down ? It's rather easy to reproduce ... fork() is only used in the functions daemonize() and in virusaction(), which is used to call an external programm if one is defined (I don't have one). So fork() should only be called once during going to be a background daemon. Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ On Tue, 20 Feb 2007, Daniel Eischen wrote: > On Tue, 20 Feb 2007, Martin Blapp wrote: > >> >> Hi, >> >>> Probably because it's fork()ing and trying to call >>> non-async-signal-safe functions without doing an exec()? >> >> The strange thing is that I can't see any any fork() calls >> with libc_r or libthr. How's that possible ? Or do you think >> that its stuck at the beginning ? But why does it work then >> for some degree ? > > I dunno, maybe ktrace was confused and it wasn't a fork()? > > -- > DE > _______________________________________________ > freebsd-<email removed> mailing list > http://lists.freebsd.org/mailman/listin … sd-threads > To unsubscribe, send any mail to \ "freebsd-threads-<email removed>" > _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-20 19:21:02
On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi Daniel, > > Any idea how I can track this down ? Try commenting out the fork() in daemonize(). Also try setting kern.threads.virtual_cpu=1 if you have more than 1 cpu (this will limit it to 1 KSE). > It's rather easy to reproduce ... > > fork() is only used in the functions daemonize() and in > virusaction(), which is used to call an external programm > if one is defined (I don't have one). > > So fork() should only be called once during going to be > a background daemon. -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-20 23:18:34
Hi, > Also try setting kern.threads.virtual_cpu=1 if you have > more than 1 cpu (this will limit it to 1 KSE). Our boxes have 2 CPUs and HTT is activated. Interesting. With kern.threads.virtual_cpu=1 it works also in the libpthread case, but while there are still some fork() calls it works better. With kern.threads.virtual_cpu=2 it's slower. And With kern.threads.virtual_cpu=4 its really slow. If I run clamd in foreground mode everything works as it should. More interesting, the CPU load drops significantly (500-1000%) ! Something is seriosly broken here. Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-20 23:29:17
On Tue, 20 Feb 2007, Martin Blapp wrote: > Hi, > >> Also try setting kern.threads.virtual_cpu=1 if you have >> more than 1 cpu (this will limit it to 1 KSE). > > Our boxes have 2 CPUs and HTT is activated. > > Interesting. With kern.threads.virtual_cpu=1 it works also > in the libpthread case, but while there are still some fork() calls > it works better. With kern.threads.virtual_cpu=2 it's slower. And > With kern.threads.virtual_cpu=4 its really slow. > > If I run clamd in foreground mode everything works as it should. > More interesting, the CPU load drops significantly (500-1000%) ! > > Something is seriosly broken here. Running in foreground with or without kern.threads.virtual_cpu adjusted? If running in foreground works regardless of kern.threads.virtual_cpu (there will still be some inefficiencies here because of the scheduler and KSE overhead/contention), then it is probably due to the application doing a fork() from a threaded application without doing an exec(). This is prohibited by POSIX and libpthread makes no attempt to allow it. -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-20 23:39:22
Hi, > If running in foreground works regardless of kern.threads.virtual_cpu > (there will still be some inefficiencies here because of the > scheduler and KSE overhead/contention), then it is probably due > to the application doing a fork() from a threaded application > without doing an exec(). This is prohibited by POSIX and > libpthread makes no attempt to allow it. Yes it works regardless of kern.threads.virtual_cpu. Yes, fork() is called without exec(). So what's the proper way ? Start clamd in foreground mode with daemon /usr/local/sbin/clamd And redirect stderr ? Or can this be solved differently ? Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-20 23:48:23
Hi, Useing daemon(3) instead of fork seems to fix the problem. Thank you very much ! Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ On Tue, 20 Feb 2007, Daniel Eischen wrote: > On Tue, 20 Feb 2007, Martin Blapp wrote: > >> Hi, >> >>> Also try setting kern.threads.virtual_cpu=1 if you have >>> more than 1 cpu (this will limit it to 1 KSE). >> >> Our boxes have 2 CPUs and HTT is activated. >> >> Interesting. With kern.threads.virtual_cpu=1 it works also >> in the libpthread case, but while there are still some fork() calls >> it works better. With kern.threads.virtual_cpu=2 it's slower. And >> With kern.threads.virtual_cpu=4 its really slow. >> >> If I run clamd in foreground mode everything works as it should. >> More interesting, the CPU load drops significantly (500-1000%) ! >> >> Something is seriosly broken here. > > Running in foreground with or without kern.threads.virtual_cpu > adjusted? > > If running in foreground works regardless of kern.threads.virtual_cpu > (there will still be some inefficiencies here because of the > scheduler and KSE overhead/contention), then it is probably due > to the application doing a fork() from a threaded application > without doing an exec(). This is prohibited by POSIX and > libpthread makes no attempt to allow it. > > -- > DE > _______________________________________________ > freebsd-<email removed> mailing list > http://lists.freebsd.org/mailman/listin … sd-threads > To unsubscribe, send any mail to \ "freebsd-threads-<email removed>" > _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-21 00:09:16
Sigh, > > Useing daemon(3) instead of fork seems to fix the problem. Of course deamon(3) calls fork, and after a short time I had again the same ktrace output and CPU usage. Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-21 02:05:35
> Sigh, > >> >> Useing daemon(3) instead of fork seems to fix the problem. > > Of course deamon(3) calls fork, and after a short time I > had again the same ktrace output and CPU usage. Using daemon(3) works some times, but from time to time I see then again the old behaviour, looping kse calls and some forks(). I wonder why those calls never happen in the libthr and libc_r cases. Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-21 06:03:10
On Wed, 21 Feb 2007, Martin Blapp wrote: >> Sigh, >> >>> >>> Useing daemon(3) instead of fork seems to fix the problem. >> >> Of course deamon(3) calls fork, and after a short time I >> had again the same ktrace output and CPU usage. > > Using daemon(3) works some times, but from time to time > I see then again the old behaviour, looping kse calls > and some forks(). > > I wonder why those calls never happen in the libthr > and libc_r cases. libc_r and libthr don't have to kse initialization and have different locking. Regardless, I think the code (clamd) is wrong. There is no guarantee that fork() from threaded application will work for libthr and libc_r either. If clamd needs to daemonize, it can always exec() itself with an added argument that means "I've already fork()'d, don't fork() again". -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-24 11:30:09
Hi, After some busy days I found again time to work on this. I still have no clue what could be responsable for the problems I see. ktrace still shows lot of fork()s, but I can't see where whey are called. Running ktrace with libc_r or libthr I can't see a single call to fork(). Is ktrace lying to me ? Why do I only see fork()s with libpthread ? Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-24 16:13:54
On Sat, 24 Feb 2007, Martin Blapp wrote: > > Hi, > > After some busy days I found again time to work on this. I still have no clue > what could be responsable for the problems I see. > > ktrace still shows lot of fork()s, but I can't see where whey are called. ktrace is probably lying to you if the code is really only fork()ing once. > Running ktrace with libc_r or libthr I can't see a single call to fork(). > Is ktrace lying to me ? Why do I only see fork()s with libpthread ? The kse upcalls might look like forks to ktrace? -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-24 19:06:32
Hi, >> Running ktrace with libc_r or libthr I can't see a single call to fork(). >> Is ktrace lying to me ? Why do I only see fork()s with libpthread ? > > The kse upcalls might look like forks to ktrace? > That could be. Is it easy to fix this for ktrace ? Anyway, that still doesn't solve the problem for me, since clamd with libpthreads is extremly CPU hungry and is slow. I really miss something to profile like strace on solaris to see where it spends most CPU cycles. The problem is not that it doesn't work, the problem is that it works far too slow. Do you have any ideas ? Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Daniel Eischen
Date: 2007-02-24 19:47:43
On Sat, 24 Feb 2007, Martin Blapp wrote: > > Hi, > >>> Running ktrace with libc_r or libthr I can't see a single call to \ fork(). >>> Is ktrace lying to me ? Why do I only see fork()s with libpthread ? >> >> The kse upcalls might look like forks to ktrace? >> > > That could be. Is it easy to fix this for ktrace ? I don't know... > Anyway, that still doesn't solve the problem for me, since > clamd with libpthreads is extremly CPU hungry and is slow. > > I really miss something to profile like strace on solaris to > see where it spends most CPU cycles. The problem is not that > it doesn't work, the problem is that it works far too slow. > > Do you have any ideas ? Yes, fix clamd so it doesn't fork() without doing an exec(). POSIX says that behavior is undefined when threaded applications fork() and call non-async-signal-safe functions. -- DE _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-24 22:52:02
I already did that. With libc_r and libthr I don't see a single fork() here. > Yes, fix clamd so it doesn't fork() without doing an exec(). > POSIX says that behavior is undefined when threaded applications > fork() and call non-async-signal-safe functions. _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-24 23:24:42
Let me make a summary: With kern.threads.virtual_cpu=1 clamd works best with libpthread. It uses almost no CPU. With kern.threads.virtual_cpu=2 or kern.threads.virtual_cpu=4 and growing scanner load, the CPU load grows exponentially. The initial fork() seems not to be the problem. It happens in the foreground mode (without any fork call) too, but only if there is some amount of scanner load. Setting maxthreads in clamd.conf seems to make the situation even worse. libthr and libc_r work without any problems and ktrace here doesn't show any fork() calls at all. I wonder if something else could get libpthread spinning on kse activation stuff. I still think something is wrong here. Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-25 16:04:40
Hi all, I'm including now clamav-<email removed> too, since it's not clear if this problem is clamd or system library related. The problem is that clamd works only well with FreeBSDs libthr 1:1 kernel threading library. Using libc_r (userland threading only) seems to consume already a bit more resources. But clamd together with the N:M kernel threads library (libpthreads) is really slow. For some strange reason setting the sysctl kern.threads.virtual_cpu=1 to allow only one CPU for clamd works well and takes the CPU load significantly down. After reading http://www.gossamer-threads.com/lists/c … sers/28795 especially this part: [...] > >We've been running 6.x our scanner boxes for a while now, but it's only >been with the more recent security/clamav-devel port installs that we >noticed a problem much like this. Most connections to the daemon (made >through clamav-milter in our case) timed out, and the only way to bring >down the daemon was with a kill -9. > >For us, the 20061029 devel snapshot was fine, but the current one >(20061217) has problems. > [...] I've now tried to go back to the development snapshot 20061129 and so I've reversed the clamd threading patches up to this date. But no luck. First it works well, then it becomes suddenly slower and more and more connections time out. Also, only 'kill -9' kills clamd. This isn't the case with lc_r and libthr btw ... Could it be that libpthread from it's kind of nature is a lot slower than libthr in that special case ? Maybe clamd expects system scope thread semantics and so libpthreads is slower ... Of course our current workload could then be too much for the poor clamd and a even more growing workload may slow it down due to resource bottlenecks. -- Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Martin Blapp
Date: 2007-02-26 08:57:34
Hi, > > Have you tried using pmc? Works at full processor speed... > Not yet since the pcm wiki 1) tells me Profiling threaded code: Currently buggy; Allows you to profile threaded \ applications. 1) http://wiki.freebsd.org/PmcTools/Features -- Martin _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: John-Mark Gurney
Date: 2007-02-26 08:40:37
Martin Blapp wrote this message on Sat, Feb 24, 2007 at 19:06 +0100:
> >>Running ktrace with libc_r or libthr I can't see a single call to \
fork().
> >>Is ktrace lying to me ? Why do I only see fork()s with libpthread ?
> >
> >The kse upcalls might look like forks to ktrace?
> >
>
> That could be. Is it easy to fix this for ktrace ?
>
> Anyway, that still doesn't solve the problem for me, since
> clamd with libpthreads is extremly CPU hungry and is slow.
>
> I really miss something to profile like strace on solaris to
> see where it spends most CPU cycles. The problem is not that
> it doesn't work, the problem is that it works far too slow.
Have you tried using pmc? Works at full processor speed...
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
_______________________________________________
freebsd-<email removed> mailing list
http://lists.freebsd.org/mailman/listin … sd-threads
To unsubscribe, send any mail to "freebsd-threads-<email removed>"
From: Julian Elischer
Date: 2007-02-26 20:42:13
Martin Blapp wrote: > > Hi, > > After some busy days I found again time to work on this. I still have no > clue > what could be responsable for the problems I see. > > ktrace still shows lot of fork()s, but I can't see where whey are called. > > Running ktrace with libc_r or libthr I can't see a single call to fork(). > Is ktrace lying to me ? Why do I only see fork()s with libpthread ? 'upcalls' from the KSE threading library are erronwously reported as returns \ from fork they show RET fork 0 > > Martin > > Martin Blapp, > ------------------------------------------------------------------ > ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH > Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 > PGP: > PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E > ------------------------------------------------------------------ > > _______________________________________________ > freebsd-<email removed> mailing list > http://lists.freebsd.org/mailman/listin … sd-threads > To unsubscribe, send any mail to \ "freebsd-threads-<email removed>" _______________________________________________ freebsd-<email removed> mailing list http://lists.freebsd.org/mailman/listin … sd-threads To unsubscribe, send any mail to "freebsd-threads-<email removed>"
www.FreeBSD.se | www.NetBSD.se| Disclaimer
Co-location and bandwidth generously provided by Phonera