The Frustrated Programmer

At work we have a lot of issues with monitoring and logging. We run a lot of different services as EJBs in Weblogic clusters. They get a lot of use, so much so, that logging can actually make a real performance problem. The main issues are:

• The logs are not aggregated in any one place.
• There are a lot of ad-hoc scripts to do remote greps via ssh.
• Any type of central aggregation will cause increased network load and increased CPU load on the servers.

While I was thinking about this, I remembered IBM used to have a HTTP log analyzer that, instead of pulling log files like most do, listened on a promiscuous ethernet device and built the logs itself, in real time. These logs could be analyzed and poked through at any time, without fear of bogging down the servers.

I'd like something like this for Java Applications. Something that could listen for known protocols like RMI, SOAP-RMI, etc. It could do performance montoring (avg., min, max, std. dev.) and also track error/exception rates. In my head these seems very possible, but maybe I'm missing something.

There are several products that use byte-code modification to instrument code, but this adds unnessesary overhead. I was quoted something like a 7% increase, and its still not doing the other half I need -- error analysis.

For the first 6 months of this blogs life, the browser usage of my visitors looked something like this:

IEs: 15%
Mozillas 79%
Others: 6%

Now that things have slowed down, their browser usings looks like this:

IEs: 42%
Mozillas 52%
Others: 6%

What is happening to Mozilla's market share? Nothing really. My analysis goes like this; When I used to post several times a week, my visitors were mainly repeat visitors or ones that came through aggregators like Erik's link blog. They were people in the know, proactive types. Now, I get mostly people searching for struts issues, with queries like "struts multibox bug". These people are reacting to their environment, rather than searching it out ahead of time.

In related news browser usages statisics, along with my crude excel skills, show Mozilla will overtake IE in less than 12 months.