<!-- --><!-- --><style type="text/css">@import url(https://www.blogger.com/static/v1/v-css/navbar/3334278262-classic.css); div.b-mobile {display:none;} </style> </head><body><script type="text/javascript"> function setAttributeOnload(object, attribute, val) { if(window.addEventListener) { window.addEventListener('load', function(){ object[attribute] = val; }, false); } else { window.attachEvent('onload', function(){ object[attribute] = val; }); } } </script> <div id="navbar-iframe-container"></div> <script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script> <script type="text/javascript"> gapi.load("gapi.iframes:gapi.iframes.style.bubble", function() { if (gapi.iframes && gapi.iframes.getContext) { gapi.iframes.getContext().openChild({ url: 'https://www.blogger.com/navbar.g?targetBlogID\0757256432\46blogName\75The+Frustrated+Programmer\46publishMode\75PUBLISH_MODE_BLOGSPOT\46navbarType\75BLACK\46layoutType\75CLASSIC\46searchRoot\75http://frustratedprogrammer.blogspot.com/search\46blogLocale\75en_US\46v\0752\46homepageUrl\75http://frustratedprogrammer.blogspot.com/\46vt\0755012862196962223429', where: document.getElementById("navbar-iframe-container"), id: "navbar-iframe" }); } }); </script>
| Sunday, January 29, 2012

I like to keep the data around from my Swimmovate PoolMate and entering the details into TrainingPeaks is very cumbersome. So I wrote a quick script to generate TCX files that can be uploaded. Its very simple:





#!/bin/bash

function do_lap() {
LAP_TIME=`expr $LAP_MINUTES \* 60`
LAP_TIME=`expr $LAP_TIME + $LAP_SECONDS`
echo " <Lap StartTime=\"$LAP_START\">" >> $START_DATE.tcx
echo " <TotalTimeSeconds>$LAP_TIME</TotalTimeSeconds>" >> $START_DATE.tcx
echo " <DistanceMeters>$LAP_DISTANCE</DistanceMeters>" >> $START_DATE.tcx
echo " <Intensity>$1</Intensity>" >> $START_DATE.tcx
echo " <TriggerMethod>Manual</TriggerMethod>" >> $START_DATE.tcx
echo " </Lap>" >> $START_DATE.tcx
MINUTE_OFFSET=`expr $MINUTE_OFFSET + $LAP_MINUTES`
SECOND_OFFSET=`expr $SECOND_OFFSET + $LAP_SECONDS`
}

START_DATE=`date +%Y-%m-%dT%H:%M:%S.000Z`
cat header.tcx > $START_DATE.tcx
echo " <Id>"$START_DATE"</Id>" >> $START_DATE.tcx
echo -n "Yards or Meters (y/m): "
read YARDS
echo -n "Number of sets: "
read SETS
CURRENT_SET=0
MINUTE_OFFSET=0
SECOND_OFFSET=0
while [ $CURRENT_SET -ne $SETS ]; do
echo -n "Set $CURRENT_SET Distance: "
read LAP_DISTANCE
if [ "$YARDS" = "y" ]; then
LAP_DISTANCE=`echo $LAP_DISTANCE \* 0.9144 | bc`
fi
echo -n "Set $CURRENT_SET Time (Min Sec): "
read LAP_MINUTES LAP_SECONDS
LAP_START=`date -v +${MINUTE_OFFSET}M -v +${SECOND_OFFSET}S +%Y-%m-%dT%H:%M:%S.000Z`
do_lap "Active"

#Default to 60 second reset between sets
LAP_MINUTES=0
LAP_SECONDS=60
LAP_DISTANCE=0
do_lap "Resting"

CURRENT_SET=`expr $CURRENT_SET + 1`
done
cat trailer.tcx >> $START_DATE.tcx



Here is header.tcx

<?xml version="1.0" encoding="UTF-8"?>
<TrainingCenterDatabase
xsi:schemaLocation="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2 http://www.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd"
xmlns:ns5="http://www.garmin.com/xmlschemas/ActivityGoals/v1"
xmlns:ns3="http://www.garmin.com/xmlschemas/ActivityExtension/v2"
xmlns:ns2="http://www.garmin.com/xmlschemas/UserProfile/v2"
xmlns="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ns4="http://www.garmin.com/xmlschemas/ProfileExtension/v1">
<Activities>
<Activity Sport="Other">


And trailer.tcx

</Activity>
</Activities>
</TrainingCenterDatabase>

| Tuesday, December 26, 2006

UPDATE: Added more diagnostics to the code. If you were having issues, run it again and post the output to in a comment.
UPDATE2: Changed the second regular expression to be a little more loose. Hopefully this will resolve some of the problems.
UPDATE3: Blogger was removing some code that it thought was HTML causing an endless loop in the script. If you tried the script and it was just hanging, try it again.
UPDATE4: Fixed two things: One - Napster 4 changed its image naming conventions (Thanks Don). Two - This program never handled multi-byte characters well. Its somewhat hacked to fix this.

So, Napster isn't so good at album art for some reason -- no trace of it anywhere in my library. Funny thing is, Napster has its own stash of all the covers in C:\Documents and Settings\All Users\Application Data\Napster\image. Unfortunately its stored by some crazy obfuscated name like 12303503.jpg. Good news is that the Napster database, db30.bin, has the nessesary glue to piece things together. I hacked together some Perl to find my missing artwork and copy it to the album's directory as folder.jpg. This lets Windows Media Player tag the wma's correctly so they even transfer to my Zen with artwork.

Its a little ugly as you can see. I had to resort to some nasty regular expressions to track everything down without learning the entire file layout format. Please steal, improve, maintain, hack this as you see fit.


#!/usr/bin/perl
# This program will scan your music directory for missing album art (library). Then it will scan the napster
# database file (db) for a matching album. Then it will look up the missing image for that album in napsters
# own image archive (imgdir).
#
# Hopefully someone will come along and improve this, maybe make a plugin to WMP and improve the searching.
#

use File::Copy;

my $debug = 0;
my %hash = ();

$db = 'C:\Documents and Settings\All Users\Application Data\Napster\bin\db30.bin';
$imgdir = 'C:\Documents and Settings\All Users\Application Data\Napster\image';
#Your Napster library directory
$library = 'C:\Documents and Settings\CHANGEME\My Documents\My Music';
#Your Napster Username
$username = 'CHANGEME';


print "Fixing Napster Album Art v1.3\n";

# Find the albums with missing art
print "Searching for missing album art:\n";
opendir(LIB, $library) or die "Can't open $library: $!";
while( defined ($artist = readdir LIB) ) {
next if $artist =~ /^\.\.?$/; # skip . and ..


opendir( ART, "$library/$artist" );
while( defined ($album = readdir ART) ) {
next if $album =~ /^\.\.?$/; # skip . and ..
if ( -d "$library/$artist/$album" ) {
if( ! -f "$library/$artist/$album/folder.jpg" ) {
#get the first song in the album
opendir( ALB, "$library/$artist/$album" );
while( defined ($song = readdir ALB) ) {
next if $song =~ /desktop.ini/;

$songReg=$song;
$songReg=~s/[\x7f-\xff]/../g;
$songReg=~s/\W/./g;
if( -f "$library/$artist/$album/$song" ) {
$hash{$songReg}{'missing'} = 'true';
$hash{$songReg}{'album'} = $album;
$hash{$songReg}{'artist'} = $artist;
print " - Missing album art for $artist - $album\n";

last;
}
}
closedir(ALB);
}
}
}
closedir(ART);
}
closedir(LIB);
$size = keys( %hash );
print "Missing album art for $size albums\n";
if( $size == 0 ) {
exit 0;
}

# Find a song in that album
print "\nSearching for missing albums in napster database\n";
open(DB, $db) or die "Can't open $db: $!";
while(<DB>) {
while(m/\000$username\000(\w+)\0001\0000\0001\000\w+\000([^\000]+)\000([^\000]+)\000/g) {
$songID = $1;
$songName = $2;
$songFileName = $3;
#Normalize non-printable characters to periods
$songReg=$2;
$songReg=~s/\W/./g;

if( exists $hash{$songReg} ) {
print " - Found missing albums $hash{$songReg}{'album'} - first songID: $songID\n";
$hash{$songReg}{'ID'} = $songID;
}
}
}
close(DB);


# Find the album ID of that song
print "\nSearching for albumID and artwork in napster directories\n";
open(DB, $db);
while(<DB>) {
while(m/\000$username\000(\d+)\000(\d{5,20})\000(\d{5,20})\000([^\000]+)\000/g) {
my $songID = $1;
my $artistID = $2;
my $albumID = $3;
my $songShortName = $4;

for my $song ( keys %hash ) {
if( $hash{$song}{'missing'} eq 'true' && $hash{$song}{'ID'} eq $songID ) {
$dir = "$library/$hash{$song}{'artist'}/$hash{$song}{'album'}";
$img = "$imgdir"."/"."$albumID".".jpg";
print " - Found albumID for $dir: $albumID\n";

if( -f $img ) {
print " - Found artwork for album: $img\n";
#Finally copy the image for that album to its new home
copy( $img, "$dir/folder.jpg");
$hash{$song}{'missing'} = 'false';

} else {
print " - Error missing album image for $hash{$song}{'album'}: $albumID\n";
print " - img is: $img\n";
}
}
}
}
}
close(DB);


Steps to use this program:
1-Download and install perl from http://www.perl.com
2-Save that script to a file something like fixNapsterArt.pl
3-Edit fixNapsterArt.pl changing the CHANGEME lines to the appropriate values
4-Run fixNapsterArt.pl

| Sunday, November 14, 2004

At work we have a lot of issues with monitoring and logging. We run a lot of different services as EJBs in Weblogic clusters. They get a lot of use, so much so, that logging can actually make a real performance problem. The main issues are:
  • The logs are not aggregated in any one place.
  • There are a lot of ad-hoc scripts to do remote greps via ssh.
  • Any type of central aggregation will cause increased network load and increased CPU load on the servers.
While I was thinking about this, I remembered IBM used to have a HTTP log analyzer that, instead of pulling log files like most do, listened on a promiscuous ethernet device and built the logs itself, in real time. These logs could be analyzed and poked through at any time, without fear of bogging down the servers.

I'd like something like this for Java Applications. Something that could listen for known protocols like RMI, SOAP-RMI, etc. It could do performance montoring (avg., min, max, std. dev.) and also track error/exception rates. In my head these seems very possible, but maybe I'm missing something.

There are several products that use byte-code modification to instrument code, but this adds unnessesary overhead. I was quoted something like a 7% increase, and its still not doing the other half I need -- error analysis.

| Saturday, November 13, 2004

For the first 6 months of this blogs life, the browser usage of my visitors looked something like this:

IEs: 15%
Mozillas 79%
Others: 6%

Now that things have slowed down, their browser usings looks like this:

IEs: 42%
Mozillas 52%
Others: 6%

What is happening to Mozilla's market share? Nothing really. My analysis goes like this; When I used to post several times a week, my visitors were mainly repeat visitors or ones that came through aggregators like Erik's link blog. They were people in the know, proactive types. Now, I get mostly people searching for struts issues, with queries like "struts multibox bug". These people are reacting to their environment, rather than searching it out ahead of time.

In related news browser usages statisics, along with my crude excel skills, show Mozilla will overtake IE in less than 12 months.

| Sunday, October 03, 2004

I started a new job last month, which is why I haven't blogged in a while. I enjoyed working at a small Java consulting company, but the margins are too small, and life is a little too uncertain for me.

I'll never forget two quotes from the most recent project I worked on:

Early on in the project, a new developer was reporting on his progress of creating XML config files for Jasper reports. He was late, and was asked why. We were a little stunned with his reason:
Well, XML is very verbose. Every time you start a tag, you have to end it.


The second quote came towards the end of the project when we first entered User Acceptance Testing. They flew 20 users in from across the country to test usability of the application. The application had some performance problems with 20 simultaneous users and the manager said to me:
Look, this isn't good. The first time we get some load on the application and it can't keep up.

Well, it wasn't a shock to me or the other contractor at all. Software that is untested should be expected to fail. This client even had licensed copies of LoadRunner and we had asked to get a day or two to setup regression tests, including performance tests without luck. But this has taught me to stick to my guns, document my concerns and cover my butt more.

| Saturday, August 28, 2004

Last year you wrote some middleware using EJBs. This year your boss tells you that a C++ client needs to talk to those beans. You consider SOAP, but are dicouraged by reading about its performace. Papers showing SOAP is 100 to 1000 times slower than RMI are all over the place. Even with the newest pull parsers it still can't meet your performance needs of sub 200ms response times for up to 1000 transactions per second.

You start to look at other technologies, some more traditional, others a little newer.


CORBA

If your EJB container supports it, you can start using CORBA right away. But CORBA adds a lot more moving parts to the system. Your client code is more complicated. And its slower than some of the other solutions.

JNI Bridge


As strange as it might sound, there is actually a commercial product built around this. C++ wrappers are generated, which invoke RMI stubs via JNI. The client interface is simplier and faster, but you level of 'moving parts' is still fairly high. You need a JVM installed on each of your C++ clients.

Native C++ JNI


I couldn't find much information on this, but if your generating C++ wrappers, why not just generate an entire C++ RMI client. I found a small sample application / proof of concept.



So what do you do? Choose one of these? Maybe something more complicated with MQ to MDBs? Maybe a non-standard remoting method like Hessian?

| Tuesday, August 24, 2004

Kelly was a hard-core emacs user, while I used vi. We went our separate ways when Fusura.com imploded. In the last year I've fallen in love with IDEA, but the reasons I use it are common to most IDEs. Now that we are working at the same company again, I've tried to explain these reasons to Kelly with no results. This is an open letter to him in an attempt to explain why spending a few hours up front to configure an IDE will save days in the end...

Navigation

  • Fast opens - In IDEA I just use ^N to open any file in the project. No more cds, not more clicking through directory trees. I just start to type the file name and IDEA gives me the available matches across all the directories in the project.

  • Find usages - I used to have to generate JavaDoc or find+grep to find all the callers of a method or variable. This is a ton easier.

  • Goto declaration/implementation - Not completely unique to an IDE, since I can some of this with ctags, but its still a huge help.

Refactoring - There are plenty of refactorings available in IDEs, but these are the ones I commonly use.


  • Extract Method - Not only is IDEA really handy at breaking up complicated code, it also finds where I've cut and pasted this code and cleans those up too!

  • Rename - On my new project, I'm starting to attend code reviews. Its obvious people have never used an IDE with refactoring support. There were three differnent naming conventions used in one file. When its this easy to fix things, it makes it so painless.

  • Safe Delete - Not only does IDEA check is your file is used across Java code, but it also checks my struts-config.xml, web.xml and my JSPs.


Other

  • No compile errors - Sounds simple, but I went back to VI and saw a compile error for the first time in a year.

  • Code Generation

    • I write instance variables and generate getters, setters and constructors.

    • I write my struts action and while I'm doing it I call a method in my business logic that doesn't exist yet, passing it a constant from another file that doesn't exist yet, IDEA generates the stubs for these for me.

    • When I cut and paste code, IDEA automatically formats and adds missing imports for me.




These are my reasons, they're features I use on a daily basis. If you have more, please add a comment. If you've actually tried an IDE and are still using an editor, please leave a comment too.