Code Comments

If you want to compile Java files at the command line, you use the javac command.

javac ../Java/*.java

This will create a .class file for every .java file that could be compiled.

But let’s say you are referencing one of many external Java libraries that are packaged as .jar files. You probably don’t want to just stick the .jar files in the same directory as the .java files. That would be sort of messy. In my case, I’m putting those in a sub-directory called lib. But the Java compiler doesn’t always know where to look for those .jar files. Below is one simple way to tell it.

javac -Djava.ext.dirs ../Java/lib ../Java/*.java

February 5th, 2010 | Category: Java | Leave a comment

Sometimes you just need some extra vertical space in a LaTeX document. There are a couple of simple ways to approach this. This first is the vspace command:

\vspace{5 mm}

This approach gives you a lot of flexibility because you can specify exactly how large you want the vertical space to be.

However, sometimes, you simply want a vertical line that is the same size as other lines in your document. An easy way to do this is to use the blankline command:

\blankline

Enjoy!

January 13th, 2010 | Category: LaTeX, Tip | Leave a comment

I installed cygwin, which allows you to execute Linux-based applications on your Windows machines. But what I wanted to do was connect (using ssh) to a Linux box and run an application that has a GUI from there. In this case, I wanted to be able to connect to server and run firefox from the server. This would enable me to download some files to the server, etc.

When I ssh’d into the machine:

ssh me@server.edu

And then tried to start firefox:

firefox &

I got the following error:

Error: no display specified

In searching around, the solution I found was that you have to enable X11 Forwarding. A simple way to do this is to specify the -X parameter when you ssh into the server:

ssh -X me@server.edu

Then when you try to open firefox, it should work:

firefox &

December 28th, 2009 | Category: Linux | Leave a comment

In R, you can plot a variety of symbols when you are plotting points on a graph. The default is a hollow circle. But it is very flexible. To do this, you use the pch parameter when you create a plot. You can find information in the help files (?pch), but it only describes the symbols and doesn’t illustrate what they are. I was wondering for my project, so I put together a little script to do this. Below is the code for symbols 1–25. And here’s the graph

par(mar=c(2, 0, 0, 0) + 0.1)
pchRange = 1:25
 
for (i in pchRange)
{
  if (i==1)
    plot(rep(i, 10), 1:10, pch=i, xlim=c(min(pchRange), max(pchRange)), xlab=0, ylab=0, yaxt="n" )
  if (i>1)
    points(rep(i, 10), 1:10, pch=i)
}

December 4th, 2009 | Category: R | Leave a comment

Let’s say you did a search for files matching a certain pattern in a directory using Python:

import glob
 
filePaths = glob.glob("C:\\Temp\\*.txt")
print filePaths

This will list the full file paths with a .txt extension in the C:\Temp directory. For example: C:\\Temp\\test.txt.

But if you wanted to get just the file name, how would you go about that? It took me a little while to find an answer, and the method not super obvious, so I’ll post it here.

import glob, os
 
filePaths = glob.glob("C:\\Temp\\*.txt")
 
for filePath in filePaths:
  print os.path.basename(filePath)

December 3rd, 2009 | Category: Python | Leave a comment

In this post, I explained how to find quartiles in a list of numbers. As a slight add-on to that functionality, you can easily get the interquartile range. This basically means that you are finding the difference between the first and third quartiles.

So I use the code to compute the quartiles and then use simple math to subtract the difference.

    public static double InterQuartileRange(ArrayList values) throws Exception
    {
	double[] quartiles = Quartiles(values);
	return quartiles[2] - quartiles[0];
    }

October 16th, 2009 | Category: Java, Math | Leave a comment

For a brief overview of what quartiles are, you might read the Wikipedia page on this topic.

Basically, what it means is that if you were to break a list of numbers into four even parts, these would be the values that would separate them. But it gets a little complicated when your list doesn’t break into even chunks, etc. But I won’t get into the details of that here.

I threw together some code for getting the quartiles, which is below. I’ve also included some helper methods that I created to help with it. Also, note that I reused the Median method described here.

public static double[] Quartiles(ArrayList values) throws Exception
{
    if (values.size() < 3)
    throw new Exception("This method is not designed to handle lists with fewer than 3 elements.");
 
    double median = Median(values);
 
    ArrayList lowerHalf = GetValuesLessThan(values, median, true);
    ArrayList upperHalf = GetValuesGreaterThan(values, median, true);
 
    return new double[] {Median(lowerHalf), median, Median(upperHalf)};
}
 
public static ArrayList GetValuesGreaterThan(ArrayList values, double limit, boolean orEqualTo)
{
    ArrayList modValues = new ArrayList();
 
    for (double value : values)
        if (value > limit || (value == limit && orEqualTo))
            modValues.add(value);
 
    return modValues;
}
 
public static ArrayList GetValuesLessThan(ArrayList values, double limit, boolean orEqualTo)
{
    ArrayList modValues = new ArrayList();
 
    for (double value : values)
        if (value < limit || (value == limit && orEqualTo))
            modValues.add(value);
 
    return modValues;
}

October 16th, 2009 | Category: Java, Math | Comments (1)

This is a pretty simple tip, but I still thought I would share it for anyone who is interested. To find out whether an integer is odd, you can use the modulo operator. This operator tells you the remainder after dividing the number by some other number. If you divide any integer by 2, you would expect a remainder of 1 if it is odd or no remainder if it is even.

public static boolean IsOdd(int number)
{
    return number % 2 == 1;
}

October 16th, 2009 | Category: Java, Math | Leave a comment

I came across an interesting summary of research that was performed at Microsoft regarding the effectiveness of commonly used software development practices, including test-driven development, code coverage, small teams, etc. It was interesting that the researchers tried to quantify how well these practices work, rather than rely on what supposed “experts” say. See the summary, along with links to research papers, here.

October 8th, 2009 | Category: Tip | Leave a comment

Cygwin is a software tool that allows you to run Linux programs in Windows. I have had good success in using it to run commands locally and to connect to other servers via ssh, scp, etc. I just got a new computer and installed the latest version of Cygwin, but when I tried to ssh to another server, it told me the ssh command could not be found. This was weird, because it had worked just fine on my previous computer with the base install (I think).

After looking long and hard for a solution to this, I found someone suggesting that it simply wasn’t installed. So then I installed basically every program/library that Cygwin has to offer (since disk space is not a concern right now), and that seems to have solved the problem. The URL above tells you how to do this. If you want to be more selective about which libraries to install, that is fine. I just didn’t want to figure that out at the time, and I thought I’d see what other programs Cygwin has to offer.

September 29th, 2009 | Category: Linux | Leave a comment

This bit of advice is not new to me nor to the software development community. But recently I had one of those experiences where I put together a quick solution (to keep my code as simple as possible), and later I ran into a performance problem. And it was because I was violating this principle.

I am generating a very large string (based on values that are read from a text file). I had instantiated a string and was appending to the end of it, like so:

String output = "";
for (String line : fileReader)
  output += line;

And it was going v….e….r….y…s….l….o….w. After figuring out where the slowness was occurring, I used the StringBuilder instead. And it started going fast! Here’s the general idea of the change to the code.

StringBuilder output = new StringBuilder();
for (String line : fileReader)
  output.append(line);

The reason the latter is so much faster is that it only has to create one object in memory: the StringBuilder object. If you just use a String, it has to recreate a new String object in memory each time you append to it.

Just something to keep in mind.

September 18th, 2009 | Category: Java, Performance | Leave a comment

I’m learning something new about Linux every day, so this is something that will probably be obvious to many of you out there. But I needed today to read the contents of a file into a variable in a bash script. The simple way to go about doing this is to use the following command:

myvar=`cat filename.txt`

September 11th, 2009 | Category: Linux | Leave a comment

Imagine you wanted to come up with a “classifier” to predict whether Georgia would win a given American football game. So you might get (training) data from all their games from the previous two years. Then you might come up with rules based on that data. For example, if the quarterback throws for 300+ yards, the running back runs for 100+ yards, and the defense gets 2+ interceptions, then you think they will win. Then you test those rules on the same (training) data, and it turns out that voila! it is correct 100% of the time. But you don’t really know whether that classifier (set of rules) is generalizable because you only tested it on data you had already looked at. So you could apply the rules to 2009 data (test set) and see how well it performs there. But it could be that the team is so different in 2009 from last year that these rules don’t hold. So while the classifier “fit” the training data well, in fact it “overfit” the training data because it didn’t also “fit” the test data. That’s why it’s important to hold out a test set so you can evaluate the classifier.

Your training data wouldn’t necessarily have to be from 2007-2008. You could pick five games from each of the last 10 seasons and have that be your training data. And then you could pick five other games (non overlapping) from each those seasons and make that your test set. The important thing is that they don’t overlap. Or you could use cross validation in which 1 game from each of the last 10 seasons is used as the test set, and the remaining 9 games from those seasons are used as the training set. This is repeated 10 times, each time using a different game from each seasons for the test set and the remaining games for the training set.

September 11th, 2009 | Category: Statistics | Leave a comment

I had an occasion to learn a little about programming in C for the first time. One thing I needed to be able to do was save text to a file, but it was hard to find help for this on the Internet. This may be because it is called something slightly different in C than what I’m used to.

Anyway, below is some sample code.

#include  <stdio.h>
 
FILE *fp;
fp = fopen("/home/Bill/path/test.txt", "w");
fprintf(fp, "%-15.15s\n", "Hello, world!");
fclose(fp);

That should get you started at least. There are other options, such as appending to a file (example below), writing numbers, etc.

fp = fopen("/home/Bill/path/test.txt", "a");

September 9th, 2009 | Category: Tip | Leave a comment

I am running an application in one location on the file system, and I need to be able to invoke an application that is in a different directory on the file system. Due to personal preference, this other directory is not specified in the PATH variable. So I’m wondering how I can get Linux to know that it has to look for the second application in that other directory.

A simple way to approach this is to use the cd command in combination with the call to the second program. This will change the active directory to the second one and execute the command. This works for me because that call is being made in a different session than the one the first application is running in. Not sure if the terminology is right there. Please correct me if I’m wrong. Anyway, here’s the syntax:

cd /home/Bill/secondapp;./scriptname

September 9th, 2009 | Category: Linux, Tip | Leave a comment

In Linux, you sometimes want to run a command when the server first starts up. I was doing this recently where I would have to manually go in and run a command each time the server got rebooted. This was a pain, so with a little help from my friend Martin, I learned how to have that script run automatically each time. There is a more advanced way to handle startup scripts, which is to use the /etc/init.d/ directory. I won’t go into that. You can Google it if you want more detail. To have a simple command run at startup, go to the /etc/rc.local file. And past the command at the end.

Below is what my file looks like (or close to it). This runs a script called MyScript. Make sure the script has execute permissions and that you have the full path to the script.

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
 
touch /var/lock/subsys/local
 
/path_to_script/MyScript &

September 4th, 2009 | Category: Linux, Tip | Leave a comment

In this post, I explain the problem of sorting strings that contain numbers. If you just sort values using the default approach, it will not work properly. For example, it will sort “a10″ before “a2″ even though in your application you may want it to consider the alphabetic characters separately from the numeric ones in sorting. Or in other words, because the first character is the same, you may want it to treat the remaining portions as numbers. So “a2″ would be sorted before “a10″ because 2 is less than 10.

I converted the Python code from the other post to Java code. I won’t explain how it works, but let me know if you want an explanation.

import java.util.*;
 
public class NumericString implements Comparable
{
    private String _rawValue;
 
    public NumericString(String rawValue)
    {
	_rawValue = rawValue;
    }
 
    private static int CompareTwoVals(String xVal, String yVal)
    {
	if (IsDouble(xVal) && IsDouble(yVal))
	    return new Double(Double.parseDouble(xVal)).compareTo(Double.parseDouble(yVal));
	if (!IsDouble(xVal) && !IsDouble(yVal))
	    return xVal.compareTo(yVal);
	if (IsDouble(xVal))
	    return -1;
	return 1;
    }
 
    private ArrayList GetValueList()
    {
	ArrayList valueList = new ArrayList();
	String tempVal = "";
 
	for (int i=0; i<_rawValue.length(); i++)
	{
	    String val = _rawValue.substring(i, i+1);
 
	    if (IsDouble(val))
		tempVal += val;
	    else
	    {
		if (!tempVal.equals(""))
		{
		    valueList.add(tempVal);
		    tempVal = "";
		}
 
		valueList.add(val);
	    }
	}
 
	if (!tempVal.equals(""))
	    valueList.add(tempVal);
 
	return valueList;
    }
 
    public static boolean IsDouble(String value)
    {
	try
	{
	    Double.parseDouble(value);
	    return true;
	}
	catch (Exception ex)
	{
	    return false;
	}
    }
 
    @Override
    public int compareTo(Object obj)
    {
	if (obj == null)
	    return -1;
 
	NumericString compareObj = (NumericString)obj;
 
        if (this._rawValue == null) return -1;
        if (compareObj._rawValue == null) return 1;
        if (this._rawValue.equals(compareObj._rawValue)) return 0;
 
        ArrayList xList = this.GetValueList();
        ArrayList yList = compareObj.GetValueList();
 
	for (int i=0; i<xList.size() && i<yList.size(); i++)
	{
	    int compareResult = CompareTwoVals(xList.get(i), yList.get(i));
 
	    if (compareResult != 0)
		return compareResult;
	}
 
	return new Integer(xList.size()).compareTo(yList.size());
    }
 
    @Override
    public String toString()
    {
	return _rawValue;
    }
}

Here’s how you might call it:

NumericString a = new NumericString("a10");
NumericString b = new NumericString("a2");
 
ArrayList list = new ArrayList();
list.add(a);
list.add(b);
 
Collections.sort(list);
 
for (NumericString x : list)
  System.out.println(x);  // a2, a10

August 19th, 2009 | Category: Java, Tip | Leave a comment

I’m writing this post because I just spent a couple of hours banging my head against the wall, trying to figure out how to run an R script from the command line. It was working if I simply ran it at the command line. But when I try to run the same command from Java (I know, this sounds convoluted) using the following code, it was behaving strangely and wasn’t executing.

Runtime r = Runtime.getRuntime();
r.exec("R CMD BATCH RScriptFile.R");

So I finally found out that there is a utility in R that is designed to help you execute scripts at the command line more easily. It’s called RScript. Now I’m doing it the following way, and it’s working beautifully.

Runtime r = Runtime.getRuntime();
r.exec("Rscript RScriptFile.R");

Seems obvious, but it wasn’t to me, and it took awhile to find an answer.

August 12th, 2009 | Category: Java, R | Comments (1)

In my last post, I explained a nifty way to invoke a function dynamically when you have the name of the function in a character object (in R). However, this didn’t explain how you could pass parameters to that function. I found a way to do this, which I will explain below, though it is possible that there is a simpler way to do it, of which I am not aware. But the way I describe should be very flexible for all types of dynamic invocation.

Before I get into the details, I will explain a possible use case for this type of functionality. It might seem far fetched, but it’s the way I’m approaching one of my research tasks right now, at least as a workaround until I can find a more elegant way to do it. So…most of my code for this project is written in Java. But I need to be able to do some statistical processing in R (which has far richer statistical capabilities than Java). The way I’m approaching this is to communicate between Java and R using command-line invocation (I can provide more details if anyone is interested). I need to tell R via the command line that it should load source code from a given file and invoke a specific function using specific parameters. Then in the R code, it needs to parse those values and then call the functions specified. Hopefully that made sense…let me know if not.

Anyway, here’s how you do it. Suppose you had a function like this:

x = function(param1, param2)
{
  print(paste("value of param1:", param1))
  print(paste("value of param2:", param2))
}

The normal way you would invoke this would be:

x("abc", "def")

To invoke it dynamically, you could do this:

eval(call("x", "abc", "def"))

August 12th, 2009 | Category: Tip | Leave a comment