Archive for October 11, 2007

Weka & Stock Data

October 11, 2007 @ 4:54 pm · Filed under Computing, Economics, Finance, Mathematics, Quants

As part of my job, I do a lot of data analysis and interaction analytics. Lately, I have been playing around with this tool called Weka, from the University of Waikato, New Zealand.

And last night, I was bored and decided to play around with some of the historical stock data from the S&P 500 with Weka and see if I could do some interesting things.

Initially, I had trouble getting JVM to work with more memory and it turned out that this was a fairly common memory problem with Weka. I finally managed to fix it (but it still is only running on 1 GB of memory; for whatever reason I couldn’t get it to go beyond that).

Since I got that figured, I’ve been trying to run a clustering algorithm on the data for a while now (and it’s been taking forever). So far, I’ve found some interesting patterns, but here are some interesting teaser visualizations.

And if you are interested in playing with it yourself, here is the ARFF file for the stock data (zipped).

Update: This dataset is from 12/31/2004 until 10/09/2007.

Comments (5)