Indy Telecom & Industrial Media

Here are some links for this week –

• Analysts Lose 17% for Investors in Brokerage Picks — Twice the decline of the Standard & Poor’s 500 Index.
• Warren Buffet’s bet against hedge funds — I agree with a lot of what Mr. Buffet has to say; this is just not one of them.

As part of my job, I do a lot of data analysis and interaction analytics. Lately, I have been playing around with this tool called Weka, from the University of Waikato, New Zealand.

And last night, I was bored and decided to play around with some of the historical stock data from the S&P 500 with Weka and see if I could do some interesting things.

Initially, I had trouble getting JVM to work with more memory and it turned out that this was a fairly common memory problem with Weka. I finally managed to fix it (but it still is only running on 1 GB of memory; for whatever reason I couldn’t get it to go beyond that).

Since I got that figured, I’ve been trying to run a clustering algorithm on the data for a while now (and it’s been taking forever). So far, I’ve found some interesting patterns, but here are some interesting teaser visualizations.

And if you are interested in playing with it yourself, here is the ARFF file for the stock data (zipped).

Update: This dataset is from 12/31/2004 until 10/09/2007.