NYCJUG/2005-09-13
Disruption, future meeting format, information visualization, regression statistics
Meeting Agenda for NYC JUG 20050913
1. Group project: assignments->progress?
a. H - work with grid control to organize and present data?
J vs. a third-party control?
b. D1 - save & restore for: screen states, search lists, etc.: interface?
c. J - indexer on "parseDir"?
d. D0 - organization of pieces -> version 1.0?
2. How shall we organize future meetings? Do we want to set aside time
for more individually-driven concerns? What would be the organizational
implications of this?
3. Starting a website: what goes on it? Who can do what? How shall
we host it? Can we use J on it?
4. Learning materials: establish some categories.
5. What application packages would be helpful for J to have (e.g.
quadratic solver)?
Meeting Notes
This meeting didn't cover much as we were disrupted by another group meeting in the same place. A media freelancers organization was having a networking party in the space where we normally have our meetings, so it was more crowded than usual and some of the people were interesting to talk to, so we did that.
One person I talked to quite a bit was someone I already knew who's a top Java consultant. He had some good suggestions on a topic we've brought up before at some of the NYCJUG meetings: information visualization. He suggested looking at Brad Paley's Computer Visualization and Living with the Genie. He also mentioned work by Ken Perlin, who worked on the movie "Tron" (the first full-length, computer animated movie, which came out aeons ago), who is now at the NYU (New York University) Media Research Lab.
We also talked about a facet of data visualization that Jim has raised as a long-term interest of his, perhaps as a group project: how do we focus on data details, especially tasks, while keeping them in the larger context, e.g. tasks within a large project?
Jim and I had attended a presentation at the New York Academy of Sciences a few years back by a Stanford graduate student named Tamara Munzner who works with presenting networks of information. These networks are presented so that the focus of interest is centered on the screen and enlarged while the connecting nodes are smaller and more on the periphery the more distant they are, a "fish-eye" view of information. See http://graphics.stanford.edu/~munzner/talks/nyas02/mgp00010.html to get an idea of this; http://www.cs.ubc.ca/~tmm/talks.html has more topics by Munzner.
The little bit of NYCJUG business we did cover was further ideas on re-structuring the meetings: perhaps having individual assignments, keeping the beginner's section to a short time at the start of the meeting, and having Jim and Harteg work on a presentation of Chris Burke's paper comparing APL to J.
Group Website
From: John Randall <randall@andromeda.rutgers.edu> To: devon@acm.org Date: Sep 12, 2005 7:53 AM Subject: Re: NYCJUG I sent you an email a while back about the host websitesource.com that I use for a domain. For under $10 a month you get MySQL, PHP, and a load of other things. This is essentially a commodity price, but I have had no trouble with this particular host. I think using J as a server-side scripting language may have security issues. I would suggest: - For static web pages, just use HTML directly. - For dynamic web pages, use PHP. - For a database, use MySQL. - For a wiki (which allows distributed administration of web pages, and which Dan is keen on) there are a variety of solutions. The top of the line is mediawiki, used by Wikipedia.
An Example of Regression and Statistics Related to it
$sga=. tablf2mat CR-.~fread 'c:\global\fof\current\szGroImpAlphaWNames.csv'
14 7
sga
+----------+---------+-------+-----+------+----------+-------------+
|Asset ID |Name |Weight |SIZE |GROWTH|Total Risk|Implied Alpha|
+----------+---------+-------+-----+------+----------+-------------+
|+ACGM-S |ACGM-S |15.00 %|-0.54|0.38 |0.17 |0.003347 |
+----------+---------+-------+-----+------+----------+-------------+
|+ComStkEIG|COMSTKEIG|45.00 %|0.19 |-0.18 |0.13 |0.001814 |
+----------+---------+-------+-----+------+----------+-------------+
|+LCGCW |LCGCW |20.00 %|0.13 |0.62 |0.15 |0.002624 |
+----------+---------+-------+-----+------+----------+-------------+
|+MAP |MAP |6.00 % |-0.70|0.03 |0.15 |0.002193 |
+----------+---------+-------+-----+------+----------+-------------+
|+MCGM-S |MCGM-S |4.00 % |-1.42|0.47 |0.18 |0.003762 |
+----------+---------+-------+-----+------+----------+-------------+
|+SP50EIG |SP50EIG |5.00 % |0.43 |-0.06 |0.13 |0.001351 |
+----------+---------+-------+-----+------+----------+-------------+
|+SCVEIG |SCVEIG |5.00 % |-2.51|-0.22 |0.16 |0.002840 |
+----------+---------+-------+-----+------+----------+-------------+
|+ACVM-S |ACVM-S |0.00 % |-0.13|-0.05 |0.14 |0.001597 |
+----------+---------+-------+-----+------+----------+-------------+
|+CapApM-S |CAPAPM-S |0.00 % |0.00 |0.41 |0.16 |0.002717 |
+----------+---------+-------+-----+------+----------+-------------+
|+MCVEIG |MCVEIG |0.00 % |-1.28|-0.38 |0.14 |0.002110 |
+----------+---------+-------+-----+------+----------+-------------+
|+MCVM-S |MCVM-S |0.00 % |-1.15|-0.11 |0.17 |0.002824 |
+----------+---------+-------+-----+------+----------+-------------+
|+SCVM-S |SCVM-S |0.00 % |-2.84|-0.14 |0.18 |0.002590 |
+----------+---------+-------+-----+------+----------+-------------+
|+LCVM-S |LCVM-S |0.00 % |0.23 |-0.04 |0.13 |0.001491 |
+----------+---------+-------+-----+------+----------+-------------+
nn=. ".&>0 0 0 1 1 0 1#|:}.sga
]reord=. /:2{nn
5 12 7 1 9 3 11 2 8 10 6 0 4
nn=. nn{~&.|:/:2{nn NB. Order by alpha
sgc=. (|:1,2{.nn)%.,2}.nn
est=. sgc +/ . * 1,2{.nn
sgc1=. sgc%mean est%2{nn
est1=. sgc1 +/ . * 1,2{.nn
]xl=. 'xlabel ',' ' punclist reord{}.1{"1 sga
xlabel SP50EIG LCVM-S ACVM-S COMSTKEIG MCVEIG MAP SCVM-S LCGCW CAPAPM-S MCVM-S SCVEIG ACGM-S MCGM-S
va=. stddev"1 nn
plot nn%"(1 0) 1,~(2{.va)%2{va
('key ScaledGrowth ScaledSize EstAlpha Alpha;pensize 2;',xl) plot (2{.nn%"(1 0) 1,~(2{.va)%2{va),est1,:2{nn
pd 'save wmf C:\global\fof\current\SzGroAlphaEst.wmf'
require 'opengl'
coinsert 'jzopengl'
glfile 'C:\global\fof\current\SzGroAlphaEst.bmp'
glsavebmp 1270 1070
regression
4 : 0
x=. 1,.x.
y=. y.
b=. y %.x
k=. <:{:$x
n=. $y
sst=. +/*:y-(+/y) % #y
sse=. +/*:y-x +/ .* b
mse=. sse%n->:k
seb=. %:({.mse)*(<0 1)|:%.(|:x) +/ .* x
ssr=. sst-sse
msr=. ssr%k
rsq=. ssr%sst
F=. msr%mse
r=. ,: ' Var. Coeff. S.E. t'
r=. r, 15.0 15.5 15.5 12.2 ": (i.>:k),.b,.seb,.b%seb
r=. r, ' '
r=. r, ' Source D.F. S.S. M.S. F'
r=. r, 'Regression', 5.0 15.5 15.5 12.2 ": k, ssr,msr,F
r=. r, 'Error ', 5.0 15.5 15.5": (n-k+1), sse,mse
r=. r, 'Total ', 5.0 15.5 ": (n-1), sst
r=. r, ' '
r=. r, 'S.E. of estimate ', 12.5":%:mse
r=. r, 'Corr. coeff. squared', 12.5": rsq
)
(|:2{.nn) regression 2{nn
Var. Coeff. S.E. t
0 0.00192 0.00013 14.47
1 _0.00052 0.00011 _4.90
2 0.00182 0.00036 5.05
Source D.F. S.S. M.S. F
Regression 2 0.00001 0.00000 18.87
Error 10 0.00000 0.00000
Total 12 0.00001
S.E. of estimate 0.00036
Corr. coeff. squared 0.79050
sgr=. 0.00192 _0.00052 0.00182
estr=. sgr +/ . * 1,2{.nn
('key ScaledGrowth ScaledSize EstAlpha Alpha;pensize 2;',xl) plot (2{.nn%"(1 0) 1,~(2{.va)%2{va),estr,:2{nn
pd 'save wmf C:\global\fof\current\ScSzGroAlphaJEst.wmf'
glfile 'C:\global\fof\current\ScSzGroAlphaEst.bmp'
glsavebmp 1270 1070