NYCJUG/2005-02-08
< NYCJUG
Jump to navigation
Jump to search
Group project, Save Work, persistent data, raising interest in J, language as culture, picture albums, date arithmetic, catalog obverse, CSV files, beginner questions
Meeting Agenda for NYC JUG 20050208
1. Group project: focus on good design - see Cooper*, et al.
a. What will it be? Suggestions so far:
i) brief tutorial on my backup utility; perhaps also alarmMsg ->
sub-project of persistent window data -> user preference tracking.
ii) sketch of an OpenGL project: boolean solids operation;
(show de Casteljau algorithm);
iii) further thoughts on what we might do with MIDI or other
music-based idea;
iv) collections utilities: mass rename, tracking of MP3s, photos;
v) backtrack and code collection utility;
vi) any other ideas? General-purpose utilities like CSV read/write?
b. How should we structure it?
c. What are small sub-projects into which we can break it?
i) what are general classes of utility functions? e.g.
date-time (see "DateAdd" attached), file-handling.
d. Use version control?
See http://www.ericsink.com/scm/source_control.html
Or use J projects? SubEthaEdit?
e. Coding standards? Aim for multi-OS version?
2. Learning and creating learning materials: how to start this? See point
1.c. above - a general framework will give a broad target at which to aim.
How to structure it? As a FAQ? Hypertext?
What's in my "J/primer" directory? Show DHMIdioms.txt - too idiosyncratic?
Feedback to J Software.
3. Preliminary assignments: who can commit to how much by next month?
4. Show and tell: "DateAdd" -> dates.ijs & sfl.ijs: illustrating
the difficulty of re-use.
- The Inmates are Running the Asylum by Alan Cooper; see
Proceedings
Save Work Project - Possible Core: parseDir
I told a little bit about how "parseDir.ijs" works: File:ParseDir.ijs.
This is the backup facility I've been using for years - it could form the core of a "Save Work" project.
One feature of the script file is that it outputs the most commonly-used commands to the J session upon loading:
load '~user/code/parseDir.ijs'
6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=: getDirFlInfo_parseDir_ ''C:\'''
(<'\amisc\') fileVar_WS_&.>'FLNMS';'FLDTS';'FLSZS';'FLPARENT';'DIRNMS';'DIRDEP'
'batfl cmds'=. buildBatFl_parseDir_ 700e6;'C:\Temp\Recent\'
winexec ('cmd /c ',batfl);1
This makes it simple to avoid typing long commands by grabbing the lines from above that are close to what I want and modifying them if necessary.
First, build the global variables with information about all the files and directories on my C: drive (timing it):
6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=: getDirFlInfo_parseDir_ ''C:\''' 785.195 0 60#:785.195 13 5.195
This was run on my laptop - 1.5GHz Thinkpad, 512M RAM, 31GB disk with 3.2GB free. As you can see above, it took about 13 minutes.
Now build the .BAT file to create target directories and copy 100M (=1e8) of the most recently-changed files to my thumb (K:) drive:
'batfl cmds'=. buildBatFl_parseDir_ 1e8;'K:\'
Run the .BAT file (asynchronously):
winexec ('cmd /c ',batfl);1
+--+------------------------------------------------------+-+
|33|cmd /c C:\DOCUME~1\DMCCOR~1\LOCALS~1\Temp\CDMDCopy.bat|1|
+--+------------------------------------------------------+-+
Build another command file to dump more files to a network drive:
6!:2 '''batfl cmds''=. buildBatFl_parseDir_ 5e8;''U:\'''
3.55641
winexec ('cmd /c ',batfl);1
+--+------------------------------------------------------+-+
|33|cmd /c C:\DOCUME~1\DMCCOR~1\LOCALS~1\Temp\CDMDCopy.bat|1|
+--+------------------------------------------------------+-+
AlarmMsg: GUI Example
"AlarmMsg.ijs" (File:AlarmMsg.ijs) is a simple GUI application I wrote that allows me to have a window pop up after a specified time to remind me of something.
It's interesting in relation to the "Save Work" example only in that it exemplifies how one might save information between invocations of code. This code uses the file "AlarmParms.ini" to retain the settings entered by the user between invocations. The simple text file consists simply of "Name=Value" pairs corresponding to the three input fields on the dialog box:
ALRMSG=NYCJUG at Heartland on 34th at 18:30 SECS=0 MINS=525
So, each user update is written to this file to be available the next time the code is called. This persistence can be very useful as we often specify very similar parameters on subsequent invocations of code. It's a nice touch if the code remembers what we usually do and makes it available.
How To Get People Interested in J: Games?
Date: Wed, 26 Jan 2005 12:36:26 -0500 From: "Miller, Raul D" <rdmiller@USATODAY.COM> Subject: Re: [Jforum] Programming language (fwd) To: JFORUM@PEACH.EASE.LSOFT.COM Fraser Jackson wrote: > When I asked one small group of 12 year old boys what they wanted > to program their answer was simple - games. If Raul can write up > and share some of his explorations of OpenGL it would be great. > If we could have a 'Beginning OpenGl Game Programming'(Astle and > Hawkins) but using J and its marvellous capability of interactive > function development it would be a vast step forward in creating > an environment for wider use of J. I don't think I'm really ready to tackle these kinds of issues yet. My knowledge of OpenGL at the moment is extremely sketchy and well below the level needed for a game more advanced than "pong". However, it's my impression that if J were to serve as a game engine (as opposed to a game design tool), you would need to incorporate some bits of C into the application, to get performance to acceptable levels. Of course, if you're using opengl 2.0 (shader language), you're also using C (or something extremely close), but the compilation and loading of that code is handled by the opengl drivers. I'm talking about the bits of code which run on the J side of the opengl interface. For example (and this example might be more a demonstration of my ignorance of opengl than anything else), I'm under the impression that the fastest way to pump vertex information into an opengl driver is using glVertexPointer -- which has some implications for the use of display lists (I'm presuming that a display list would persist beyond individual calls to glVertexPointer). Anyways, I'm likely to put together some small physics simulations well before I'm likely to put together something which I think fits in the "game" classification. [And keep in mind that my explorations of OpenGL are, at most, only a few hours a week.] But I will try to take a look at the Astle and Hawkins book. Thanks, -- Raul
Language as Culture
[from http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=273&page=4] [interview with Alan Kay in ACM's Queue magazine]
[Alan Kay] In a history of Smalltalk I wrote for ACM, I characterized one way of looking at languages in this way: a lot of them are either the agglutination of features or they’re a crystallization of style. Languages such as APL, Lisp, and Smalltalk are what you might call style languages, where there’s a real center and imputed style to how you’re supposed to do everything. Other languages such as PL/I and, indeed, languages that try to be additive without consolidation have often been more successful. I think the style languages appeal to people who have a certain mathematical laziness to them. Laziness actually pays off later on, because if you wind up spending a little extra time seeing that “oh, yes, this language is going to allow me to do this really, really nicely, and in a more general way than I could do it over here,” usually that comes back to help you when you’ve had a new idea a year down the road. The agglutinative languages, on the other hand, tend to produce agglutinations and they are very, very difficult to untangle when you’ve had that new idea.
Also, I think the style languages tend to be late-binding languages. The agglutinative languages are usually early-binding. That makes a huge difference in the whole approach. The kinds of bugs you have to deal with, and when you have to deal with them, is completely different.
Some people are completely religious about type systems and as a mathematician I love the idea of type systems, but nobody has ever come up with one that has enough scope. If you combine Simula and Lisp—Lisp didn’t have data structures, it had instances of objects—you would have a dynamic type system that would give you the range of expression you need.
It would allow you to think the kinds of thoughts you need to think without worrying about what type something is, because you have a much, much wider range of things. What you’re paying for is some of the checks that can be done at runtime, and, especially in the old days, you paid for it in some efficiencies. Now we get around the efficiency stuff the same way Barton did on the B5000: by just saying, “Screw it, we’re going to execute this important stuff as directly as we possibly can.” We’re not going to worry about whether we can compile it into a von Neumann computer or not, and we will make the microcode do whatever we need to get around these inefficiencies because a lot of the inefficiencies are just putting stuff on obsolete hardware architectures.
I just think that’s a two-culture divide. I’ve seen many meetings where people are unable to communicate just because of the stylistic differences in approaches.
Building Picture Albums
Date: Tue, 25 Jan 2005 09:24:37 -0800 From: "Joey K Tuttle" <jkt@QUED.COM> Add to Address BookAdd to Address Book Subject: Re: [Jforum] Programming language (fwd) To: JFORUM@PEACH.EASE.LSOFT.COM On Fri, 21 Jan 2005, Brian M. Schott wrote: > to produce a dynamic webhosted application. Being a > (retired) kid at heart, myself, I know that I have had to > seek out perl for two of my own apps -- to get them on the > web -- because J cannot do so. > At 21:17 -0500 2005/01/24, Brian M. Schott wrote: >afaik: J only supports html (right?); At 08:40 -0500 2005/01/25, Brian M. Schott wrote: > I mean that the only web-related feature provided by >j is html encoding. My terminology may be wrong (worse yet, >my last statement may be wrong), but I mean that there are no facilities for cgi interfacing, though. Brian, I meant (tried) to post this last friday when it was more timely, but I failed twice (my own problems and distractions). I'm not sure what you did with the PERL, but I happily use j as CGI calls on my Apache web server - that is, a user somewhere on a browser can create dynamic pages that are built by j - this requires no knowledge of what is in the background, and I like writing CGI in j much more than in PERL -- although PERL can be a "module" in Apache, j could be too with a little porting effort. This would make things even faster and more efficient - but they work very well already. As an added thought from one of the other posters, the most used such application I have is building picture albums dynamically. I gave a talk on this subject to the APLbug (SF Bayarea User Group) a couple of years ago. - joey
Date Arithmetic
A common thing we want to do is to find the difference between dates or add a certain number of days, weeks, or months to a given date. Visual Basic has a nice function "DateAdd" which does this.
DateAdd=: 3 : 0
NB.* DateAdd: limited mimic of VB DateAdd - DateAdd <unit to add>;date;num.
NB. Date in form YYYYMMDD.
ts=. i. 0 [ units=. 'YMDW' NB. Units are Year, Month, Day, Week.
'unit dt addnum'=. y.
assert (unit=. {.toupper unit) e. units
dpu=. (372 31 1 7){~units i. unit NB. 372=12*31: provisional "year"
mobase=. 0 12 31
assert. dt>18000101 NB. todayno only goes back to this date
dt=. 0 100 100#:dt
select. unit
case. 'D';'W' do.
dt=. 100#.todate (dpu*addnum)+todayno dt
case. 'M';'Y' do.
fakedayno=. mobase#.0 _1 0+dt
dt=. 100#.todate todayno 0 1 0+mobase#:fakedayno+dpu*addnum
end.
assert. dt>18000101 NB. todayno only goes back to this date
NB. dtdifs=. 2-/\todayno&>(<0 100 100)#:&.> dtsa=. DateAdd&>(<'D';20050101),&.><&.>i:_10000
NB. 1 *./ . = dtdifs
NB. dtdifs=. 2-/\todayno&>(<0 100 100)#:&.> dtsa=. DateAdd&>(<'W';20050101),&.><&.>i:_10000
NB. 7 *./ . = dtdifs
)
This uses following J-supplied code:
NB. From ~system\main\dates.ijs
NB. =========================================================
NB.*todayno v converts dates to day numbers
NB. converts dates to day numbers, converse <todate>
NB.
NB. y. = dates
NB. x. = optional:
NB. 0 - dates in form <yyyy mm dd> (default)
NB. 1 - dates in form <yyyymmdd>
NB. 0 = todayno 1800 1 1, or earlier
NB.
NB. example:
NB. todayno 1998 5 23
NB. 72460
todayno=: 3 : 0
0 todayno y.
:
a=. y.
if. x. do. a=. 0 100 100 #: a end.
a=. ((*/r=. }: $a) , {:$a) $,a
'y m d'=. <"_1 |: a
y=. 0 100 #: y - m <: 2
n=. +/ |: <. 36524.25 365.25 *"1 y
n=. n + <. 0.41 + 0 30.6 #. (12 | m-3),"0 d
0 >. r $ n - 657378
)
Did not find something to use exactly from Oleg's "sfl" addon (see his website [2]). This could form the basis of a more thorough version of "DateAdd".
NB. see http://www.imatix.com/html/sfl/index.htm for detailed description NB. J Script template NB. Wednesday, May 17, 2000 NB. SFL dates routines require jpath '~addons\sfl\sfl.ijs' coclass'jsfl' date_now=: ' date_now + i' cdm NB. (void); time_now=: ' time_now + i' cdm NB. (void); leap_year=: ' leap_year + i i' cdm NB. (int year); julian_date=: ' julian_date + i i' cdm NB. (long date); day_of_week=: ' day_of_week + i i' cdm NB. (long date); week_of_year=: ' week_of_year + i i' cdm NB. (long date); year_quarter=: ' year_quarter + i i' cdm NB. (long date); next_weekday=: ' next_weekday + i i' cdm NB. (long date); prev_weekday=: ' prev_weekday + i i' cdm NB. (long date); pack_date=: ' pack_date + i i' cdm NB. (long date); pack_time=: ' pack_time + i i' cdm NB. (long time); unpack_date=: ' unpack_date + i i' cdm NB. (word packdate); unpack_time=: ' unpack_time + i i' cdm NB. (word packtime); default_century=: 'default_century + i i'cdm NB. (long *date); date_to_days=: 'date_to_days + i i' cdm NB. (long date); days_to_date=: 'days_to_date + i i' cdm NB. (long days); date_to_timer=: ' date_to_timer + i i i'cdm NB. time_t (long date, long time); timer_to_date=: ' timer_to_date + i i' cdm NB. (time_t time_secs); timer_to_time=: ' timer_to_time + i i' cdm NB. (time_t time_secs); timer_to_gmdate=: 'timer_to_gmdate + i i' cdm NB. (time_t time_secs); timer_to_gmtime=: 'timer_to_gmtime + i i' cdm NB. (time_t time_secs); time_to_csecs=: ' time_to_csecs + i i' cdm NB. (long time); csecs_to_time=: ' csecs_to_time + i i' cdm NB. (long csecs); future_date=: ' future_date + i *i *i i i'cdm NB. (long *date, long *time, long days, long csecs); past_date=: ' past_date + i *i *i i i'cdm NB. (long *date, long *time, long days, long csecs); date_diff=: ' date_diff +i i i i i *i *i'cdm NB. (long date1, long time1, long date2, long time2, long *days, long *csecs); valid_date=: ' valid_date + i i'cdm NB. (long date); valid_time=: ' valid_time + i i'cdm NB. (long time); date_is_future=: ' date_is_future + i i i'cdm NB. (long date, long time); date_is_past=: ' date_is_past + i i i'cdm NB. (long date, long time); timezone_string=: ' timezone_string + *c'cdm NB. (void); local_to_gmt=: ' local_to_gmt + i i i *i *i'cdm NB. (long date, long time, long *gmdate, long *gmtime); gmt_to_local=: ' gmt_to_local + i i i *i *i'cdm NB. (long gmdate, long gmtime, long *date, long *time); NB. ========================================================= datenow=: first@ date_now timenow=: first@ time_now leapyear=: first@ leap_year juliandate=: first@ julian_date dayofweek=: first@ day_of_week weekofyear=: first@ week_of_year yearquarter=: first@ year_quarter nextweekday=: first@ next_weekday prevweekday=: first@ prev_weekday packdate=: first@ pack_date packtime=: first@ pack_time unpackdate=: first@ unpack_date unpacktime=: first@ unpack_time datetodays=: first@ date_to_days daystodate=: first@ days_to_date timetocsecs=: first@ time_to_csecs csecstotime=: first@ csecs_to_time validdate=: first@ valid_date validtime=: first@ valid_time dateisfuture=: first@ date_is_future dateispast=: first@ date_is_past timezonestring=: memr0@first@ timezone_string
Catalog Obverse
Some interesting forum traffic on the catalog verb and its obverse.
Date: Mon, 14 Feb 2005 13:09:03 -0500
From: "Roger Hui" <rhui000@SHAW.CA>
Subject: Re: [Jforum] Inverse of Catalogue
To: JFORUM@PEACH.EASE.LSOFT.COM
x=: (2 3$'abcdef');'ghij'
y=: catalogue x
uncatalogue y
|index error: uncatalogue
| uncatalogue y
On Sun, 13 Feb 2005 18:46:21 -0500, Mark D. Niemiec <mniemiec@INTERSERV.COM>
wrote:
>J does not provide an inverse of { Catalogue, but it is an easy operation
to perform:
>
> uncatalogue =: (>`(i.@#@$<@([{&>({:@$$,)@|:)"0 _])@.(*@#@$))
> catalogue =: { :. uncatalogue
>
> [ y =: 'ht';'ao';'gtw'
>+--+--+---+
>|ht|ao|gtw|
>+--+--+---+
. . .
An idea for the inverse of catalog.
Date: Sun, 13 Feb 2005 18:46:21 -0500
From: "Mark D. Niemiec" <mniemiec@INTERSERV.COM>
Subject: [Jforum] Inverse of Catalogue
To: JFORUM@PEACH.EASE.LSOFT.COM
J does not provide an inverse of { Catalogue, but it is an easy operation to
perform:
uncatalogue =: (>`(i.@#@$<@([{&>({:@$$,)@|:)"0 _])@.(*@#@$))
catalogue =: { :. uncatalogue
[ y =: 'ht';'ao';'gtw'
+--+--+---+
|ht|ao|gtw|
+--+--+---+
[ caty =: { y
+---+---+---+
|hag|hat|haw|
+---+---+---+
|hog|hot|how|
+---+---+---+
+---+---+---+
|tag|tat|taw|
+---+---+---+
|tog|tot|tow|
+---+---+---+
uncatalogue caty
+--+--+---+
|ht|ao|gtw|
+--+--+---+
|.&.catalogue y
+--+--+---+
|th|ao|gtw|
+--+--+---+
|."_1&.catalogue y
+--+--+---+
|ht|oa|gtw|
+--+--+---+
|."_2&.catalogue y
+--+--+---+
|ht|ao|wtg|
+--+--+---+
-- Mark D. Niemiec <mniemiec@interserv.com>
Beginner's Session
We had a computer available so we showed how to answer some questions people had.
NB. [Session started: 2005 2 8 20 29 33.137]
NB. Jim has a matrix of boxed items he wants to convert to plain
NB. ASCII (including the boxing display characters).
nn=. 3 3$'';'x';'y';'min';'3';'2';'max';'9';'10'
nn
+---+-+--+
| |x|y |
+---+-+--+
|min|3|2 |
+---+-+--+
|max|9|10|
+---+-+--+
NB. He has only to do this:
":nn
+---+-+--+
| |x|y |
+---+-+--+
|min|3|2 |
+---+-+--+
|max|9|10|
+---+-+--+
NB. Looks the same but the shape tells us it isn't:
$ ":nn
7 10
NB. I would probably put it a more common format (e.g. .CSV):
matrix2csv nn
"","x","y"
"min","3","2"
"max","9","10"
NB. Example of standard function to use to split, e.g. headers from data:
split 1 2 3 4
+-+-----+
|1|2 3 4|
+-+-----+
2 split 1 2 3 4
+---+---+
|1 2|3 4|
+---+---+
NB. It works on items, so:
split nn
+------+----------+
|++-+-+|+---+-+--+|
|||x|y|||min|3|2 ||
|++-+-+|+---+-+--+|
| ||max|9|10||
| |+---+-+--+|
+------+----------+
NB. In discussing the difficulties of handling .CSV files efficiently,
NB. particularly the overhead of checking whether a cell is numeric or
NB. not, Dan suggests grouping entire columns thusly:
d=.5$,:4;'hello';'devon';9j9
d NB. We have mixed cells but columns
+-+-----+-----+---+ NB. have homogenous types.
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
NB. So, group elements of columns together.
tocol=.<@:>"1@:|:
tocol d
+---------+-----+-----+-------------------+
|4 4 4 4 4|hello|devon|9j9 9j9 9j9 9j9 9j9|
| |hello|devon| |
| |hello|devon| |
| |hello|devon| |
| |hello|devon| |
+---------+-----+-----+-------------------+
NB. My own, much-used, matrix-to-CSV-file function.
m2c
4 : 0
NB.* m2c: Matrix x. to .CSV file y.
dlim=. '' [ writeInPieces=. 0 NB. Default to write as single piece.
if. 1=L. y. do. 'writeInPieces y.'=. y. end.
if. writeInPieces do. NB. Underlying function handles writing
(1;',') matrix2csv x.;<y. NB. in pieces.
else. NB. A "file number error" here
(matrix2csv x.) 1!:2 <y. NB. may indicate a held file.
end.
NB.EG data m2c filename
NB.EG data m2c 1;<filename NB. Write file in pieces as we go to save space.
)
NB. and its inverse:
c2m
3 : 0
',' csv2matrix 1!:1 <y.
:
x. csv2matrix 1!:1 <y.
)
csv2matrix
3 : 0
NB.* csv2matrix: change contents of .CSV file -> matrix of items from
NB. Comma-Separated-Values file y. Left arg is delimiter if not comma.
NB. Handles quoted strings.
',' csv2matrix y.
:
if. x.-:'' do. delim=. ',' else. delim=. x. end.
if. 0=L. y. do. vec=. l2v y. NB. Box lines if simple vec ->
else. vec=. y. end. NB. vector of text lines.
mat=. matMakeMaskedPtn vec;delim;<'"'
mat=. mat-.&.>'"' NB. Remove quotes.
)
NB. Example of using to read in .CSV file->J boxed matrix:
$fl=. c2m '\gtaap\gaa\data\rawdatfl.csv'
365 101
5{.fl
+----------+-----------------------------------+----------------------...
|Name |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (...
+----------+-----------------------------------+----------------------...
|Code |BRITPUS |JAPYNUS ...
+----------+-----------------------------------+----------------------...
|Currency |£ |Y ...
+----------+-----------------------------------+----------------------...
|12/31/1974|#n/a |#n/a ...
+----------+-----------------------------------+----------------------...
|1/31/1975 |#n/a |#n/a ...
+----------+-----------------------------------+----------------------...
NB. Split off the header from the data:
'hdr fld'=. 3 split fl
hdr
+--------+-----------------------------------+------------------------...
|Name |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (GT...
+--------+-----------------------------------+------------------------...
|Code |BRITPUS |JAPYNUS ...
+--------+-----------------------------------+------------------------...
|Currency|£ |Y ...
+--------+-----------------------------------+------------------------...
3{.fld
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|12/31/1974|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|68....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|1/31/1975 |#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|76....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|2/28/1975 |#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|81....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
'hdr fld'=. tocol each 3 split fl
hdr
+--------+-----------------------------------+------------------------...
|Name |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (GT...
|Code |BRITPUS |JAPYNUS ...
|Currency|£ |Y ...
+--------+-----------------------------------+------------------------...
fld
...
|9/30/1986 |1.44800|0.00647999|0.49350|0.60730|0.150700|0.43670|0.6284...
|10/31/1986|1.40650|0.00612600|0.48480|0.58430|0.148700|0.42940|0.6417...
|11/28/1986|1.43650|0.00617899|0.50710|0.60940|0.154900|0.44880|0.6508...
|12/31/1986|1.48300|0.00632900|0.52030|0.62230|0.157000|0.46020|0.6655...
|1/30/1987 |1.51350|0.00651499|0.54660|0.64810|0.163900|0.48600|0.6622...
|2/27/1987 |1.55500|0.00652500|0.54670|0.64850|0.164100|0.48480|0.6805...
|3/31/1987 |1.60750|0.00685899|0.55430|0.66310|0.166500|0.49040|0.7062...
|4/30/1987 |1.66150|0.00710699|0.55800|0.68100|0.167100|0.49320|0.7020...
|5/29/1987 |1.62950|0.00695200|0.54840|0.66230|0.164300|0.48690|0.7123...
|6/30/1987 |1.61750|0.00681200|0.54800|0.66050|0.164200|0.48700|0.7211
...
NB. Example of converting text to numbers with an error flag of _99:
_99 ". '1 2 3 4'
1 2 3 4
_99 ". '1 2 3 4'
1 2 3 4
_99 ". '1 2 3 4 crap'
1 2 3 4 _99
NB. My own (external format) Number to J number function:
n2j
_&".@(-.&'+')
NB. and its inverse:
j2n
'S'&J2StdCvtNums
J2StdCvtNums
3 : 0
NB.* J2StdCvtNums: convert char rep of num from J to "Standard", or
NB. vice-versa if left arg is 'J' or '2J'; optional 2x2 left argument allows
NB. 2 arbitrary conversions: col 0 is "from", col 1 is "to" char.
NB. Monadic case changes Standard numeric representation to J representation.
(2 2$'-_Ee') J2StdCvtNums y.
:
pw16=. 0j16 NB. Precision width: 16 digits>.
diffChars=. 2 2$'-_Ee' NB. Convert '-'->'_' & 'E'->'e'
toStd=. -.'J'-:''$'2'-.~,x. NB. Flag conversion J->Standard
if. 2 2-:$x. do. diffChars=x. NB. if explicit conversion.
elseif. toStd do. diffChars=. |."1 diffChars end. NB. Convert other way
NB. if. 0=1{.0$y. do. y.=. pw16":y. end. NB. Numeric to character
if. 0=1{.0$y. do. NB. Numeric to character
fmts=. (8=>(3!:0)&.>y.){0,pw16 NB. Full-precision floats only
NB. whvn=. isValNum &.> y.
NB. tty=. fmts":y. NB. If this is too slow, go back
y.=. fmts":y. NB. If this is too slow, go back
NB. y.=. whvn}tty,:y. NB. If this is too slow, go back
NB. y.=. pw16":y. NB. to this.
end.
y.=. y.-.'+' NB. EG 1.23e+11 is ill-formed & the
wh=. y.=0{0{diffChars NB. '+' is unnecessary.
cn=. (wh#1{0{diffChars) (wh#i. $y.)}y. NB. Translate chars that need it
wh=. y.=0{1{diffChars NB. but leave others alone.
cn=. (wh#1{1{diffChars) (wh#i. $cn)}cn
if. -.toStd do. NB. Special handling -> J nums
if. '%'e. cn do. NB. Convert nn% -> 0.nn
cn=. pw16":0.01*".cn-. '%'
end.
cn=. cn-.',' NB. No ',' in J numbers
end.
cn
NB.EG 'S' J2StdCvtNums _3.14 6.02e_23 NB. Convert J numbers to std rep
)
matrix2csv
3 : 0
NB.* matrix2csv: create .CSV file from matrix of items y.
NB. Inverse of csv2matrix but doesn't write to a file; does quote literals.
NB. Optional left arg is (scalar) field separator character.
NB. Do large mats in pieces to avoid using up memory.
NB. Optional flag 0{x. is 1 if we are to write file instead of just
NB. creating result to write to file.
',' matrix2csv y.
:
ctr=. i. 0 [ csvfl=. '' [ doWrite=. 0
if. isEnclosed x. do.
'doWrite x.'=. x.
'y. flnm'=. y.
end.
y.=. (_2{.1 1,$y.)$,y. NB. Ensure is a matrix.
if. 0>4!:0 <'flnm' do. flnm=. 'TEMP.TMP',~getTempDir '' end.
flnm=. boxopen flnm
NB. Estimate Number of Rows Per Iteration: was a global.
NRPI=. 100>.1000*<.0.5+0.001*(#y.)%(7!:5 <'y.')%15e6 NB. 15e6 for 256MB mem.
numrows=. NRPI<.#y.
if. doWrite do.
'' 1!: 2 flnm NB. Initialize file because we'll append later.
end.
while. 0 < # y. do. NB. Do in pieces if too large.
mat=. numrows{.y.
whchar=. ' '=&.>1{.&.>0$&.>mat NB. Where non-num (=char) fields?
if. 1 e. ,whchar=. ;,whchar do.
quotes=. ($mat)$whchar#&.>'"' NB. Put quotes around char fields.
if. +./,>'"'e.&.>mat do. NB. Duplicate any quotes that
mat=. (>:&.>mat=&.>'"')#&.>mat NB. are part of data.
end.
mat=. quotes,&.>mat,&.>quotes
end.
if. 1 e. whnum=. ,-.whchar do. NB. Convert numerics to char so
shape=. $mat NB. can append field separator.
mat=. ]&.>,mat NB. Enclose if simple array.
mat=. (":&.>whnum#mat) (b2i whnum)}mat
mat=. shape$mat
end.
mat=. mat,&.>x.
mat=. ;(_1}."1 mat),.(_1}.&.>_1{."1 mat),&.><CR,LF
if. doWrite do.
mat 1!:3 flnm
else.
csvfl=. csvfl,mat
end.
y.=. numrows}.y.
numrows=. numrows <. # y.
end.
csvfl
NB.EG Either
NB.EG (matrix2csv datamat) 1!:2 <flnm
NB.EG or
NB.EG (1;',') matrix2csv datamat;<flnm
)
NB. Dan shows us the tree display form he finds handy.
NB. Start with a long train of constant functions:
g =: 1: 2: 3: 4: 5: 6: 7: 8: 9:
NB. Display it (first word in the namelist consisting only of 'g')
NB. with "5!:4":
5!:4{.;:'g'
+- 1:
+- 2:
--+ +- 3:
| +- 4:
+----+ +- 5:
| +- 6:
+----+ +- 7:
+----+- 8:
+- 9:
g
1: 2: 3: 4: 5: 6: 7: 8: 9:
NB. Try it with a more familiar train:
(+/%#)
+/ % #
g=. (+/%#)
5!:4{.;:'g'
+- / --- +
--+- %
+- #
5!:4{.;:'fetch'
-- {::
NB. Jim wants to know how to activate or execute a J expression given
NB. as a text string; specifically, how to assign a variable indirectly.
c=: dyad define
". x.,5!:5{.;:'y.'
)
NB. Function "c" applies J evaluation of x. to values y. :
'A=:' c 3 4 5 6
3 4 5 6
NB. Now "A" is 3 4 5 6.
A
3 4 5 6
;/ 3 4 5
+-+-+-+
|3|4|5|
+-+-+-+
A
3 4 5 6
NB. Left-hand side doesn't have to be assignment, can be any expression.
B =: '+/'
B c 3 4 5 6 7
25
3 ; 4; 5 ; 6
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+
C=:3 ; 4; 5 ; 6
C
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+
5!:5{.;:'C'
3;4;5;6
3;4;5;6
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+
Scan of Meeting Notes
