NYCJUG/2005-03-08

From J Wiki
Jump to: navigation, search

public domain licensing, persistence of information, "Save Work" project, APL/J Rosetta Stone, date conversions, design considerations, learning APL and similar languages


Agenda

0. Group project: legalities - draft agreement

1. Group project: report on assignments
    a. Harteg - initial screen
    b. Dan - persistence of information
    c. Jim - library report
    d. John - thoughts on desired features
    e. Devon - design considerations - see topic 2 below.

2. Assignments: going forward - what next?
    a. Extreme programming -> unit tests _before_ code.
       Can we combine screen persistence with automating user input for
       testing?
    b. Integration - use J projects?  something else?

3. Learning and creating learning materials: a general framework
   How to structure it?  As a FAQ?  Hypertext?
   Suggested directory structure: what are main categories?
   e.g.
       Main Project
       Utilities
       Design documents (data dictionary?)
       Documentation (Chris Lett)
       Feedback to J Software
       Web site, public release?

4. Howard Peele's talk - date and format?
 "Best times for me seem to be: end of April, end of May, or early June.
   Also:
  Who is your audience? How many would you expect?"

Bonus Facts:

NB. The significance of the number 21 in a 21-gun salute is derived
NB. from adding the digits of 1776.
   +/1 7 7 6
21

NB. A quick check I use to test the validity of a prime-number generator:
   -/p: i.499
1776

Proceedings

[Everyone at the meeting received a letter from me a few days before outlining goals and establishing a framework for our group project. I'll point to it here as a separate document as it is rather lengthy: [1]

All present agreed that the group project should be released as a public domain work. We looked at some example agreements such as the "GNU General Public License" (File:Gpl.html) and a draft for us based on licenses with similar objectives (File:GroupAgreement.txt). We also considered the open-source licenses (File:OpenSource cpl1.0.php.html) but are wary of any restriction that would prevent any of us from commercializing this work if any of us want to. Here is an overview of open source licensing: File:OverviewOfOpenSourceLicenses opensource.html.

We agreed that we wanted to protect our use of any group code so that no one could license it and forbid us to use our own code.

Dan has been working on the "persistence of information" problem. Since the last meeting, I've spent some time perusing his "lexical closure" code - I still don't know what this means - but it allows you to handle name-value pairs specified in a file. The values look to be arbitrarily complex J data structures though maybe better suited to generating data than storing arbitrary data? I'm not sure about this.

We talked a little about being able to have config files with partial data if necessary. Dan's code (which he may have distributed the meeting before) has a nice default value mechanism which would be useful here.

"Save Work" Project

Where we want to go with this: we want software that will naturally conform itself to how each particular user usually uses it. Using the "save work" example, a user might have one or two or more typical, customized backup scenarios that are run at different frequencies. So, a quick, every-day backup should perhaps exclude certain large database files whereas a weekly, fuller backup would include these.

We had discussed an average user to whom the bulk of our interface would be targeted. I'm basing this on a composite of some friends of mine who use computers but are not computer experts. Our primary user is called "Vicki".

There was some speculation that the only things an average user, our "Vicki" perhaps, would ever back up are in her "My Documents" directory. This also raised the question of whether people know, or should have to know, where their files are.

Reports from Members

Jim reported on finding many more utility libraries than he knew existed before. We still lack principles on which to organize this. There has been discussion on the J Forum over the past few days on this topic - see the note from Neville Holmes "J for Beginners" on 4/4 and Raul D. Miller's reply on 4/5. Jim has suggested putting in a mention of our work here - maybe a little later.

Jim also provided a draft of a "Rosetta Stone": in this case, a rough translation between APL and J: File:APLJ Rosetta.rtf.

Also, let's not forget Chris Lett who, while not able to attend meetings in person, has graciously offered to document whatever we come up with.

John spent some time looking at the "parseDir" code I'd provided and had some good suggestions for organizing the hodge-podge of date conversion functions I've evolved over time. He'll look at something using regular expressions to parse dates in different formats.

Project Design

I spoke a little about design considerations - I'd like to incorporate the "Extreme Programming" idea of unit tests before code - well, maybe alongside code. Dan's lexical closure code provides an example of something of what I'd be looking for in his example section at the end of the code. I'm more interested in working examples than comprehensive code coverage. Dan mentioned how Roger Hui writes test scripts for J itself using assertions.

We looked at a couple of essays from Paul Graham on what good design means: [2] and [3]

We didn't agree on whether to use the J projects facility or not - does anyone have any experience with this?

When we talked about a general framework for learning and creating learning materials, Dan mentioned YAML - Yet Another Meta-Language - as a possibility for specifying complex data structures. We probably also need to explore the various options of keyfiles vs. APL workspaces vs. 5!:5 (linear representation) or 3!:5 (internal J representation) for storing state and work in progress. We may have also looked at something called a "Wiki" as a way to pool our work: File:WikiTool.ps.

I showed the group a type of file-directory linking that the OS (at least XP and NT) knows about: if you copy the file, the associated directory automatically gets copied as well. I don't know if this is really useful or just something that breaks cross-OS compatibility.

Learning APL and Similar Languages

There was an interesting exchange on the J Forum recently talking about understanding code and the relation between human language and computer language comprehension.

It looks like the thread was started by this response to an earlier message:

> From     Alain Miville de Chêne <amd@INFODEV.CA>
> Date     02/22/2005 07:54 AM
> To       JFORUM@PEACH.EASE.LSOFT.COM
> Subject Re: [Jforum] inter J compiler : formal and informal names
>
> Alan Stebbens wrote:
> ...
>> So, which is easier to parse, read and comprehend?
>>
>>    a=.,.<;._1 ' ',"$123.45 $345.67 ($99.99)"
>>
>> or
>>
>>    ravel box cut ' ','$123.45 $345.67 ($99.99)';
>>
>  or
>
> a=. ,. < ;._1 ' ', '$123.45 $345.67 ($99.99)'
>
> make it stand out visually. The ;._1 is explicitely shown as a unit.

This prompted other responses and provoked some reminisces such as the following.

In reply to this message from Don Guinn:

Don Guinn <donguinn@PVTNETWORKS.NET>02/22/2005 05:11 PM
To JFORUM@PEACH.EASE.LSOFT.COM
Subject Re: [Jforum] inter J compiler : formal and informal names
...
What I meant was that at first the compactness of a J definition is a
little overwhelming until I get into it.  Where the Basic code is so
verbose that I can't see but a little piece at a time so it feels like I
am accomplishing something even though I am actually moving very
slowly.  It's kind of like going 70 miles an hour in a sports car where
you are close to the ground seems fast, where the same speed in a
motorhome seems slow because you are farther from the road and can see
so much more.  By the way, in a motorhome freeways look like country
roads!  Could this be a parallel that in J or APL we think of solving
some problem as almost trivial where solving that same problem in Basic
or C as a major undertaking?
...

Mr. Gauthier wrote:

Date:	Tue, 22 Feb 2005 18:09:33 -0600
From:	PGauthier@CHECKFREE.COM  Add to Address BookAdd to Address Book
Subject:	Re: [Jforum] inter J compiler : formal and informal names
To:	JFORUM@PEACH.EASE.LSOFT.COM

I started with APL in 1974.

I was always stunned, every time I touched upon other languages, to see
people loving the contortions they had to go through to do anything more
than simple "demo style" programming.

So to me it's the other way around, I kept asking myself the question:
"But what is it they enjoy in XXX language ?" It would be so simple to do
it in APL/J.

I did find good reasons to take very specific pieces of extensively used
code and shovel it in an AP for obvious performance/scalability reasons.
Thus programming in "C" or machine language from time to time (after
having prototyped it in APL/J).

Now, when I hear/read about the difficulties to read/understand APL/J I
usually can classify it in two broad categories:

A- The person is not a native speaker and it's not his/her first exposure
to a programming language but has a genuine need for APL/J.
B- The person is not looking at understanding/resolving problems by
finding their primitives and would rather have a "point and click" RAD.

Those in the A category need to "bite the bullet" and start developing
their "native speaking" with the books/examples available at
www.jsoftware.com.

Those in the B category will probably not find a lot of motivation using
APL/J.

In other walks of life I am the person A and sometimes the person B.

Interest/motivation vary considerably from one person to the next. It
maybe so that VB or some other variations is the tools you need. My
suggestion is to try to focus on some basic questions:

- Do I need to understand and resolved problems in a complete and general
way ?
- Do I long for a clear, not ambiguous tool of though to help me resolve
problems ?
- Are the problems I want to resolve already resolved ?
- Do I want to invest a sizeable chunk of my time in the acquisition of a
mathematical nomenclature that will help me resolve/understand problems ?
- What will I gain from learning a general mathematical notation which is
executable on a computer ?
etc.

To conclude, there is one thing I am sure of for myself. APL/J are very
easy/simple to read and developing with them is a breeze and has been for
30 years. When I have a need to understand a "new concept" I usually try
to find some "top gun" code and "bite the bullet".

BTW, J Phrases are like good exercises to master (after you have the
basics covered).

Whatever it is you are looking for, I wish you find it.

Paul

Raul Miller replied on the question of intrinsic ease of understanding code:

Date:	Tue, 22 Feb 2005 18:29:19 -0500
From:	"Miller, Raul D" <rdmiller@USATODAY.COM>
Subject: Re: [Jforum] inter J compiler : formal and informal names
To:	JFORUM@PEACH.EASE.LSOFT.COM

I think the issue of how "easy" or "hard" it is to read someone else's
code depends on a lot of factors.

One factor is: how well was the code written?

Another factor is: how well do you understand the problem being solved?

Another factor has to do with your expectations: what do you expect to
get out of reading the code.

Just for fun, compare j's dot product with am analogous expression
written in sql.

J:

dot=: +/ .*

SQL:

select
		t1.i,
		t2.k,
		sum(t1.v*t2.v) as v
	from
		t1
		inner join t2
			on t1.j=t2.j
	group by
		t1.i,
		t2.k

The J expression is more general, but for some people the sql expression
is "simpler to understand".  Why?  Does the sql expression give greater
insight into the problem?  In my opinion: not really.

But the sql expression does nail down some specifics.  You know, looking
at the sql expression that you're dealing with two specific
two-dimensional tables.  And, you know some of the column names.  You
don't really get a better sense of what's going on, but you know a bunch
of details, and that makes some people more comfortable.

Of course, you could have used a different J expression:

t1 +/ .* t2

This is closer to the sql expression, since it's now specific to two
tables.  But in J t1 and t2 could be anything -- at some point, t1 could
be a scalar, later, it could be a vector, then a rank 2 array, then rank
3, ...  SQL also has some flexibility, but not it's not flexible like J.

Or, you could write something like this in VB.  I'm not comfortable with
VB, so I won't attempt it, but I'm pretty sure the code would be fairly
concrete (like the sql expression) and fairly verbose (like the SQL
expression).  And, again, there would be lots of little details to latch
on to, that lets a person glibly say "I understand that", even if they
don't really know the architecture or the design of the code very well.

Alternatively, when someone is talking about "easy" vs. "hard", they can
mean many different things.

--
Raul

Bill Harris had this to say, comparing the effort of understanding common language to the effort of understanding code.

Date:	Tue, 22 Feb 2005 15:08:06 -0800
From:	"Bill Harris" <bill_harris@FACILITATEDSYSTEMS.COM>
Subject: Re: [Jforum] inter J compiler : formal and informal names
To:	JFORUM@PEACH.EASE.LSOFT.COM

Stuart Bruff <stuartbruff@YAHOO.CO.UK> writes:

> I didn't see this as expressing a  'preference', just a comment on the
> degree of mental concentration required to interpret a J statement.  J
> is akin to translating from Russian or Japanese to English and then
> trying to understand the content.  VB is akin to translating from a
> local dialect, much less mental effort involved and it's tantamount to
> a single mental task.  Once you're fluent in a language, the
> "translation" effort drops off and it becomes easier to read and
> understand, but it's pretty wearing if you're a novice or not familiar
> with the specialist vocabularies.

Stuart,

I think you nailed it.  When I moved to Germany to work (years back), I
had studied German in high school and college, but my process for
reading, writing, hearing, or speaking involved translating from one
language to another.  The first few months were really tough mentally --
I'd go home at night with headaches from working actively and only in
German (i.e., translating) for 8+ hours, but I kept to two principles:
only deal in German (i.e., refuse to deal in English unless I had to)
and look up new words and grammatical constructions on a regular basis.
The incentive?  Besides my internal drive, speaking German was a
necessity for work and for basic life tasks (eating, etc.); the second
language where I lived was French, not English).

After no more than 6 months, I discovered I was dreaming in German.
While I was always reluctant to call myself fluent (reserving that for
those who could pass as a native speaker) and my grammar didn't have a
similar step-function improvement, I found I now was suddenly able to
express and assimilate thoughts directly in German without any
translation.  I could keep up with others in speaking and listening.
When I later moved back to the USA, I found that, for years, I'd
naturally think about subjects I engaged in deeply in Germany in German
first and then translate them to English.

I suspect (without proof) I would have learned German earlier had I been
exposed to it earlier in what I always knew of as the Berlitz total
immersion method: no grammar, just talking and listening in German all
day long.

I think that's what the standard advice for learning J is all about:
grab your computer, and use the Primer and other tools to immerse
yourself in a dialog in J with your computer.

It's hard, it's frustrating (by comparision, there were times I really
wanted to speak English for a bit, but I held out), and it's eminently
rewarding when your mind does that flip and J becomes natural.

I think that's what the Morse code Web page was trying to convey.  I
think that's what my experience learning German was all about: reading
more grammar books, memorizing rules, and learning more vocabulary would
never have gotten me to where I needed to be; only forcing myself to
think in German by prohibiting me from thinking in English got me there.
I think that's how I learned the Dvorak keyboard: switching it over and
then only typing with the new layout (for the first 2 weeks, I prayed
for emails I could answer with "Yes" or "No" so I could type my
responses within 5 minutes! :-).

For me, the hard thing about J was finding time _in front of my
computer_ to have that dialog.  Lacking a handheld computer, I was
tethered to a laptop or worse, and I was used to learning technical
information from the printed page.  I'm confident that my problems with
J today are a result of not having been consistent enough in only using
J to learn it.  (My singular focus on immersion stops after
internalizing something; I think it's very beneficial to be able to use
multiple approaches, be it Dvorak/Qwerty, J/C, English/German, ....)

As for the suggestions for syntax sugar for J (spaces around tokens, for
example), what about ;:?

Bill
- --
Bill Harris                                  3217 102nd Place SE
Facilitated Systems                          Everett, WA 98208 USA
http://facilitatedsystems.com/               phone: +1 425 337-5541

Even though this repeats some of a messages included above, for completeness, I'll include the whole of Don Guinn's response to Stuart Bruff, where he raised the issue of the virtues of a compact notation.

Date:	Tue, 22 Feb 2005 16:11:04 -0700
From:	"Don Guinn" <donguinn@PVTNETWORKS.NET>
Subject:	Re: [Jforum] inter J compiler : formal and informal names
To:	JFORUM@PEACH.EASE.LSOFT.COM

Actually about my remark on reading Basic vs reading J or APL.  I find
it much easier to read J or APL than Basic.  Working with Basic I feel
that I have one hand tied behind my back.  So many things are almost
there that it is frustrating.  And Microsoft manuals repeat themselves
so much from page to page that I find myself recognizing and skipping
sections after I have seen them twenty or so times.  I can go through a
two inch Basic manual in an afternoon.  Because I have skipped so much,
what have I missed?

What I meant was that at first the compactness of a J definition is a
little overwhelming until I get into it.  Where the Basic code is so
verbose that I can't see but a little piece at a time so it feels like I
am accomplishing something even though I am actually moving very
slowly.  It's kind of like going 70 miles an hour in a sports car where
you are close to the ground seems fast, where the same speed in a
motorhome seems slow because you are farther from the road and can see
so much more.  By the way, in a motorhome freeways look like country
roads!  Could this be a parallel that in J or APL we think of solving
some problem as almost trivial where solving that same problem in Basic
or C as a major undertaking?

Stuart Bruff wrote:

>> BTW if you prefer looking at 5 pages of VB rather than a single J
>> line, it
>
> looks to me as if the power and terseness of J is eluding you, maybe the
> initial barrier is higher but so rewarding ;O)<
> =Funny thing.  I found it not unreasonable spending an hour or so reading
> several pages of Basic code to understand what it did, but I have
> trouble spending five minutes reading a J statement which does the same
> thing.=
>
> Not wishing to speak for Don, but merely as my interpretation of what
> he wrote ....
>
> I didn't see this as expressing a  'preference', just a comment on the
> degree of mental concentration required to interpret a J statement.  J
> is akin to translating from Russian or Japanese to English and then
> trying to understand the content.  VB is akin to translating from a
> local dialect, much less mental effort involved and it's tantamount to
> a single mental task.  Once you're fluent in a language, the
> "translation" effort drops off and it becomes easier to read and
> understand, but it's pretty wearing if you're a novice or not familiar
> with the specialist vocabularies.
>
> As an aside, I believe that Kanji has some similarities to J, in that
> a symbol can have different meanings depending on whether it's stand
> alone or adjacent to other symbols (Of course, not being a Japanese
> speaker, I may be completely wrong :-\ ).  I would be interested to
> hear the opinions of the Japanese J users on how easy J is to pick up
> or specific learning problems.
>
> Stuart
>
> ----- Original Message ----- From: <PGauthier@CHECKFREE.COM>
> To: <JFORUM@PEACH.EASE.LSOFT.COM>
> Sent: Tuesday, February 22, 2005 6:28 PM
> Subject: Re: [Jforum] inter J compiler : formal and informal names
>
>
> Personally, my favorite is:
>
> b=.(<;._1)"_ 1(L: 0)
> v=.' ', '$123.45 $345.67 ($99.99)'
> m=.>3#<v
> c=.2 3$<m
>
>   $b v
> 3
>   $b m
> 3 3
>   $b c
> 2 3
>
> NB: I had to read other people code in both APL and J and never found it
> difficult to the degree you described.
>
> What I did notice is that some people are better designers than others
> and their code is not only easier to understand but also better
> modularized(put into verbs/adverbs/conjunctions) by more useful
> modules in the context.
>
> Syntax coloring could help but we usually deal with so few at any one time
> that inserting spacing in a piece of code is not an incommensurable task.
>
> BTW if you prefer looking at 5 pages of VB rather than a single J
> line, it looks to me as if the power and terseness of J is eluding
> you, maybe the initial barrier is higher but so rewarding ;O)
>
> Paul

Scan of Meeting Notes

NYCJUGMeetingNotes20050308 50.jpg