Wiki/Report of Meeting 2024-02-15
Report of Meeting 2024-02-15
Present: Ed Gottsman, Raul Miller, and Bob Therriault
Full transcripts of this meeting are now available below on the this wiki page.
1) Ed has radically improved the search capabilities of J Viewer and this has eliminated the restrictions on the number of hits that are returned for a query. Also, that the jsoftware GitHub hits are prioritized to the top of the GitHub and followed with J scripts from other GitHub sites. Whitespace can create issues with code search since whitespace can not be generally removed as there are times when it is important. Ed is still working on including the new google groups forums in the J viewer. Archives prior to January 1st 2024 are included for the J Viewer. Bob noted that the browser window is occasionally slow and this could be due to a variety of reasons including the JQt framework and Internet connectivity. Bob had noted that the general timer was used in the J Viewer. He found this by doing a JQt demo which affected timers and had some effects on the viewer. Ed will switch over to ptimer which will limit the timer to the J Viewer locale and should isolate it from the General timer.
2) Bob had a quick look at the CSS grids and these look like a good solution for some of the more complicated tables. This should mean that it will be easier to make changes to the wiki since the markup for complex tables like NuVoc can be very difficult to navigate. Raul had been doing some work with CSS Grid on his 9!: page https://code.jsoftware.com/wiki/Vocabulary/Foreigns/9. Bob showed the Welcome to J page https://code.jsoftware.com/wiki/Category:Newcomers_N where he had moved the table of contents up higher on the page to make it more accessible to the viewer. Bob also removed the question mark that was linked to a MediaWiki site. The link was removed using CSS on the page. This should be done for all of the category pages except possibly for selected page involving contributions to the wiki.
3) Bob discussed the concern that Raul had raised in the previous meeting that the content should provide more information to the user to facilitate navigation.https://code.jsoftware.com/wiki/Category:Community_C Raul felt that this would be a step in the right direction and it may require a solution that involves a number of tactics.
4) Bob had filled in missing pages by creating pages with a link to the addons' GitHub source https://code.jsoftware.com/wiki/Addons/format/zulu-bare . This was a stop gap and may still require additional information on the linking page so that the user does not have to dive into GitHub to get a general sense of how appropriate the page is for their needs.
5) Raul has made some progress on his part of the X-Ray project. Raul then showed the work that he had done at https://code.jsoftware.com/wiki/Vocabulary/Foreigns/9 using the CSS Grid method.
6) A discussion unrelated to the wiki followed. This concerned the X-Ray project to get visual insight into J sentences.
For access to previous meeting reports https://code.jsoftware.com/wiki/Wiki_Development If you would like to participate in the development of the J wiki please contact us on the J forum and we will get you an invitation to the next J wiki meeting held on Thursdays at 23:59 (UTC) Next meeting is February 22, 2024.
Transcript
So what's the update on the JViewer?
Well, let me share my screen.
I will show you something nifty, which you may already know about.
So Raul has been trying to tell me for some time, and I've just been obtuse about it,
how to get better search performance out of SQLite, basically.
So I had a search limit of 200 results, irrespective of how many real results there were, and I
had an explanation and a sort of an apology.
And it turns out that if you do it right, you don't need to have that limitation.
Limitation was put in because of performance.
More than a couple of hundred results and everything just slowed to a crawl.
But now what it does, if you search for, let's say, slash, boom, 131,000 hits.
If you search for slash co, boom, 5535 hits.
The numbers we're getting in terms of the performance of the search, the slash was,
I think, 102 milliseconds, slash co was six milliseconds.
So it's pretty good.
You can, I mean, slash presumably returns all the documents that we've got, and it's
fast and scrolling just works.
And under the covers, what's happening here is that we do a search and we get back identifiers
for all the documents that matched.
And then as you scroll, we do additional queries just to return this, the current page worth
of results.
So it's faster.
The other thing is that by default for GitHub results, J software results are supposed to
sort to the top.
Are they doing that or not?
I didn't see that being implemented yet.
Last message you sent to me said that was still on your to-do list.
I think I sent you a follow-up message saying that I had too done it.
I thought so too, but I wasn't able to find that message when I was looking at it today.
So I think I know what was going on.
So here's a bunch of GitHub J software hits.
Why is it doing that?
It's really interesting.
It looks like it's pulling in results dynamically as you're browsing.
It is pulling in results dynamically.
Let me do this.
Let me just, let me use the new all or none switch.
Just turn on GitHub.
So here's slash co.
So I was searching for slash co slash co and that doesn't occur in J software apparently.
Unless that here it does though.
That's interesting.
It looks like I've still got some work to do.
If you just search for slash co, you do get J software hits initially.
In fact, that's mostly what you get.
And then eventually we should get to, yeah, so here's some hits that are non-J software
hits.
So that's the news.
I'm doing something wrong with my screen updates here as I'm scrolling.
There's this sort of, occasionally there's this sort of wave effect of line updates that
I'm not entirely understanding.
Are you, are the search results that you requested sort in the same order that you're displaying
them?
No.
That would be a problem with dynamic update.
If I, well, no.
I mean, if I, when I search, I get a document ID.
I get the source code.
That is, was it a GitHub hit or a Rosetta code hit or what have you.
And I got a priority, which I may not ever describe to you.
So based on the source code, I sort the results that I got back from the query.
Now ideally I would tell SQLite to sort based on the source code and just get the results
sorted to begin with.
But that was the whole problem I was trying to solve.
Was it SQLite only recruits one index per table per query.
And I'm already using an index.
So I do the sorting manually.
It's something a relational database should do for me, but that was not performant.
Once I've got the IDs sorted, I then request them on the fly just enough to fill up whatever
the current page is.
In other words, whatever the scroll bar happens to be.
Okay, so as long as you're not adding to the ID list after you've sorted, that shouldn't
be a problem.
I agree.
I think I may be doing something bad with the is a draw control.
I think I might not, I might be issuing a pay command early or something like that.
Something strange is happening.
I'm not, I don't think it's a database problem.
I think there's something else going on.
But this is all, by the way, it's just what I happen to be working with right at the moment.
Things are, things are good.
I am working with one gentleman who's trying to get it to work under OpenBSD, which turns
out to be a problem.
Bill Lam was kind enough to compile the SQLite binaries for OpenBSD.
But this user is running into, I think he's running into problems with the regex library.
We're still trying to track it down.
It is nice to have a log that you can turn on and see how far the application gets before
it falls over.
Well, worst case, there is that home, home, rude regex library that I slapped together.
Oh, true.
The gentleman's name is Benedict, Ben.
And he, on a couple of occasions, we've been going back and forth for a while, has said,
look, I don't want you to go to a lot of trouble.
There aren't a lot of OpenBSD users out there.
But I don't have anything else going on right now.
So I'm more than happy to help.
And I think that's basically it.
I have, I'm still not yet ready to confront the Google Groups archive problem that's going
to involve becoming a Google Cloud Platform developer, an amateur GCP developer.
And I'm not ready to take that leap yet.
But I'll get there.
And that's it.
Are there questions?
I guess just to reiterate, you got it working on the Mac platform, you got working on Windows,
and you've got working on Linux.
Linux, at least some Linuxes.
Yeah.
And it's the OpenBSD that's the only one that's your...
To my knowledge.
Yeah.
To my knowledge.
That's still really good.
That's the majority for sure.
There's Raspberry Pi, there's Android, there's iOS.
I don't know that this UI will work on any of those.
Well, it's JQT dependent.
So I think the answer to that is probably no.
Or QT dependent, I guess I should say.
I'd say JQT, but that's just because you're working with that instance of QT, right?
I'm not sure what the architectural diagram looks like, Bob, if you were to draw it out.
I'm not quite sure where all the dependencies are.
JQT has a dependency on QT.
And originally it was QT4, and then it was QT5.
And now they're trying to migrate to QT6, because each upgrade solves some problems
and creates new problems.
It was never thus.
That is the world of software.
And you got the big speed increase.
Is that because you weren't doing the sort on SQL?
You were doing the sort outside of SQL?
That's basically what it boils down to, Bob.
Yes.
Trying to do...
Nevermind sorting, just trying to do a query where you say, "Only give me Quora posts."
That would force a table scan of that 130,000 row document table, looking for Q for Quora
in the source column.
And it was just...
I actually didn't realize how bad it was until I dug into it a little bit, and I managed
to take 45 seconds to search for just GitHub posts in one of my queries.
It was embarrassing.
Well, yeah.
So I'm very happy.
And again, I thank Raoul to have found a way to address all of that.
Compared to the speed now, when you're talking about a tenth of a second is the slowest one.
Yeah, it's pretty consistently under 100 milliseconds, and it's often under 10 if you have a small
number of results.
So I'm pretty happy with it.
Yeah.
No, that's very quick.
I've noticed it's a bit slow to pull up pages, but that's more, I think, to do with my internet
connection and the server.
I would like to understand that myself.
Raoul has also observed that it's sometimes slow to bring up pages.
I mean, it used to be because it was stealing cycles from the WebView component, but it
doesn't, to my knowledge, doesn't do that anymore.
So I'm not really sure what the source of the...
There's a lot going on.
You know, there's DNS lookups, there's firewalls, there's caching, there's...
Sure, but why is it only sometimes?
I think it's only sometimes.
I mean, all of those are sometimes issues.
All of those involve a lot of servers and caching, and caching is, generally speaking,
a mechanism to deal with slow underlying servers.
Yeah.
That's true, certainly, but I never, or very, very, very seldom observe that kind of delay
in a browser, which presumably...
A conventional browser, which presumably has to deal with all of those same issues and
possibilities.
But it's also, conventional browsers have a lot of optimization to work around some
of these issues.
That is true.
I don't know how much...
I don't know how sophisticated the WebView is.
Yeah, that may actually be the question, right?
Because essentially, you're using the WebView for a browser, you're not using your regular
browser.
Right.
I mean, you can, and that actually is...
The main thing that's missing from the browser, and I haven't completely given up on this,
but is a find function.
So for GitHub hits, things are pretty good.
GitHub is set up, and again, I owe Raul for telling me how to do this, to let you jump
to a particular line in a file, highlight it, and it will be the line that has whatever...
It has some component, some part of the search that you executed.
But trying to do that for the wiki is more challenging.
There is a mechanism on the web, in the browser, I think it's called snippets.
And the idea is you can actually construct a URL that on the end of it has some words
to search for in the file, or some tokens, or some string basically to search for in
the file.
And that's great.
The problem is if I search for slash co slash co, and I find it...
Well, let's see.
Maybe I find a wiki article that has it.
Yeah, I see it in that one.
Yeah, I'm just trying to get a wiki article.
Come on.
I see.
Come on, you.
I'd like to be able to jump to it.
The problem is it might be slash co slash co, or it might be slash co space slash co.
I don't really have any way of knowing.
And if I don't have the space in there, the snippet isn't going to click.
So that's a problem.
So I'm not sure what to do about that.
Yeah, and then there are times that the white space is important, so you can't just eliminate
it.
Yeah, unfortunately, that is true.
And even if it's not important, but it exists, I've still got a problem.
So that's very much backburnered at this point.
I'm not sure whether anything's going to happen with it.
But Modulo, the OpenBSD problems, it seems to be pretty stable.
And except for trying to get the archive of Google Groups in, I'm kind of out of ideas.
Was that reference that Skip gave you, was that any use?
That was a roadmap, and I will use it if I decide to take the plunge.
Whether it's of any use, I just don't know yet, because I haven't taken the plunge.
I very much appreciate your getting him to give me that.
That was very kind of you.
Yeah, well, he had said in the past he worked with it, and then it became apparent that
he hadn't worked with that part of it.
But he did have contacts who did, so I thought that was really creative to pass that along.
Yeah, I think it'll be straightforward.
There's just a lot to learn, that's all.
Yeah, and Chris would already have to be in that position, right, to be able to run the
Google Groups?
In what sense do you mean that?
Well, he's the administrator for the forums, right?
Yeah, right.
So would he have to have that same level?
Like he'd have to be able to-
So you're saying that I may have to set it up such that I write something that Chris
has to execute with his privileges.
Yeah, that's possible.
Possibly.
Or you may end up, you know, find out from Chris, but you may end up as a second on the
forums.
Oh, that's a thought.
Yeah, that would be-
Yeah, I mean, that's essentially what he did with the Wiki with me.
Yeah, yeah, yeah, yeah.
And then you, does that mean that you sort of, not that we're out killing birds, but
you get two birds with one stone because you've got a backup now of somebody who can get into
the same sort of things.
Not that I'm in any position to do anything like Chris can do with the Wiki, but, you
know, Raoul is probably closer to that.
But you do end up with somebody else who can get in there if in a situation and still has
access to it.
I have various times in my career did wear a pager.
It's not fun.
No, no, I don't think you should be in that situation.
I don't, I'm certainly not.
It's more just if Chris sees something before I see it, he can often jump in and fix it.
If I see it first and I can deal with it, I often go in and fix something.
And it's just, you know, many hands make light work.
And if it's useful for you to have that, those privileges, then it seems to me that it's
beneficial as well to have more than one person.
Well, I'll dig into it and see what's actually required.
In an ideal world, I won't have to bug him for anything, but yeah, it may be that a good
answer would be to give me somewhat heightened privileges so I can do this.
But we'll see.
Did we earlier talk of a way of injecting JavaScript into the web view of a page?
We have kicked that idea around in the abstract.
Yes.
Are you thinking of the find problem?
Yep.
Keep talking.
There's two ways of dealing with that that I can think of.
One of them is to actually inject into the web view, which might mean we need to have
timers or something to wait for something to load.
The other way would be to actually implement the necessary find on the server side for
the wiki so that it would be available, you know, with something suitable in the URL.
And I, but, you know, pursuing both of those requires a little more research on what we
know works.
And so right now it's just kind of a thought in the air.
I see.
When you said that the difference between the slash colon slash colon and slash colon
space slash colon, it seems to me that that might be just like a regex search, really.
You just say, okay, I'll forget about the spaces between.
The thing is that he's talking about a built in feature of browsers that allow you to currently
supported text simmers for browsers don't have a optional space mechanism as far as
I know.
Okay.
It seems like that'd be very useful, but currently it's an exact match probably because exact
matches are really easy to security audit and security on the internet has been a big
issue for quite some time.
You know, I, I may be thinking about this wrong.
I I've always, I highlight the snippets based on the search term or terms, the search string
basically.
But that may not be right.
What I should, maybe it's right, but what I should probably do for, I should take the
snippet and try to append that to the link, the snippet that SQLite returns.
No, that's not going to work.
I still have the same problem because those snippets are not generated from the text file,
the raw text file that was indexed.
They're generated from the translated text file.
You could do it and they sometimes work, but it'd be a partial coverage thing.
Yeah, exactly.
If you only searched for one token, like slash co for example, or plus dot, I could do that
reliably I suspect.
And slash co would work on some of the pages, just not all of them because sometimes there
wouldn't be a space.
Yeah, it's a, it's a completeness thing.
Yeah.
And then the other thing that's in the back of my mind to figure out is how to get the
whole crawling and indexing up onto Amazon web services.
And I don't feel any urgency about that because I feel fine, just had a physical, checked
out great.
But the right thing to do is to get it up, up into the cloud where others could if they
had to maintain it.
So I'm starting to think about that.
You also need to get, recruit at least one of the maintainer to get feedback on that
process.
Preferably two.
Before I build it, you mean?
If you're talking about other people maintaining it, you need to have other people involved
so that you know that what you're doing is relevant to other people.
That's true.
That's true.
That is sort of a medium to long term effort.
So I remember that you said that, but I'm probably not going to take any action on it
in the immediate future.
Yeah.
Give it 10 years.
Hopefully by then.
I hope it's still around 10 years from now.
I hope it gets that much traction.
That would be great.
So yeah, Bob, that's all, that's all that I've got at this point.
Well, that's good.
That's good.
That's I'm really impressed with that speech.
I'm impressed with the way you've handled it.
I'm impressed with the way you've handled it.
The other thing I'm thinking is, is the highlighting the page is certainly a nice to have, but
with the browser button, it's not really a need to have because you can always dive out
and pull it in the other way.
Right.
Yeah.
But that's terrible work.
I know, I know.
But it's, you know, it's, it's, you know, simple things.
It's one of those simple things are simple, complex things are possible and it shouldn't
be a complex thing.
So I don't know.
Yeah.
I agree with your assessment.
I'm just saying it's to me, it's not the sort of thing that I think it was a deal breaker,
but it's sure a nice to have, you know, no doubt about that.
I think you're right about that.
Yeah.
I think the, the really big thing I think is probably to be able to get back into the forum
posts.
Yeah.
I'm right now.
Do we see the old archives?
Yeah.
Okay.
Yeah.
I've got those sitting up in the cloud.
I just sort of re-inject them into the database every Sunday afternoon and publish again.
I'm not, I'm not crawling the old archive anymore.
I was playing around with the system timer at the same time that I had the JViewer app
and it involves sometimes changing the system timer because I was just using like a, I wasn't
going to a.
Oh, the timer.
Yeah.
Right.
And somebody told, Roe was a new.
No, go ahead.
I should use a form timer or maybe it was you, Bob.
I should use a form timer.
Yeah.
We had a.
I'm using the global timer.
I wondered about that.
Yeah.
Remember my metrics, digital rain bug that I threw at you.
Yeah.
Because when you have two things using the system timer and doing an update to an ISI
draw and the common, the net effect was that one ISI draw was drawing into the other ISI
draw on the timer and the last control of the system.
Yeah.
It's a remarkable possibility that that could happen, but yeah, you can have form specific
timers.
I need to do that.
Okay.
Yeah.
And when I, where I got into trouble is I was, I was playing through the QT demo for
timer and it actually shows you how you can go in and set your interval and then it tells
you to, to use the generic timer and then you, and then I had in the background, I had
JV running and suddenly JV starts to do really crazy.
I'm thinking, oh, am I broken this?
Like what's going on here?
And then I thought, I wonder if it's a timer that I'm playing with is, is hooked into that.
So that's what it would be.
Yeah, it absolutely is.
And then the, the advantage of having it as an object, you know, or, or its own little
locale is that it can't be affected by it.
Right.
Yeah.
Right.
And I'm just thinking when I, I haven't gone deep into timer, still confused by it, but
there was something called P timer.
Yes.
Which I think.
P timer is the form specific timer that we're.
Yeah.
That's the form that, so untangles when you have more than one thing, you doing a timer,
you got it.
You almost got to use the P timer.
Yeah.
Say almost because there's some things which don't aren't QT really, but anyways.
So that was my experience with JViewer and aside from that, and I, again, thank you,
thank you, thank you.
I'm going to share a screen.
Let's go here.
Okay.
So this is my welcome to J page.
So it's not my main page anymore.
But the things that I was concerned about with it that I talked about last time with
Raul and I took a quick look at the grids, CSS grids, and I haven't had a chance to get
in and play with it and see how it interacts within the wiki and media wiki environment.
But man, that is incredible.
Like that's, that is so much easier than doing tables and so much powerful, more powerful
than what I can, from what I can see.
I may end up taking a run at one of the lesser tables other than NuVoc to see how it could
be restructured, but I can see how that would really clean NuVoc up, make it so much easier.
You're talking about the CSS grid?
CSS grid.
Plus.
Yeah.
That's amazing.
Yeah.
And if we, and NuVoc right now is done on tables, which means those tables get really
complex really quickly.
It's hard to even define what you're looking for to direct something.
I tried to parse NuVoc source code and I gave up.
Yeah.
It's, it's, it's, it's impressive, but it's not, it's not clear.
And I think a grid on something like that would really clean it up a lot.
But you'd kind of need to rebuild it from scratch, not so much convert it, you know,
take the data, take the appearance you're going for and build it rather than trying
to migrate it.
Migrate it?
Yeah.
Well, yeah, you, what you'd be able to migrate would be the links, right?
Yeah.
And the content would, would, would remain.
Yeah.
This, the structure would be.
It would be completely, but the difference it would make to being able to change content
would, would I think benefit, would be worth it with the effort there.
Yeah.
My, my, the thing I did last week, the vocabulary, the vocabulary nine page, I used grid just
to play with it.
And I was, I wasn't completely satisfied with it, but it was, it was, it was okay.
I should pick it up.
I was, I was looking at and just looking through other people's examples and there was something
Chris Coyier came up with where you can actually create hexagons, even though it's a grid,
you actually use overlapping rectangles and then clip the corners to create hexagons.
So you end up with a grid that's effectively a hexagon grid.
I mean, just, you look at it, you go, Whoa, it's awesome.
You know, for styling it's, it's interesting, but just more the, the, the, the ability for
you to do things like that.
There's, there's certainly things I think we could use that make it a lot easier.
What I was looking at, and I will look at in cases, something like this, you notice
that while the table is centered, the video is not.
And I think that's a result of this being displayed.
It's bumped the video over and I moved the table of contents because the issue I was
dealing with is that when I got to this page, I thought it makes more sense when you get
to a page like this, that you're, you're given something that will take you to getting started
with Jay, as opposed to this, which needs to be parsed before you get to getting started
with Jay.
And so by moving the table of contents up above this, you've got something right off that
presents.
You can click on this.
It's going to take you down to here.
And now you're looking at this.
You don't have to go through this stuff, even though these same links are here.
So to me, it, it, there are things in this table that don't exist in getting started
with Jay, but it's obviously possible.
If I did it as a text base, I could get around that and I could have everything that's linked
here to go into here, but the categories would be less obvious in that case.
And that might be something I look at in the future.
The other thing that I was able to do, because this is a category, it helped, it had a little
question mark up here that was help for categories.
I didn't want that here because you're looking at a Jay site.
You don't really want to say, oh, well, this would help me.
And it, it takes you to something in Wikimedia.
To give you an example, if I go back to the homepage, that help is still there and it
takes, because it's category, it takes you there, but I can get rid of that with CSS.
And then it won't be a trigger.
And I felt, I tried it with the getting started first, because I felt that was really someplace
where you don't want somebody clicking on a help button and then sending them someplace
else.
It's less likely to be an issue with the home, but I'm, I'm going to change it with all the
category pages, except for the category pages that involve the wiki itself and the construction
of the wiki.
And in those cases, when I get down to something like this, it may make sense to leave it up
there just because now it's useful information or it's put as a link in here.
So it's the information is, is in the, you can find it, but it's, it's sort of in an
area where you can see where you would be looking for it.
I don't think having that on the top of the front and center is, is that beneficial and
now I can get rid of it.
So it's not going to be a distraction so that I'm glad about that.
I haven't done very much with the homepage at least from what I was talking about with
Ra last week.
But this is probably your first look at the homepage, the revamped homepage, Ed.
You can see I went away from the icons and the, and the emojis.
Almost, almost.
Yeah.
Left one here.
Just to, you know, cause it's getting started.
I also added a direct link to the Jviewer.
Oh, that's the main page.
Well, yeah, but I think that's actually really important when you're thinking about the sort
of things.
Yeah.
You want it front and center.
You know that, that to me, that's, that's you know, it, it, it is prominence.
It's got to be put in a place where people can find it easily.
And I think what I'm also looking at doing is moving notices and announcements, probably
just changing this to announcements, moving it into this table and possibly wiki hints
down to this table.
And then this whole area will become a bit more clean.
And that's essentially where I am now with the homepage.
The, when you go to these particular pages, if they're not the welcome to J page, they're
going to have this more stable look that you're looking through.
And in going back and reviewing the minutes or the, the listening to the meeting that
I had with Raul last time, my sense Raul is that the, the issue that you had had to be
with the content, not providing you enough information that you could actually feel like
you knew where you were going.
And so to me, that's not so much a, a navigation that's more, there's, I should, I should be
focusing more on these areas to provide information that tells you whether this is going to be
useful to you.
So you don't just see J forums.
You might say access to J forums.
You might see J forums are the predominant way of communication, communicating through
email on J.
There's a lot of valuable information here and anything else that sort of tells somebody
what a forum is for J.
I can definitely see that being useful.
And I think that's a good area to put effort into.
I don't think you can solve all of the issue that way.
Yeah.
I'm wondering whether it's, it's an incremental thing though, that you work at different ways
to solve the problem.
Yeah.
Or do you see it?
Yeah.
Putting effort into there and, and maybe keeping an eye on some of the other side, side of
this is probably the right approach.
Yeah.
Because, because the problem is, is that different people have different contexts in mind and
you can only address the core ones.
At some point you got to rely on the person being motivated and working with what's available
to them.
But that said, catering to the core is pretty good.
Yeah.
And I think, I think if the content directs you and then you've got the, you've got the
categories on the side and then you match that as well.
Like you're coming at the, this huge chunk of information, which is the wiki.
You're trying to provide organization in the, the user's mind.
And it may be two or three different windows that give them the sense that they know where
they're looking for.
And what I'm thinking of is once you find something, if you're using the JViewer and
you bookmark it, well, that just made a lot of your searches much easier.
And those are the kinds of tools that you're trying to build into people that if they can,
those tools are available.
I mean, ideally the wiki is curated on a minute by minute basis and all the information has
been cleaned up and moved around and it's all done now.
But I just look at the amount of information and at least my expertise alone, I, there's
no way I can do that.
I don't think there's any one person can do it.
And I even have my doubts whether a large group of people can do it unless they're absolutely
dedicated to it.
More information you have, the more inertia it has.
Yeah.
But the way, the thing you can do is if you give the person who's looking for the information
enough tools, they may be able to find what they're looking for quicker and be able to
go back to it quicker again.
And it also may be start to point to directions we can go in.
If there are faults in the organization that we can, we can organize it better to fit what
most people are finding.
But I think the issue you were talking about before, I finally got it when I listened to
the recording of the last meeting.
I thought, no, he's trying to tell me just get the content in that actually helps you
find that next thing.
And it's just a question of getting the content that tells you if you've found what you're
looking for, if this is a fruitful path to go down or whether you go, yeah, that's not
going to be it.
And you can only do it in a general sense, but the general sense helps enough.
I think that it gives you directions to go.
And the more you can avoid fruitless rabbit holes, I don't want to avoid rabbit holes,
but fruitless rabbit holes.
I think that that's the helpful way to do it.
And I think that's about all I've got.
That's about all I think I've got to share in that area.
There's lots of work, but to me, I feel like I've got a handle on what's required.
It's still an immense amount of work, but it comes down to a sense of probably the next
step is to try and document how this stuff gets built out, things like template pages
and stuff like that.
And then part of the Wiki thing is bushfires.
So as people, I think recently Henry was dealing with somebody who was trying to find out where
they could download Jay.
And it was a number of different places.
And I was installing a person who was confused by the structure of the Wiki for how to install
Jay.
Yeah.
I came at it from a search perspective and wound up on some stale pages or something.
And those didn't link to the current pages.
And he didn't know to go to the top and come in that direction.
Yeah.
He didn't come.
I think it's pretty obvious he didn't come in through the homepage.
But it does mean that there could be work to go in, find stale pages that he might have
hit and then redirect them to the pages you want.
My approach there would be to hit the viewer and type in install and then maybe installing
and then visit those pages and each one put a link to the top level of the current install
instructions, especially for all the old ones.
Stem equals Porter.
You only need to type install.
Gotcha.
What?
The full text indexer that's available with SQLite will do Porter stemming.
So when you search for install, install or installing, you'll get all of those.
Yeah.
I'm not sure about installation.
Probably not.
I think for the most part, it doesn't matter because I think most of the people will be
doing code search.
But I could be wrong about that.
And to the extent that I am, Porter stemming is there to help.
Yeah.
But also, you know, typing another word and reviewing it isn't necessarily a waste of
effort because sometimes you notice stuff on second pass.
Yeah.
My sense is if there were installation pages that are out of date, we should be redirecting
them.
I don't know about removing them.
I don't know that that's completely correct because you can still install old versions
of J. But what we should.
I mean, this J602 still runs just fine on most machines.
Not on all machines, but on many machines.
Yeah.
I sometimes use J602 when I'm looking at a version dependency issue.
Mostly not for, you know, heavy lifting.
But when I want to see if something has been changed, that's one of the first places I
go to check.
I'm guessing what I'm thinking is there are certain, and I think it's an installation.
I can't remember where it was, but there was at one point, Eric had done something with
the JHS in the cloud.
And it showed up on one installation page and not on another installation page.
And that's the sort of thing that I think.
Yeah.
Especially for current stuff, merging is good.
But if there's actually valid historical instances that has information that's now out of date,
but was valid then, that's probably worth keeping separate.
It's kind of a, it's the archive concept.
It is the archive concept.
Yeah.
And, but the archive shouldn't be, if a person lands in the archive and the person's not
aware of it, the archive should, I think, point also to whatever's current so that the
person doesn't get stuck.
Again, a lot of work to do all that curation, but there probably are pages that should have
a warning icon across the top saying out of date or archived or whatever.
And yeah, this is old information.
The current ones here.
Yeah.
Pretty much a standard template.
Might be worth doing a template.
Yeah.
Support that effort so that it's, you can get into the habit of just typing in this
one thing.
Yep.
Pop it in at the top and then you don't have to think about it again.
And of course, then you still have to do the link to whatever is current if you've got
one.
Yeah.
You have to find it.
There's some work here, but it's one time work.
It's not.
Yeah.
It's not.
It's one time work for each page.
It's not, it does get repeated across pages, but it doesn't get repeated on the same page
multiple times, which is what happens if you don't address it.
Because then you have people keep coming back to you and saying, well, what about this?
Yeah.
And you had emailed me about the pages that I'd introduced that were basically just providing
links into GitHub.
Yes.
Which I did as a stopgap, basically, so that there was something there and it wasn't just
a note, there was no page.
That's another, that's just, I put that in the same category as my vocabulary page effort,
where it needs follow up to make it useful.
You know, if I just leave it where it is right now, that vocabulary thing, it becomes just
an annoyance because it's, why did somebody do this thing that doesn't hold out for me?
Why isn't it there?
I really wish it was there.
But yeah.
But what I probably really should have done rather than just put in those pages and save
other people a lot of effort is I should have gone and made a request to anybody who was
creating those add-ons that GitHub and say, we need a page in the wiki because they're
the ones that should be able to describe it.
That would not work for all of them, but where there is an active developer, that would be
maybe a little bit slower approach, but it would be a valid one.
I think the problem is that most of those pages had, you know, it was, they're not the,
the add-ons aren't really an active development anymore.
So it's somebody's past effort and that person might not even be around or if they are, they're
focusing on something else entirely different.
I think actually the example you gave me was one of Chris Burke's.
Well, Chris is pretty busy though.
I mean, he talks about-
Absolutely.
And that's why, yeah, that's why I'm not unhappy with just putting a link that will take you
to the GitHub.
And the thing was I was going to the main page where, I mean, what I probably could
have done is put a link right into the script, but then it's dependent on you being able
to read a script and figure out if there's a high level description, the main page-
Don't you have a GitHub link already?
I've got a GitHub link, but it just takes you to the level of a readme, right?
Yeah.
And that's fine, I think, because I mean, the readme is also interesting.
The script is there.
I don't think that that decision is one that you should change.
I think that what needs to be done is just for each one of those pages, go back through
and one, grab the readme and pull out whatever part of the readme seems descriptive of it,
maybe not the whole readme because some of it's unrelated.
And then also do a little script to pull out the global names defined in the add-on, which
you can, I think I sent you an example of how to do that.
And then use those names as a checklist to pull comments out of the, or whatever, out
of the script.
Or even just list the names because often the names themselves are informative.
But because there's a whole bunch of them, it needs an organized approach.
And because each of those is kind of an open-ended problem, you're not going to get it right
on the first pass.
And our tendency towards wanting to get things right is going to be a blocker.
Yeah.
Of course, I think the next step is to have, where possible, a paragraph on that page with
the link.
Same idea as what we were talking about with the content.
It's enough information that tells you whether you want to go look at this or not.
Yeah.
A statement of purpose would be a big step in the right direction for each add-on.
This add-on accomplishes that.
Or if we don't know what, if we, looking at it, we don't know what it is.
We found this X in the sand.
We don't know what's underneath it.
Maybe you want to dig.
Yeah.
Treasure is here.
We don't know what it is.
Yeah.
But yeah, I think that's the, again, it comes back to that content giving you direction.
I think that's an important concept that I had overlooked.
So thank you for that.
I think that's about all I've got really with the wiki stuff.
You have anything, Raul?
I've got two things that are really short, I guess.
One of them, I made a tiny bit of progress on the X-ray thing.
Yes.
I guess the issue I'm going to there is the original eval wrap mechanism I had was designed
to do the whole sentence in a single pass.
And once I'm doing trace, that's not how it works.
It's finding the smallest sub-expression and doing that and going on to find the smallest
sub-expression that'd be actually doing that.
And I didn't have it organized so that that made sense.
And initially it was just crashing bugs that I had to fix.
And now it's down to there's some pieces that aren't quite connected there that I have to
go in and wire up the plumbing so they connect.
And then once that's done, I get into how do I animate this, which is probably going
to be another pass of missing information.
But that's kind of where I'm at right now on that.
The other small thing was I could go and do a small dive into the grid CSS thing that
I was talking about, if that would be useful.
Sure.
So yeah, that page nine you were talking about.
Yeah, let me do that.
Let's see if I share screen.
Where is...
There we go.
I'll share this thing.
So there is my incredible nine vocabulary page, which has one little link here.
It needs what, about 20?
And if I...
I'm not logged in here, but that's okay.
If I look at the page itself, here's my grid view declaration, which is pretty basic.
It just defines columns.
It doesn't define rows.
You have mandatory display grid.
And the thing that I was noticing is you can do grid to let you do ordered columns or let
you do named columns.
And I'm doing both here.
And that is just a tiny bit awkward because you'll see here I'm using the ordering.
In other words, I got three columns and there's nothing in the markup that says which of the
three columns you have when I'm using ordering.
So I'm just using spaces to set that off.
But in some cases, like here, I'm only doing two of the three columns.
So I need to use the column labels.
And when I go back to the page itself, you can't really tell the difference between...
I think this one I have, for emphasis, I put a label on it.
You can't tell whether it's a label or organizing, unless it looks wrong.
But when one skipped, then that's where that becomes useful.
And so anyways, the programmer side of me doesn't like that sometimes I'm using position
and sometimes I'm using labels and I mix them.
But if I use labels everywhere, it becomes noise.
And if I use position everywhere, that becomes noise because now I have to supply a blank
for that one.
Maybe I should do...
Anyways, when you get into Nubock and think about Nubock, that's kind of a similar issue
you might run into there is which grid organization are you going to use?
And does it make sense?
Does it make sense for all cases?
And is it good to mix and match?
That's my quick dive into grid.
I just saw a typo down below with your emics.
I think you missed the two.
Yeah.
Nine bang.
Oh, you're right.
Yeah.
I'll need to log in for that.
Yeah, don't worry about it.
I just noticed it.
Is it possible?
I haven't tried this yet, but is it possible to do a grid within a grid?
I don't see why you couldn't.
I mean, it'd be a div which contains a grid.
Yeah.
And in those cases, like for instance, I'm thinking in Nubock, you've got the legend,
which is sort of on its own.
And so, you can't do a grid within a grid.
You can't do a grid within a grid.
You can't do a grid within a grid.
Yeah.
I've added that missing tune now.
Okay.
That was the feeling I got when I was just doing a quick scan down that and looking at
the opportunities there.
It's like, holy smokes, can this stuff, can this do a lot of things and maintain structure
and be responsive?
Yeah.
And I think that's a really good point.
I think that's a really good point.
I think that's a really good point.
If you get to a vertical column, that column has to come in the right order.
It can't be zigzagging across a bigger page.
I think the only thing other than the wiki stuff
that I've got to show is I can show you,
I got Jig running in JHS, which is cool.
So just to give you an example,
I've got something like that.
What I've done is I've hooked Jig up to a hotkey,
so I just, or a user key, so the Jig is up.
(laughs)
And now when I type A, it's going to show you
that these are all different types,
and they're also different shapes,
and you'd have no idea looking at that.
And of course, then you can sort of find overall
what you've got working with there,
and you can take any expression you want.
Of course, it is just going to evaluate it,
and then you can just hover over it,
and you've got the information.
And if you click on the information,
this comes up as a little window,
but the nice thing is you can copy and paste that
if you wanted.
So it's not just a tool tip.
- I think Eric would like this.
Have you talked to him about what he thinks
would be a good way of incorporating it into JHS?
- For Eric?
- Yeah.
I haven't, I've just got this working in JHS.
I had it working in JQT for quite a while,
and this one is actually sort of based on the same ideas.
And I had shown Eric that,
but it was years ago that I showed him JIG.
So, and I had it working on JHS at that time,
but I had it working a completely different way.
The way I had it working is I had to go in
and make some pretty big modifications to input and output,
and I still have to make some.
So including in the script,
I do have to, oh, yeah, this is the right one.
Oh, interesting.
I thought I had to, oh, this is weird.
I don't know if I'm working on the current file here.
This is interesting.
To me, that doesn't strike me
as what I'm actually running from,
running from, because what I'm actually,
have been running from,
I actually do have to use the input and the output
within JHS.
I have to make slight alterations to them
so that JIG's recognized, and it also,
like for instance, what it's really showing you here
is visual in the JIG locale and then I6,
and that's how it's showing this.
But it's taking this back out again,
so you don't ever see it.
And there's other things like,
well, in fact, if I hit return again,
I'm gonna get the visual, the visual,
'cause it's gonna just only see visual with no return,
and that's a verb,
and it's just gonna blast it back out at me.
And I'm not sure how I'm gonna do that,
because I can't just say empty space,
because I have a way of showing an empty space.
(laughs)
And I wouldn't be able to do that
if I have some kind of a marker that will,
whatever I decide is a marker to not display
would show up for one thing,
and then the other thing is,
I wouldn't ever be able to use that
because it would show, it would kill the display.
So that's where I've got with,
and just trying to think,
this is, oh yeah, this is some of the cool stuff.
When you get into empty spaces,
you have no idea, like you think this is,
these are all obviously empty spaces.
They've got zeros in their shape,
but you actually have no idea what the actual shape is.
So what I did is if you only have zeros in your shape,
it's gonna be a white empty space.
If you have a leading non-zero in your shape,
it's gonna have a darker gray.
If you got a leading zero in your shape,
but there's other numbers,
you're gonna have a medium gray.
But the reason you get this sort of a shape,
and it actually happens with regular jig as well,
so if I do this, is because it's taking the different shapes
here it's saying this is a shape two O,
but this is a shape zero zero.
So it's expanding this box to be the same height
as this box, which is two of zero wide.
And you have no idea what's in here
when you look at the way boxes,
box empties are represented.
Similar here, you've got zero three,
but here you got three zero.
And because it's three of nothing,
it's skinny and it matches here.
So you know it's gonna have a zero at the end,
but it could be a three one as well, I suppose.
But you don't know why it's so high,
but it's high because it's matching to this,
or this zero three is matching to this three zero
is what's happening there.
- And not to mention all of the-
- Sorry, go ahead.
- I was thinking there's also the whole,
I know you've already dealt with this in Jig,
but there's, you have a string on this thing
says two space zero.
You don't know where that space is a character,
whether that space is between two,
whether the two and zero are complex
or whether they're rational,
whether they're, so many things that can go on
that normally it doesn't matter,
but sometimes it does because they're programmers.
- Oh, you mean like if you have an empty array
you need to know the type?
- No, non-empty arrays.
Non-empty arrays have similar ambiguities.
- So what's the ambiguity?
You would look at-
- Look at quote two space zero quote,
look at, yeah.
- Quote two space zero.
- Zero, look at what that displays.
Now type in two J zero space zero R one,
and guess what type that's gonna be?
(silence)
- That's gonna be a complex result.
- Oh, I gotcha, gotcha, okay.
So if I do the same things with Jig on,
then this is gonna display as three characters
and you'd be able to tell what they are as well,
which is kind of nice.
In the case of characters,
I actually can tell you where they are in the path too,
which is nice.
And if I do this one,
(silence)
it's gonna tell me that it's complex
and it's a shape too.
So this is a complex two and this is a complex zero.
- Yeah, so one's a shape three, one's a shape two.
So non-empty arrays have plenty of ambiguities as well.
That's what I was trying to say.
- Yeah, yeah.
And this can outline,
it could tell you the shape and the fact
that there's two items and they're both complex.
And the other thing I kind of liked,
it's useful when you've got, say,
oh, I'm just trying to think of,
I'm just kind of coming up with something
again for the really complex faction.
I'll go 234 divided by 6739 X.
Oh, that's not that bad.
But what I did is I've highlighted the separators here too,
because if you get long,
you get fractions where it's very close.
And so you've got a very long denominator
and a very long numerator.
The R, if it's a different color, really stands out to you,
which is really useful because if you don't do that,
of course, the R is just the same color as it,
as a-
- Now, I'm very sorry.
I have to feed my daughter's cat.
My daughter's cat is kept on a short leash in two senses.
One, it is actually a leashed cat.
And two, it has just enough food.
So to be even a couple of minutes late is a terrible thing.
It's been explained to me.
So I'm going to have to run.
Thank you both very much.
- Bob, I will say I like very much decorating,
the idea of decorating or annotating nouns
in the transcript, in the terminal.
I don't think I should have to turn Jig on and off.
I think it should always be on,
but it should only manifest when I mouse over a noun.
You see what I mean?
- I do.
The challenge is that what you're actually seeing
when you see it in Jig is actually a very large SVG.
It's not a small thing.
- We've talked about trapping ourselves
in a particular implementation library before.
I don't need to talk about it anymore.
I just think it's a great idea.
I think it should be more ready to hand than it is now.
So I will see you both next week.
Thanks for all.
Take care.
- Yeah.
- Bye-bye.
- Bye.
- And that's about all I had to show.
And I guess just to follow up on what Ed just said,
'cause he was in a rush.
My sense is that's what the X-ray thing will do.
The X-ray thing will be is you can actually do
that SVG stuff and it's not taking up extra space.
- Let me take two steps back and then move forward here.
For your example case where you want to R
in the middle of something.
- Yeah.
- Try two R three to 100, to the 100th power.
- Okay, back to this.
So two R three, two R three.
- Or you can do five R seven, you know.
But yeah, yeah.
- To the 100th power.
- Two, 100.
Let's see what happens here.
Yeah, it's huge.
It turns out the R is here.
If I do the same thing with Jigon.
- Might be three R two might be better
because it moves the R over further to the right.
- Okay, but that's what it does.
And that's an example of where you don't have
to go searching for where that R is in the middle
of a long string, it really jumps out, yeah.
- Now for the Ed's thing where he was talking
about having it there in the background ready for use.
- Yeah.
- What I think the approach I think would be handy
would be to distinguish between the representation
and the information necessary to construct
that information, that representation.
And then having maybe a JavaScript mechanism
that on the fly can take the minimal presentation
and turn it into the visible representation.
- Yeah.
- That might be a complete rewrite though,
because of all the issues already in the limits,
when you do a display, there'll be ellipses some of the time
because some stuff gets chopped off that kind of thing.
- Way.
- And you can't keep the nouns around indefinitely
because that seems very large.
- Yeah, the way that it's done in JQT
is I actually pirated Henry's line feed,
I think it's not line feed.
I forget what it's called, line something.
And you can actually go in and pull the lines
that your cursor is on.
And then you can submit that to Jig
and it shows up in a different window.
So that's closer to what Ed's talking about.
It doesn't change the display you're working in your IDE.
It actually creates another window
where you can look at that display
and you just have to click on it.
- That would go from source lines to Jig representation
where what I think Ed was talking about
was having the display lines have a latent Jig alternative,
tooltips, that sort of stuff able to come up in your console,
in your session.
- Okay, I see what you're saying.
So that, yeah, I guess it's kind of the same challenge
I was finding with the input and the output
is when you look at what's coming into the J processing,
the J do, I think is what they call it within the JHS,
you're getting this character brought in.
It's a character line, right?
- Right.
- But you don't actually want it to be a character line
because you wanna evaluate it.
- Well, you evaluate characters, so that's fine there.
- The thing is, is that this line, that's the result,
you have it here twice, once with J and once without.
- Right.
- And what would be convenient is if the default display
could support drilling down into it
to see what it had been at the time
that's not being displayed.
- Oh, I see, you're saying you have the SVG,
you have the whole display.
- Well, I'm not saying have the SVG,
I'm trying to say, how do you get the SVG
without having the SVG there?
Because the SVG is bulky and huge.
So how is it possible to annotate in something
that's not visible, let's talk about JHS
rather than JQT right now.
In JHS, you actually have an HTML presentation here
and you can have text,
but then you can also have markup on the text
that doesn't immediately do anything.
But hypothetically speaking, could be enough information
to create the SVG presentation.
- Oh, I see, so, okay.
So what you actually do is what you would display,
say in this whole line, in addition, there'd be markup
that would possibly pull this out?
- Maybe that, but that is getting iffy
because that's gonna require nouns in some cases
and un-defined words.
But maybe what it could say is,
this has a shape of whatever, this is a type of whatever.
And then for each box, maybe something similar.
Just shape and type is really all you need,
that's all the information that's there.
Everything else is presentation.
Right?
- Not sure what, so shape and type, so shape of the result?
- So yeah, this line of, this text,
this part of the text on display
corresponds to a J object with,
here it's a--
- It's an atom.
- Rational, it's rank zero.
And of course that's the shape,
so the shape is an empty list.
So here we say rational, empty,
it's the shape and type.
And with that, and something that knows enough
about how J displays,
kind of a reverse display mechanism,
you could hypothetically speaking,
go from the raw text that has a shape of zero,
or a shape of empty and a type of rational.
You could have a routine that knows how rationals look,
and then come back and do this highlighting.
And--
- Boy, I see what you're saying.
What you're talking about is, in this case,
you just changed the color in the R
in the regular J display?
- I wouldn't change anything.
I mean, the thing that Ed was coming at
was that it wouldn't,
J wouldn't be there at all,
except for when a person wanted it.
- Right.
- So there's no markup on the R,
there's just markup on the line.
And then you have some JavaScript mechanism
that could parse the text that's being displayed
according to what the shape and type is
to add the highlighting, and add the colorizing,
and add the background colors, and add the tool tips.
- Okay, so to back it up, the way that--
- Possibly to create the SVG.
- Yeah.
But to, like, so when you've got an atom,
that's pretty straightforward, right?
'Cause you've just got an atom and you say,
okay, well, this is the type and shape,
and bang, you're done.
But it's when you get into things like the box to raise.
- Yep, then you have to have each of the boxes
is gonna have its own shape and type.
- Exactly, and so the way I dealt with that is I don't,
and honestly, well, here's an example.
Let's see, file, box,
seven, U, three, five.
So there's the box of that Unicode,
and if I do the same thing, I have that.
So I'm not, this is one advantage of JIG,
that another advantage of JIG is
it computes the size of the box.
- Right.
- So you don't get the broken lines,
because it says, okay, I know this is one character.
- Because it's, yeah, you don't wanna have box characters,
'cause it's not, it's an actual box, not a--
- I've actually drawn a box around it.
So in order to do things like that,
what I've done is I am not looking at what the result is.
I did look at that at one point,
but this is what it gets you into,
is it's kind of a problem that way.
But what I did do is I look at this
as the originating source of it.
And so I use that to create my display.
- That requires that the person be willing
to reissue the statement,
which is kind of the opposite of what Ed was looking for.
- Except that, and that hadn't occurred to me to use markup.
If you put a markup on this regular display
that was, say, clickable in some way,
that it would then trigger the visual.
So you've got the display, you've got what you already,
no, I want more information.
Whether it's a little dot or a little button,
you click on that and it would display
the whole thing for you.
- There's approximations there, but yes.
- Yeah.
- I mean, anything is gonna run into technical issues
once you try and get it completed.
- Yeah.
- Hypothetically, at least, I guess you could just have
a class for rank and a class for type
and have that and maybe do a mouse over class
and then just do colorizing that way.
That'd get you probably 80% of the solution.
- Yeah, but what, and again, this is just,
I'm thinking it through, so it might be wildly impractical.
But what I'm thinking is you would see this line here,
because it's displaying for you,
and you might see a dot, a little red dot,
either here or maybe back here.
And what that red dot is, it's a markup
and it's captured this information.
And if you click on that dot,
then it runs Jig on that information
and it'll create Jig for you just like this
without having to re-enter it again.
- You wouldn't even have to have it be a red dot.
I mean, if you're gonna preserve all the information
needed to re-execute the line,
it doesn't have to be visible.
It just has to, you could have a data attribute
on the thing that contains that information.
The problem is that in the general case, that becomes large.
So you would have some cases where it wouldn't work.
- So what would be large, the result?
- If I have an array that's 108,
one to the maybe 100 million entries,
it's only gonna show a small part of that on the display.
And I probably don't want to preserve
the 100 million entries in my markup.
- It would show 100 million entries.
Now, yeah, it can.
- It would crash my browser.
- It would probably crash your browser.
Yeah, but yes, and I haven't gone up to anything that high,
but a line of 10,000 characters, it can show.
And it just extends it.
That's what it does, it just runs it out the side.
- Practical limitations means at some point
there's gonna be something that doesn't work.
- Absolutely, yeah.
- And those are worth understanding.
- Yeah, no, no.
- I always complain about it.
- No, I agree with you, but the thing is the actual,
and that's when that gets back to the problems
when Jig is displayed, and absolutely those problems exist.
And that's why you wouldn't want to run it,
I think, in your IDE all the time.
But if you had a way, and as I say,
it might be involved with putting markup with this,
is when you turn Jig on, it includes,
'cause I'm sure I'm already playing around with the output,
it includes in the output something
that you can just click on that then will redisplay
what this is in the next line,
rather than having it, having to turn on Jig,
you would just click on something here,
and then this would show up,
and you would see exactly what you were dealing with.
- Yeah, that's for Ed's approach.
I think the approach I would do
is something less than Jig for, on mouseover.
Now, on mouseover, you can show tooltips,
you can highlight things, you can colorize things,
but you're not gonna do the full Jig support,
and then maybe in your tooltip,
you can have Jig support referenced,
or maybe in the menu you have Jig support,
or some way of keeping Jig handy
for people that don't know about it,
and have Jig actually be turning on and off
for people that find that more complete
and more robust presentation of boxes is worth--
- Yeah, I think of it more--
- Not being completely backwards compatible.
- Yeah, I think of it more like, let's say debug,
where you turn it on,
and then you're working with more information.
- Right.
- But you don't have to go in and reenter everything
every time you turn debug on.
That matches my thinking,
and that also gets me into how JxRay
is gonna be different from both Jig and from Dissect.
I think of these things not as,
I don't, there's Swiss Army Knives and there's Kitchen Sinks
but I think generally speaking,
you want specialized tools that are just easier
to maintain and build
than fully general tools that do everything.
With a big enough team, you can do an IDE
that does almost everything,
but that takes a lot of,
that's a major effort to keep those things up to date.
- Yeah.
- And a large team is kind of necessary.
- And the thing I'm thinking of with the X-Ray is,
at least the route I'm looking at now,
is that you generate a separate page,
and I'm thinking about tacit stuff,
I'm not thinking about explicit,
but if you were trying to analyze a tacit line,
you go to the page, you generate a separate page in JHS,
so it's like another window or a pop-up you could do as well.
And within that page, you've got a line that you can enter,
but you've got this kind of information
that's coming back at you.
You're not trying to do it in your IDE,
you're using it like a Jig.
You're setting something up on a Jig so that it's held
and you can take a look at it in different ways.
And so I don't see the X-Ray being part of the IDE,
I think it's a separate window on something
that you're looking at.
And then a lot of the issues that you're dealing with
in terms of the amount,
until you get to really, really huge numbers,
go away because you're not asking the IDE
to display all this stuff,
you're trying to figure out what's going on
within a line of tacit code and maybe two arguments,
and you can find out a lot just by putting in one zeros
and maybe up to the number 10,
and maybe up to rank three,
you can create a lot of information.
You can probably figure out what's going on
in that tacit code just by changing those inputs
and seeing how the results change.
Yeah, so anyway, that's the way I was looking at it.
But I was kind of happy to get Jig working in JHS.
And now actually, I think the next step I would take is,
I would look at seeing whether there's a way
that I can actually, as you say,
provide some kind of a markup
and it will actually change the output,
but because you're dealing with HTML,
who cares if there's a red dot ahead of the number, right?
It's just like a prompt really.
Turns out if you click on that prompt,
it'll expose a wider view.
- Or you can do mouseover.
I mean, I wouldn't say anything wrong
with doing mouseover support.
- But I don't wanna do it every time.
I wanna have somebody intentionally wanna look at it.
Otherwise, every time a mouse slides over,
it's suddenly gonna create this SVG.
- Oh, I see.
Yeah, for SVG, you definitely don't want,
I was thinking the more modest version that just-
- Oh, I see, you just do it as a div
and you give it a tooltip.
- Jig light.
- Yeah, yeah, yeah, yeah.
Yeah, I'm just thinking a lot of times though,
where I run into problems,
well, the obvious ones are the types,
knowing the difference between a character and an integer,
when there's no way really of telling
until you try and do something with it, it blows up.
And then the shape is the other thing.
So those are the main areas,
but the other places that it gets me in trouble is in boxing.
I don't know what I bought inside my boxes.
- The data hyphen arbitrary name attribute concept in HTML?
- No.
- Let me throw that at you
because it sounds like it might be convenient
for your data attributes.
That's good enough, I guess.
Yeah, I'll just throw this.
In the chat, where's my chat window?
Where is, let's see.
Nope, chat.
I threw in a link into chat.
- I see they think in chat.
I'm saving chat so I can't lose it and I'll click on it.
So, okay, so what you're thinking using data attributes?
- Yeah, a data attribute is basically an attribute
that begins with D-A-T-A hyphen
and the rest of it's just an attribute name.
And the point of it is that the browser
is not gonna do anything with it
unless you have JavaScript that interprets that attribute.
- And can you define what the attributes you want are?
- You can make whatever ones you want.
- Okay.
- And so if you wanna include the expression there,
if you wanna include the shape or the rank,
you can also use classes if you wanna have
some direct support from CSS for what's going on.
But that requires that your data be,
you could put the information into the format,
into the formats and types and constructs
that CSS supports.
This is for things that,
where that just gets in the way of what you're trying to do.
Like, for example, having the J sentence
that you executed as an attribute.
- So what we'd be changing would be the output.
We would be adding this information to the output
and then as well, we would be including extra information
for tool tips to display that on the hover?
- Well, yeah, I mean, these are all,
I mean, you could have a data tool tip, for example,
and then anything with a data tool tip,
when you hover over it, you could pull it out of there
and have JavaScript that would bring you that as a tool tip.
Of course, you can also already do that with title.
You know, there's a title attribute,
which has the same way, so you don't need to do that.
But that's an example of the approach
is you can have arbitrary data, which is used by JavaScript.
You have to have a JavaScript layer in there to interpret it.
But that can do whatever you want.
- JHS already has a JavaScript layer.
- Right, right.
- Yeah.
- And another thing you can,
another thing I've been thinking about off and on
is having a local WASM JPlayground interpreter
to supplement the JHS interpreter.
And if having two interpreters like that,
that could in principle communicate with it,
would be a fun thing to play with.
- So, well, okay.
So the JPlayground is already running in your browser,
but you're saying you would run it locally?
- Once the machines are big enough,
and JHS is running somewhere.
- Yeah.
- And having a JPlayground locally might make some things,
it's just an interesting project at this point,
but having a local J implementation
that's talking to remote J implementation
might be a fun thing to play with.
- Well, when the, I mean, you could test that out right now
because like the JHS AWS is remote, right?
- Right.
- At the same time I could be running locally
on my own machine.
- What we don't have is a concept of a session
where both those interpreters are involved.
And that's something that would have to be thought about
a little bit.
You either have a JHS session
that has a local JPlayground interpreter
that doesn't have a session,
or you have a local JPlayground session,
which has a session, which isn't a JHS session,
but some muscle parts,
or you have a session that combines the two somehow,
maybe, and then that gets into,
which one are you talking to?
But it's doable,
but it gets into interesting problems
that I haven't thought through.
- Yeah.
I know you can, like you can create multiple,
JHS windows because all you're changing is your URL,
your port, right?
- And hypothetically,
you know, your browser can have a cache copy of JPlayground
that each one of those windows could use.
The thing is we haven't really been building things
that are big enough to benefit from two copies of J
that are communicating like that.
And we definitely haven't built protocols.
For them to cooperate.
- Well, that's actually,
there's been a lot of discussion
that I've been reading about threads and stuff.
- That's just, there's that.
This isn't quite the same thing though.
This is more of a coprocessor system
than a threading system.
In threading, everything shares the same address space.
That's not what would happen here
because they're on different machines.
Well, they're in different processes,
at least even if they're not on different machines.
But they definitely don't have a shared address space.
The address space for the WASM version is in the browser.
The address space for the JHS session
is outside the browser.
- Right.
See, what I was thinking is
you could have multiple JHS systems,
assign them each to their own thread.
- That's still multi-processing rather than multi-threading.
- Yes, it is.
Yeah.
- But that's not,
doesn't mean it's a worthless exercise.
It's actually probably a fun exercise.
- Very interesting.
- And something that people would love to play with
if they had examples to start from
and a few interesting cases to get them on ramps.
- Well, it almost starts to look like sockets, right?
Where you can send information back and forth between the two.
- HTTP connections always do use sockets.
It is a socket-based.
And WebSockets also is applicable here,
which is more of a browser-based
superstructure built around sockets.
Yes.
- Yeah.
Anyway, thanks for that about the data stuff.
That's interesting.
Yeah, I'd have to think about how useful Jiglite is.
But the advantage would be that you're not making huge,
changes to the display.
You're just adding extra attributes
to whatever you're displaying.
- Right.
It definitely loses some of what Jig can do.
It doesn't get you the nice Jig boxing.
It doesn't get you the freedom from ellipses.
It doesn't even get you,
it might not even get you good tool tips.
I'd have to think about that.
- Well, tool tips are pretty limited anyway.
Like I've learned a lot about tool tips.
And it has to do with implementation, obviously.
But there's, to me, the frustrations are,
if you're going to be able to make your display look bigger,
you should be able to make your tool tips look bigger too.
You can't.
Tool tips lock into a set size.
Can't change the size.
- I've seen those set sets.
If you do your own, roll your own.
- If you roll your own, you can.
Yeah.
But then you're actually doing a fair amount of work
to do that.
I'm not sure the return is there
and the complexity grows quite a bit.
- Yeah.
This is characteristic of anything
that gets a huge number of people involved
is that the more people that are involved,
the more connections there are.
And at some point that saturates
each individual person's ability
to cope with that number of connections.
And so you get specializations
and the specializations start wandering
in their own direction.
And yep, that's the life.
- One of the things I learned,
I was concerned about how long it was taking tool tips
to come up.
Like there's a pause before they show up.
And at least on Apple computers,
there's a P setting, is it a P setting?
Whatever they call their sort of metal level settings.
There's a setting you can put on
that will change the speed at which a tool tip comes up.
And it's a system setting and you can change it.
And then things pop up right away,
but it isn't the browser.
You're actually looking at your machine and saying,
"I want tool tips to pop up fast."
To hide it into the system settings, to me,
it's just like, this is something that you should be able
to click, drag and it's not even within the settings
in your Mac.
Like you don't have access to it
in your usual GUI system settings.
You have to go dig for it, change that and then it works.
- That's the nice thing about standards.
There's so many to choose from.
- So many to choose from, you got it.
- Going the other way,
you've got 14 standards, we should do this right.
Okay, now we've got 15 standards.
- Yeah, that's exactly right.
Anyway, I shall let you go.
Thanks so much for the information.
That's really useful.
We'll take a look at it.
- All right.
- And I'll see what I can come up with before next week.
In terms of what I do,
in terms of having something clickable in the JHS output,
that would be, I think that would actually be
the neat way to do it.
You don't have to worry about setting up a control,
you don't have to set up a user key or anything.
It's always there.
- The trick that I imagine you run into is
format doesn't give you information,
like you format boxes
and it doesn't give you information
on what's inside each box.
So you'd almost have to rebuild format
to include the extra information
rather than having,
that's where the challenge is gonna lie.
Which you probably already done for JIG.
It's just that this is a different way
of encapsulating information.
- Yeah, to me, it's just a matter of capturing
the input line, 'cause that's all I need to create it.
And attaching that to what the display is
as a markup that's activated by a red button.
- Well, I guess what I'm thinking is
it would JIG you have complete control
over what you're generating.
With JIG Lite, you're also introducing
a level of backwards compatibility.
So you need to somehow generate,
still generate the same characters that,
if you do a copy and paste it's--
- Oh yes, copy and yeah.
Yeah, copy paste on SVG is interesting.
You have to find out what you got there.
- I mean, hypothetically, your SVG could have
associated with it somewhere buried in the markup
of the result of format.
And have that be what you get when you copy.
- Yeah, actually what I was thinking,
when I've thought about in the past,
I think that is the way to approach it.
You actually create the information.
And again, it would be coming back to,
well, if you had it with the red button,
then you're only gonna copy and paste
what you see there anyway.
The red button wouldn't be copied.
But if you were, say for instance,
you want the display to be copy and pasteable,
then you have to be able to isolate what the input was.
Or if you think if you use, is it five bang colon six,
is the linear in, you can do it that way too?
- Yeah, five bang colon six is the parenthesized.
Five bang colon five is what you should use for linear.
- Okay, so it was five and six.
- Both linear, but six has added parentheses.
- Okay, okay.
Yeah, so in that case, what you would do
is you would have a way to capture five, five, five, five.
- I'm thinking of maybe recommending five, six a bit more
to beginners because it helps them see how JPARC is.
- Yeah.
No, I think there's a, in fact,
in spite of what Ed had said about,
I don't want anybody messing around
with what I've put in there.
I think you do wanna display it,
at least to parenthesize.
And my feeling is you wanna display it as a box format,
but maybe both.
But I don't think you wanna just rely
on what they've put in.
I think you wanna take what they've put in
and then be able to show a structure to it.
That's very useful to people.
And if you don't do that,
and you just show colors and stuff, they're not, yeah.
- Different people have different ideas of useful.
Somebody who's generating music or art
is gonna have different focus
than somebody who's doing pencil analysis on structures.
Not completely different,
but not completely the same either.
- Yeah, but that's going into subject matter areas, right?
What we're trying to show is-
- Subject matter areas focus on different things.
Somebody who's only working with one dimensional arrays
isn't gonna get much use out of the things
that illustrate how multiple arrays work, for example.
- No, but that's just means
if they're working with one dimensional arrays,
they're gonna be displaying one dimensional arrays.
That's what they're gonna see.
I'm thinking of the structure of the verb.
You want to have something that shows you the structure
that's inherent in what you've done.
If it doesn't do that for you,
and that can even come down to colorizing.
If you had no parentheses and you just colorized it,
essentially you're showing structure, right?
You're providing that clue to somebody.
But I think there is an advantage when you look at it
to at least use parentheses,
parenthesizing to show how stuff separated out.
And I think it's more useful to even show the boxes
because you can show the levels
of what things are contained within other things.
And because the parentheses is a linear format,
you start to look and see what edges are matching.
Whereas in a boxed format,
you can see what's inside something else.
But that's just my feeling.
(laughing)
- Anyway, that's great.
Thank you so much.
And we shall see you next week, hopefully.
- See you next week.
- Be well.