MultiCoreSupport
Multi Core Support
Introduction
This page is to build a tool for J application developers to create additional instances of J in order to utilize the processing power of computers with multiple cores and additional computers in order to solve large problems more quickly. Although it is tagged as "Multi Core" here, it should extend to other processors on a network and even over the Internet.
For referencing I will call this tool "MC" for Multi Core. If anyone prefers another shorthand name, it is easy to change.
It is a surprise how much interest has appeared already. As such, this page should be open for all to contribute.
For now this is only one page. It will probably need to be split into multiple pages as this progresses.
Scope
MC should only be a tool for the application developer to use to distribute processing to other instances of J. MC is intended for very large problems. It should not do any automated decision making to conditionally distribute processing. Leave that to JSoftware to put in the J engine as they see fit.
Process
It is tempting to jump straight to implementation, particularly when applications exist which already provide such a service and can serve as starting points. However, I would hope that we can first get some definition of what MC should be. To get the ball rolling I will list my thoughts. Anyone, please feel free to add, change, delete or correct.
Already Skip Cave has suggested ways to test MC. This is an excellent way to help define the kinds of problems MC is to address.
Devon McCormick is assembling a page with specific, simple examples to typify increasingly complex types of problems and provide concrete, replicable problems we can use to benchmark specific proposals for parallelization.
Requirements
Clone another J instance
- In same machine
- In another machine
Send and receive nouns
- via sockets
- via mapped files
Send scripts to clones
Send and receive semaphores
Send verbs to be executed and possibly get results back
Stop cloned J sessions
- By a command to the cloned session
- Forcing clone to stop
Detect lost clone
Detect lost parent
Comments:
Mapped Files
Advantages: Reduces memory requirements Avoids copying
Disadvantages: Does not support boxed arrays Mapped files can only be used with shared memory systems
Question: If a file on a file server shared by several computers is mapped, does this make sense? What would be needed to make it work?
Cloning in the same machine should require no additional tools outside those already in J.
Some facilities listed may be easier to implement or coming with J7. We should be able to use J7 for development as it is J6 console with a few added foreigns.
Should it be implemented as a class, or just add names to the z locale? Objects would be easier to keep track of if multiple clones are created, although arrays of locatives are unwieldy.
Whether nouns are passed via mapped files or through sockets should not be a concern of the application.
If a command sent to a clone was to return a result, it should not cause the sending system to hang. Receiving the results should be a separate operation.
Right now distributing work should be a defined by the application developer. No automatic distribution of work at this time.
Application Interface
This section should be filled in with specific calls to MC. I would suggest that all calls be prefixed with "mc" unless the shorthand name is changed. Other than for that, this section is empty.
Testing
This is empty too.
Contributed by:
Don Guinn donguinn@gmail.com