Modularity is important to promote separation and code re-use, so that each individual module, such as HTTP parameter parser, could be loaded independently by other programs with a short name 'web/cgi/module'.
Each module should distinguish between private names and public interface, marked with scriptdoc NB.* comments.
Compliance with API design guidelines is essential, addressing the needs of three levels of programmers. For example, in the same area of HTTP parameters:
- Opportunistic: it provides ready to use parameter accessors handling default values, type conversions and lists
- Pragmatic: allow to substitute custom paramer provider or handler in place of standard, as in Guides/JWebServer/HttpParser
- Systematic: provide access to raw HTTP parameters in their original sources for GET or POST, while possibly hiding the lowest protocol of whether it's stdin, environment variable or socket.
- 1 API
- 2 Utilities
- 3 Middleware
- 4 Integration
- 5 Configuration
Programmatic interfaces for application developer.
- expose CGI environment variable as high-level standard set of accessors
- hiding platform specifics, e.g. SCRIPT_URL vs SCRIPT_NAME, etc.
- provide physical paths with jpath aliases, e.g. ~CGI for root of current application, ~Page for location of the current page, etc.
- high-level accessors to HTTP parameters regardless of source (QUERY_STRING or stdin) with support for default values and type coersing in quasi-declarative form
- handle multipart/form-data (for upload)
- provide key-value dictionary of parameters with given prefix, possibly coersed to numeric, etc.
- provide list of parameters with a given same name, as from a set of checkboxes, possibly coersed to numeric, etc.
- convert a set of parameters into a series of input fields or a URL string
- provide a accessor for request HTTP Headers
- provide a storage for output HTTP Headers
- automatically calculate output Content-Length
- maintain output Content-Type
- maintain output Content-Encoding
- provide stream-like interface for emitting string and typed data
- provide binary output, hide Windows text mode stdout issue
- support buffered mode with clearing and premature end of response
- support automatic encoding, specified in HTTP header
- support HTML Writer interface from utilities
- handle errors: with top level try/catch
- error logging
- output diagnostics of unhandled errors according to content type
- provide normal termination and programmatic abnormal termination as
special types of throw
- provide a accessor for request cookies
- provide a storage for output cookies
- automatic encryption and decryption, possibly delegated to other addon
- automatically forward persistent cookies
Optional session "object" support
- maintain a session between requests
- support different session methods: cookies, hidden input field, scrambled container
- provide accessor to save and get different string values
Static functionality for web-related processing, such as parsing, encoding and formating. Some of the functionality could be delegated to other addons.
HTTP Parameter Processing
- decode and encode string with URL encoding
- parse and generate URL query with a matrix with shape N,2
- split and combine URL with protocol, address, port, path, query and hash parts
- split request and combine response with headers, body and mime parts
- encode and decode text using HTML entity encoding
- convert between different character encodings: Unicode, UTF-8, standard national
- declare a list of standard mime type for validation, mapping of file extensions and support of other content generation modules, such as image converters
- decode and encode mime multipart messages
This is meant to be used by non-web server programs like GUI HTML formatters, etc. Possibly this should be a separate addon: web/html.
- programmatic generation of generic HTML elements
- automatic escaping and encoding of attributes
- verbs for some concrete HTML elements
- bulk generation: table from matrix, HTML list from boxed list, etc.
- HTML Writer interface with different output writers: string, file, generic output writer
- parse and generate HTTP headers with a matrix of shape N,2
A set of handlers for executing high-level API frameworks such as JHP, Web Services.
Handlers are selected based on configuration and current file extension or file name, etc.
Generic handler for low-level handler implementation is provided for J scripts with .ijs extension. Developers can implement handlers for output images and other specific formats.
- invoked by CGI integration layer, see Integrated execution
- obtain basic interface to CGI integration, such as configuration properties, logging, etc.
- obtain CGI adapter interfaces, environment, request, response
- setup handler-specific interfaces: JHP, Web Services, etc.
- setup error handling try/catch block
- load and process script
- execute script
- process errors in middleware specific manner, e.g. error section in web service response
- finalize execution
- resume to Integration layer
Interfacing with low-level web-server protocols: standard (default) CGI, enhanced CGI (FastCGI, SCGI), J stand-alone Web Server, as well as abstract interface with integration adapters created in the future. E.g. ASP.NET request handler can act as such adapter in a way similar to FastCGI/SCGI invoking J via COM or DLL while leveraging IIS for load-balancing, multithreading, etc.
- configuration: reading and exposing
- error handling
- selecting adapter
- executing handler
- read CGI config file
- select CGI adapter
- identify script and select middleware handler
- setup CGI services for handler and application code
- setup error handling try/catch block
- execute handler, see Middleware Execution
- process errors
- generic logging
- finalize execution
- CGI exit code depending on specific adapter
- integration interface should be very thin, delegating most of the HTTP processing to the CGI framework, and only implementing the low-level protocol, such as sockets, etc.
- integration is configurable
- much of the CGI implementation and all of the application code should be isolated from the choice of particular low-level adapter
- error reporting
- raw environment variable access: resolving and platform-specifics is done by higher level CGI
- input stream: buffering reading, parsing, splitting are done by higher-level interfaces
- output stream: buffering, combining, encoding, etc. done by higher interfaces
Setting up the front-end web server, J system, and CGI framework for mutual interaction.
Vendor-specific host web server procedures
- register specific web application or document root
- register a file extension for CGI framework, while leaving other files to be handled by host web server
- option of a dedicated instance of web server, such as "second" in JHP
- local configuration files, e.g. .htaccess, web.config
- specifics of web user and permissions settings to allow certain functionality, such as OpenGL under Mac OS X
- boot code to recognize CGI mode
- delegating to CGI framework
Configuration file(s) with settings for various layers of CGI framework.
Location:: Main config file should be located near J system to be found by the CGI boot code. Other, application-specific files, may be located in folders on the path from the web root to the current script folder.
Settings:: The following sections or properties can be configured
- execution modes
- application parameters