[moon] home

Erlkönig: Z (ζῷον)

One tiny letter; one vast universe...
- Dawn Patterson on "Z", 1999-05-10
[parent webpage]

[webserver base]

[search erlkonig webpages]

[import certificates]


Z, The Z Project: (from greek ζῷον, “zoion” creature pronounced 「ゾイアン」 or “zoh-yawn”). n, A research project intended to invert the relationship between two-dimensional and three-dimensional applications in the current desktop paradigm in human-computer interfaces. The Z paradigm supports the use of legacy two-dimensional applications within a shared three-dimensional spatial environment. Moreover, the Z environment itself is network-sharable, supporting multiple distributed users and discretionary access permissions, providing a common context for both professional collaboration, shared gaming, and private tasks.

Beyond the desktop

[zland trees with tiny leaves] The ΖΩΙΟΝ project is my personal research into spatial environments. Currently only some of the experiments are public, since the rest is still undergoing design churn. :-)

Current Status

2013-07-10 GLib's memory philosophy and the evil of logical memory

It's infuriating that GLib would hold such a promise of a set of useful containers for C, yet at the same time advocate utterly irresponsible memory allocation, (noted to capture the idea, that author doesn't speak for GLib) such that the libraries themselves cannot be trusted not to explode one's program on out-of-memory errors. This philosophy seems to be a side effect of many versions of Linux enabling logical memory by default, which causes the kernel to assume infinite memory (meaning all malloc always return success). This promise isn't tested until until a program actually tries to use the allocated memory, at which point the kernel may suddenly realize the memory isn't there, and kill the offending program - which may weeks after the malloc was done. Basically, the logical memory model makes the kernel lie.

I suppose one could have a user-defined malloc which would call the normal one, then write into each of the pages in the returned area - while catching SIGSEGV to prevent dying (if that works)- to force allocation, then free the memory if it didn't exist and return 0 from the user-defined function. Ewwwww.

Logical memory made sense when hosts had to be able to fork huge programs (more than half RAM + swap, where RAM was perhaps 1 GiB in the 1990s) that needed to run a tiny side program occasionally. Now that workstations with vastly more RAM are common (this host has 24 GiB in 2013, at a few percent of the price of the 1990s host), there's absolutely no reason for logical memory to be the default. It's pretty easy to disable it with something like this:

The defaults are '_memory 0 and '_ratio 50 (only used if '_memory == 2).
cat <<... >/etc/sysctl.d/10-no-overcommit.conf
vm.overcommit_memory = 2
vm.overcommit_ratio = 100

With the physical memory model restored, the g_mem_set_vtable() could actually be interesting, providing a way to catch memory errors and even empty a cache elsewhere in the same program to free up memory. On the other hand, one can't easily determine from that function which object's construction led to the OOM issue, making simply backing out of building that object anything but simple.

There's more information at https://www.kernel.org/doc/Documentation/vm/overcommit-accounting. It seems like a per-process version of the overcommit settings would be more useful, since one could then enable logical memory just for programs having huge sparse matrices, the aforementioned fork issue, and so forth.

2013-07-05 C macros and syntax abuse (generic collections)

Late in the night while writing collection traversal functions for operations like map to perform an action (given by function pointer) on each value in a collection, I realized the function calls are expensive, because the compiler can't easily inline what one is calling via pointers. So the idea of applying a CPP macro came to mind. I wanted something to work in a context like this (greatly simplified here):

typedef struct _Node_t { struct _Node_t *next; int value; } Node_t;
Node_t *nodes; /* initialize with a linked list */
MAP(nodes, number) {
    printf("value: %d\n", *number);

The problem is that the obvious underlying syntax for implementing MAP is a for, but for only allows the creation of variables in its first expression who have matching base type, which isn't the case for nodes and number above. And nesting another for looked useless until I thought about GCC extensions. The following works perfectly, but the final keyword may be a surprise:

#define MAP(list, var) \
  for(typeof(list) node = list ; node ; node = node->next) \
    for(typeof(node->value) *var = &node->value ; 1 ; ({ continue; }))

Whether this is an argument for or against ({ … }) working so orthagonally is left as an exercise for the reader. The idea that the continue would affect the first for makes sense to me, since we haven't entered the second one's controlled substatement. Interesting though.

2012-10-21 XSendEvent and Lathing in the UI

Dragging the window under the cursor to highlight text
(cursor reconstructed, since X cursors are in hardware)
[dragging a window under the cursor to select]

Ignoring the send_event flag abattoir for the moment, tonight yielded an enhancement to xwins where lathing (drawing/selecting across an object by moving the object with the 6axis controller rather than the pointer, or moving both simultaneously) works with various programs including:

  • gimp - drawing with the window inverted works fine
  • xev - a critical aspect of debugging MotionEvents
  • firefox - clicking on links works now too
  • Wine + World of Warcraft - ooh, but the autorepeat gets crazy

It notably doesn't work on either xterm or emacs yet, although typing at them through xwins works well.

2012-03-15 XSendEvent and XTestFake* API

So, while sitting in some code (xwins in this case) contemplating how to forward X events received by the OpenGL window on to the X app being displayed within it, the docs yield two primary API interfaces:

Status XSendEvent(Display *display, Window w, Bool propagate,
                  long event_mask, XEvent *event_send);
int XTestFakeKeyEvent(Display *display, unsigned int keycode,
                      Bool is_press, unsigned long delay;

Those are fine, if you need to forward events to a nested X display (as my xvfbtex program did so easily with XTestFakeKeyEvent and kin), which can be relied upon to target the correct window based on server coördinates, or if you need to forward events to blithe, unparanoid applications that accept events from XSendEvent. Sadly, I want to send events to windows, not the whole server, and ignoring server coördinates, and to any X client, not just the tiny number that don't simply toss events that arrive with the send_event flag set. Like Xterm, by default. Ack. Incidentally, Xterm's Secure Keyboard option, when active, allows SendEvents. WtF.

I have the icky suspicion that compositing window managers have been juggling windows around in 2D space underneath the pretty 3D displays.

2016-01-18 More grumbling about X's 2D internal bias

Normally, the X Server uses the events coördinates to decide which window has been hit. This makes stacking significant, which is useless for true 3D window manager where windows aren't even always rectangular, but very often trapezoidal or worse. The only reason to use XSendEvent is to try to get around X's window hit determination and put in in the hands of the 3D window manager, but XSendEvent events all look synthesized (i.e. untrusted). Several aspects of the X server cause problems for 3D WMs, much of it in programs/Xserver/dix/events.c...

  • Window stacking and hit management (XYToWindow)
  • Fiddling with the eventX and eventY (FixUpEventFromWindow)
  • Only focused windows evade using the cursor for window select, and evade the SecurityReadAccess check (ProcSendEvent)
  • There is DeviceEventCallback (intended for the RECORD extension) that could potentially be bogarted by a new extension to modify events at EnqueueEvent in events.c, but that's pretty involved, and requires root access. It's amusing that the callback is available both at the enqueuing and at the delivery side of things.
  • As of X server 1.18 (2015-12), XYToWindow has changed. There are some interesting comments from 2015-02 about Wayland having had a modified version for Westen-like compositors that didn't want to go through the effort of configuring windows server-side when moved. Apparently it was deemed a problem for comboboxes and menus, not solved by using window-focus, so it's been removed. However, the new XYToWindow is part of a screen, not the main server anymore, and DDXen can override it. See miSpriteTrace(), still at its heart the same 2D code from the old XYToWindow, except for new commentary about rootless mode windows may be offscreen. The wayland code is in hw/xwayland/

It looks like the only way to handle this vaguely in the mindspace of X as it stands - for example, when twisting the view of some X windows - is to construct a screen-normalized bounding box of each window that should accept non-synthetic events, then construct (some hand-waving here) a Shape region for that window (can a window manager even stuff that into a server on behalf of a client?), and then theoretically the correct window would get the XEvent, but with coördinates that hadn't been corrected by the 3D window manager because X won't let the window manager modify the event. Not really a win.

xwins - OpenGL-based X window magnifier and manipulator (2013)

Xwins showing an emacs (left) and itself (right)
[Xwindows as OpenGL textures in xwins]

The program is currently pretty dependent on having a 6-axis controller attached (my current one is a Space Pilot Pro garnered cheaply from eBay), although a few mouse mappings for moving around are enabled IFF you have a key bound to Super - which most people don't (I use Control, Meta, Super (in xwins), and Hyper (in my window manager - bound to CapsLock) as modifiers - only Alt and NumLock are pretty idle, but I'm clearly not representative). The need for more accessible mouse control will probably get addressed later, making a release of the code more useful.

Having this kind of magnifier is fabulous. I find myself using the program multiple times per day (or per hour), for:

  • Zooming in on a window, either because the detail is interesting or because some stupid web developer used ghetto antialiasing (i.e. low-contrast text/background to hide jaggies, which happens to make the text harder to read as well).
  • Rotating a window to see images from different angles, including occasionally angling them in 3D.
  • Zooming windows that normally have poor performance running at fullscreen. With xwins I can watch movies in fullscreen even if the underlying player couldn't normally handle 2560x1600 without haveing framerate issues. It's even better that xwins will pass through key and mouse events (usually), so I don't need to keep the player's window separately visible.

It has a limited window management capability which I don't use often; the other uses above are the vast majority so far.

xwins can't yet cope with windows which get resized - the image of a resized window in xwins stops updating (this shouldn't be that hard to fix at some point). Some bugs are exposed when running xwins with more than one UI window. There's also still a quirk in the input loop that shows that the last incoming event may sit unhandled until the next event occurs.

zme - prototype viewing client, working in local mode, Spaceball 4000 FLX support

Manipulating an object with a XGLTEX X server mapped onto it.
[X server in an OpenGL texture]

About 13,000 lines of C++ (and additional bits of gmake, lisp, perl, sh, and the zop/zys scene languages), zme has been rewritten from scratch using a new, hierarchic model from the zone proto-server, and using an increasing amount of threading. It renders objects read in from external .zop files and maintains them in a frame-of-reference hierarchy, which can be edited interactively in some (as yet limited) ways and then saved in its entirety back to disk. The viewpoint is also an object in the FoR, and the nested transforms of both it and the other objects can be manipulated. Basic object highlighting is used to indicate the current manipulation focus.

My favorite bugs so far have been the 4MB/s memory leak(!), and the teleporting sheep bug (a matrix stack overrun), both of which have been fixed,

Textures and full support for the Spaceball 4000 FLX have been added. Motion of the viewer is now intuitive (despite the continued evasion of quaternions), but manipulation of a non-viewer object still happens in its frame of reference rather than in that of the viewer. A control to swap viewer and focus object is present to mitigate this for now.

xvfbtex - An OpenGL program with a realtime X display as a texture - (code)

xvfbtex showing the xgltex server buffer mapped onto a rectangle
and a roaming sphere; Heretic is running and playable.
[xvfbtex display of xgltex X server as OpenGL texture]

To go with the xgltex X server, I wrote a program to map the X screen onto an object as a texture in a basic GLUT program. However, GLUT immediately revealed itself as the usual albatross, and had to be removed to allow for use of XTest or XSendEvent. It uses GL_LIGHT0 to illuminate the X screen - an oddly jarring effect, since it's never happened to a real X server in my experience until now. This is actually the seventh program in a series, each more divergent from the GGI program from which I began.

xvfbtex showing the xgltex server buffer mapped onto a rippling
X screen, with fully usable xterm, fvwm2, and mozilla.
[rippling, usable X server with Rikku as background]

2005-01-09: Pointer tracking now works on the moving sphere as well, allowing full use of X via the sphere's surface just as well as through the rippling X screen behind it. Of course, stopping the ball first eases interaction significantly. This version is being used as a basis for doing some factoring on the unproject code.

2004-01-22: Pointer tracking for arbitrarily rotated and scaled surfaces now works, including for complex surfaces with computed or animated z-axis variation, such as sinusoidal variation in z. A initial class-encapsulated model allowing for full variation along x, y, and z is in progress.

2003-04-20: This demo now supports keyboard, mouse, and button input. However, the code is still a an uncharacteristic mishmash of C, C++, and GLUT-derived style (ie: too many globals).

(initial implementation 2002-06-07 - 2002-06-15 - 6 programs)

xgltex - X server rendering into an OpenGL texture format framebuffer - (code tree, xgltex subdir, notes)

I've written a prototype X server using a memory-based framebuffer compatible with OpenGL textures, which uses shared memory to provide the framebuffer as a texturable image for OpenGL clients. Loading the texture itself frequently generates quite a bit of traffic to the GPU, since the GPU can't use the shared memory directly (ah, I miss the concept of an SGI with Unified Memory Architecture), but even a 1024x1024x32 screen works surprising well. The other formats, particularly the monochrome bitmap, still need work. The name could be improved as well, I think...

Xserver in GL texture with Xeyes as the eyes, fvwm2 and
the X worms(1) program running on its back in zme.
[X server in an OpenGL texture]

xgltex started out as XGGI from the GGI project, an impressive work that does any number of neat things. One of its best points is its generality, but for my tightly-targeted purpose, this came at too much of a cost due to the numerous extra levels of data copying and conversion involved. So, to achieve the kind of refresh rate I wanted without having to work in very low resolution X screens, I forked off a separate server dedicated to rendering into texture-compatible framebuffers.

XGGI itself looks to have been derived from the XVFB (Virtual Frame Buffer) X server, which I had also explored early on. The latter would have had a decided theoretical advantage over XGGI due to its being part of the standard distribution, since having to compile both GGI and XGGI tends to put off new users. Unfortunately, after any number of differently command line options and the associated amusing framebuffer layout clashes, porting an X server to my own nefarious ends looked easier than trying to use any existing one.

My intent is to arrange for a special Xatom to be available on screens supporting the shared memory attachment, and to allow the client to request the shmid (see shmat()) directly from the X server. By removing the need to track the shmid separately, references to X servers in textures can be a URL-style string in a form resembling: xserver://DISPLAY/shmem This URL-like approach is key to integrating X screen into the Z texture methodology, both for its textual simplicity and even more so for the advantages of the lasting value of the URL form over the transient and unpredicable nature of the shmid's themselves.

The framebuffer format itself, as represented in the shared memory segment (currently just the image data) could soon have a header or footer as well, allowing the OpenGL program to dig up all the remaining detail on the visual format from the header. This removes the need to bury screen dimensions, depth, and format information, etc., in the URL, an obvious advantage. On the other hand, the same could be achieved by making the information available from the X server shared-memory-screen property, potentially even simpler once I figure out how to set the Xatom values inside the server code instead of just creating empty ones.

Another promising possible use for some additional framebuffer data would be optimization information for which scanlines need to be updated. Adding a vector of height sequence ids corresponding to the XGLTEX servers scanlines could allow quick determination of screen subsegments for update, by simple comparison with the client's memory of the last sequence number it saw on the prior update.

(initial implementation 2002-06-13 - 2002-06-15

glsperm - 2002-06-07 - Floating tadpoles (spermatoza, actually) with illumination

[glsperm screenshot] Appearing in retrospect to be a completely crazed reimplementation of the worm screensaver to OpenGL, this program sports about 100 little wrigglers, some equipped with illuminating antennae that activate (non-simultaneously) during dusk and deactivate during dawn in the internal, one-minute long solar cycle. Up to 8 light sources are used at once, with very nice glow halos around each. The effect is particularly profound in the darkest period, when only those critters within the spheres of illumination are easily visible.

(initial implementation 2002-06-04 - 2002-06-07

radius3d - watch RADIUS logins across the earth's surface

[picture of earth wit numerous small spheres] Are you an national or global ISP? I wrote a quick hack to display the earth in 3d, complete with indicators across the surface indicating where RADIUS-based users are logging in, in realtime. Uses the incoming areacodes and translates them to longitude, latitude, and altitude using a database. Spheres over specific areacodes grow in proportion to the (approx) cube root of the traffic, and decay to minimum diameter over a 15 minutes period. The planet can be rotated and zoomed in realtime. Run on a huge projector display in our NOCC, it gives a good feel for the login habits in different geographic areas.

zland2 - full terminal and remote apps in 3d space

[zland with a translucent teletype] Terminals now have translucent, texture-mapped backgrounds. A PERL client was written which can connect to zland2 via a socket and control a matrix of squares within the space, using it as a display for a life cellular automaton, as seen in this picture.

Further development on zland2 has ceased in favor of the zme program.

zland - basic terminal and remote apps in 3d space

[zland with an early teletype implementation] Earlier versions of the program also had tty capability, but the focus to control them was a separate, black-and-text input target - a legacy from the original terminal concept implemented as a simple text window with a surrounding control brace in the zglyphs program.


[numerous spheres with tails floating about] A variant of gluttool with more complex objects, in this case little spermatozoa-like things with their tails (composed of little tetrahedrons) trailing behind them.


[numerous spheres floating about] A rewrite-from-scratch to experiment with the GLUT tookit, resulting in the realization that GLUT isn't suitable for serious work, since it's unable to attach arbitrary data to callbacks. Lots of wiggling spheres inside of a textured globe, with the word "Center" floating at, predictably, the center.


[a formation of octahedrons, text widget, and billboard] The zglyphs program, with animated texture maps on the octahedrons. a bracket-like text widget with vector-based text, texture-mapped terrain with reasonable lighting, and a floating billboard with a somewhat racy image (if you're a large, sea-dwelling mammal) image of two orcas.

trees - fractal trees in SGI GL

[a rather harshly geometric, mostly stalk, tree] A GL fractal trees program, my first fresh experiment with 3D graphics, with viewpoint control, fog, and control of the level of detail.

spaceball - hardware

For those of you wanting to use a Spaceball 4000 FLX under Linux, mine works wonderfully, except for the small detail that the joystick driver under RedHat linux is a royal pain to make work with it (worse now than [mine's actually a dark racing green Jaguar special edition] in 2001, when it couldn't even reliably recognize a 4000 FLX despite the fact that it responds predictably to a number of queries and setting options), and integrating it into Xinput is yet another dire area. However, just as I was about to start writing my own interface I discovered a related Sourceforge project named libsball along with their homepage, which definitely had a lot to offer in terms of protocol handling in more than just one type of spaceball.

Nevertheless, I eventually ended up finishing my own application-level (and currently specific to the spaceball 4000 FLX) library, in order to integrate better with C++ and produce full, typed C++ spaceball events, an incoming event queue, and xyz coördinates in right-handed OpenGL-style. Unfortunately, hoped-for support for a ~/.spaceball file and per-axis user-configurable tuning have not yet materialized, and integration with the Xinput extension isn't included, with the excuse that the objective is, after all, to replace X :-)

I've made the alpha level C++ spaceball module available for download.

If you're still trying to use the pitiful linux joystick driver (with regards to the spaceball, that is) and jsattach, which apparently can only identify a 4000 FLX during startup and even then not reliably, I once found the following shell fragment helpful in establishish the initial connection, after which you can use jstest to see if it's really working. Be sure to insmod for input, serio, serport, spaceball first.
( while ! jsattach --sball4 /dev/ttyS0 ; do : ; done ; ) &

Z - the zone server

Protocol is in development for communications with zme and other zclients.


A sound server, as though there weren't already enough, functional until the incept of an incomplete assault on distance delay and doppler effect in sheep .

I'm still curious about how to close and reopen the X connection for OpenGL use without mysteriously crashing in glXCreateContext. (Feel like debugging? Take a look at the testcode sample , a structure-flattened C++ regression test-in-progress).

Implementation Notes

The program zland2 was implemented in GNU C. Z and zme are being implemented in C++v3 compiler, originally with Mesa 3.2 and a 3dfx Voodoo3 3000 card (and more recently with an NVidia subsytem, despite the slower framerate at 1600x1200, which just isn't worth it on the NVidia. Perhaps the new Matrox will do better?)

Although moving to C++ helps in some ways, I already miss GNU C's local functions (an issue which neither namespaces nor macros really addresses), and there are numerous other grumbles I'll spare you from.

Related Research

  • openwonderland is a project from Sun, predictably based heavily in Java. The most obvious feature of the project is audio and authorization rights support, and it has a focus on shared business meeting spaces, including VOIP, which it addresses fairly well, with support for importing models, PDF content, VOIP communications, basic animation, and group-interactive objects.

    Up front, however, default avatar movement is but adequate, camera control is poor and nonintuitive compared to current gaming standards, It has portals, but they're terrible compared to Croquet's. Its Java bias means that many functions attempts to leverage existing Java 2D APIs, which spawn independent 2D client windows outside of the 3D world, which heavily interferes with the input model. Architecturally OWL is based on clients connecting to multiple separate worlds, disjunct rather than interlinked or even interpenetrating, and leaning heavily on web APIs for many support functions.

    As such, OWL can't really be considered as a replacement for a system like X windows at this time, although later versions may manage to run more then a single X window per server, and without dire aftereffects. There's no obvious way for a user to mix X applications and 3D tools together into a single scene and then use it easily from multiple sites.

  • Panda3D is a library of subroutines for 3D rendering and game development. The library is C++ with a set of Python bindings. Game development with Panda3D usually consists of writing a Python program that controls the the Panda3D library

  • MUPPETS has achieved in-situ development, where system code can be written from within the 3D system itself. Development is active and fairly rich, including but not limited to CAVE integration, data visualization (including generation of 3D representative objects on the fly from the system's internal data), inter-disciplinary virtual theatre, and VR movie creation work. Additionally, a social model has been created within RIT for upperclassmen within the project to mentor lowerclassmen, where the system is an essential component of the scholastic progression. This is part of a general objective to provide a feeling of nearly immediate achievement to lowerclassmen, allowing them to create within a 3D environment almost immediately, instead of studying for some time just to achieve the koan of “Hello World”.

    Docs for this projects are suffering from linkrot to some extent, but some materials have been cached.

  • Garry's Mod A Half-Life 2 Modification allowing cooperative modification of a shared environment, which just happens to model physics to a reasonable degree.

  • Croquet A system that appears to deliver pretty much the complete spatial environment package. Sadly, their response to being slashdotted in June of 2003 was to completely hide the project, still the case in February 2004. Some cached screenshots on this site. It is one of the few shared environments that bothers to have any kind of permission system, and appears to have a large number of available tasks already available by virtue of its implementation in Squeak A detailed review from 1999 is available at itwales.com.

  • VOS. A promising project. From their website:

    The Virtual Object System (VOS) is a hierachical distributed object system, a dessert topping, and a floor wax. It is an infrastructure for object-oriented network communication, and building flexible, distributed object networks for a variety of purposes, but mainly multiuser collaborative virtual environments. It also tastes good and will keep your floors shiny.

    The VOS idea of objects having context-carrying links to other objects without constraint to the main tree is intriguing (and more open-ended than Z's action-acquisition menu request), as is their codifying of matrix transform data as generic properties where Z instead currently (2003-10) considers such information as special. The VOS approach would simplify the addition of properties, although there might be issues in implementing the Z permission concept under an open property paradigm rather than as an intrinsic part of the Zone code.

  • Gel, Billed as "An open source initiative for multi-user internet 3-D", its stated objective to bring in a group of interested developers seems in conflict with the surprising, total lack of screenshots. Nonetheless, the theory involved is on the mark, and downloads are available.

  • Otools, a object-oriented, multidisplay OpenGL system with the following claims:

    • Multi session/display object-oriented windowing system.
    • Very good OpenGL support. Object-oriented virtual reality system.
    • Crossplatform audiosupport, with realtime 3D audio library (DTDSS).
    • Networking and Multithreading methods supported in OOP.
    • Works nicely with supported 3D hardware cards in Linux!
  • Bamboo, enabling dynamically reconfiguration applications, and written to support Kent Watsen's virtual environments.

  • The National Tele-Immersion Initiative (NTII) with a focus on full telepresence in a shared virtual environment called a telecubicle.

  • The Virtual Reality Transfer Protocol (VRTP).

  • The 3dwm window manager. (originally for the Chalmers Medialab 3D CUBE)

  • The Berlin Consortium

  • IBM: a spatial, tool-oriented RealSpace, and the striking conceptual failures of RealPhone, RealCD, and Apple's counterproductively similar-minded QuickTime 4.0.

  • XEROX/PARC: some minor tools experiments in the UI Research Group

disencrypt lang [de jp fr] diff backlinks (sec) validate printable
Cogito ergo spud (I think therefore I yam).
[ Your browser's CSS support is broken. Upgrade! ]
alexsiodhe, alex north-keys