• UserInterface

User interface notes and suggestions for VOS applications.

Note, the user application (formerly called TerAngreal) is currently (as of July 2008) being rewritten as part of S5. This application will serve as a 3D interface, as well as a general WIMP GUI for editing etc. Eventually it could show AGIL 2d graphics, in combination with 3D or not. Maybe even hypertext from HyperVos. The goal is to provide a sort of swiss army knife of UI modes that the user can combine into a configuration supporting his specific task, and which can be used for different types of vobjects. These basic UI modes can then also be applied and specialized for certain more specific views, which can also be considered "modes" (or presented to the user as choices in addition to the modes.)

The new S5 general interface application consists of several panes. Each pane has a view mode. Possible modes include tree view, list view, 3d view, property-list view, ZoomableGraphView.

This program is like a dashboard for a bunch of Vobjects out there on the net.

For a more detailed programmer-oriented design document, see IRUIDesign

--> What should we name this new interface app?

• InterView

• IRUI
• Chameleon
• Facets
• Interreality
• Charrette
• Argo/Argo Navis

?

See also Chat, Presence and Messaging Ideas

For (the beggings of) a user interface "patterns" catalog, which might be used as a guide in creating the UI app's generic UI view types and features, see http://www.cs.helsinki.fi/u/salaakso/patterns/ by Sari A. Laakso (Uni. Helsinki).

The preferences for this applications should be stored in Vobjects, and you would edit them my simply opening a view on it.

It should be possible to edit both online remote objects, and offline objects stored in a COD or XOD (by loading them as local objects).

It could be implemented mostly in a library that other apps could intergrate.

As a VOS app, it would be capable of being used as a server. Therefore it should be able to edit its own configuration in a seperate view, and save it to a storage backend. Also, it should have a command to go into background server mode, closing all windows and leaving just an icon in the system status tray.

UI Architecture

• Panels, Tabs, Windows, Miniwindows (detatched views), Minitoolbars (inside views), miniviews (for small graphs or labels), contain "views". A Group of views shares target vobject to display, etc.
• Actions, and binding actions to keystrokes, menu entries, buttons, etc. (as general "minor mode" objects?)
• Filters, sorts and other modifiers and controllers. 2 kinds:
  • In their own panel, selections affect the view group's current object
  • Can be attached to any view panel, with their own controls embedded in the view's toolbar, menus, etc.; also hooks into view's selection mechanism somehow??
• Global toolbar to create new view panes, windows or tabs.
• Button that makes a view pane "sticky" (i.e. puts it on all open windows).
• In tabbed views, include a button to open a vertical list of all tabs in a side bar (with sort commands).
• Use Qt's MVC utilities where applicable.
• Each view has a small toolbar at the bottom for common actions, or for the view type to add to. Include a common button to toggle view display settings (view type specific), which is a panel that replaces the view (ideally with a "flip" animation as if it were the back of the view, like OSX dashboard widgets, and with a different style to easy differentiate it from the view)
• Actions, tools, etc. are seperate from views ("minor mode" plugin objects?), though may only be applicable to certain view types (some may be to any).
• Any action that can be performed on one object, should also be able to be
  • applied to any set of objects (a group of children, either of any object, or the output set of a filtering object, or a user's manual selection of some objects)

Implementation Notes

• Many UI components will be represented as Vobjects, and use VOS events to
  • communicate events and state changes. This makes it easier to work with, especially wrt. concurrency. It will also make it easy to write components and plugins (included remotely-supplied) in other languages as support for other languages is added to VOS.
• Will use the MVC infrastructure
• May need an FSM infrastructure to help implement UI control?

View Types (Pane Modes) and tools/actions/controllers

• VOSApp preferences view, both for remote stuff and the UI app itself. Be able to mark certain preferences for seperate display with revert/undo buttons, and then hide the list of all preferences -- so you can select a few for temporary setting but leave them visible so you can undo them later.

• Log control (see logging)

• property list
  • TODO datatype-specific entry boxes
    • Color
    • Spinctrls for numbers
    • Button to open file dialog
  • at end of list, show "hyperlinks" to non-property children
• image thumbnails
• general "thumbnail" list with bubbles containing various bits of info

• node-edge graph (ZoomableGraphView)

• Web view, either any URL, or using Hypervos engine (built in or plug in to the program itself, so generated locally)

  • Incorporate Web Inspector: http://labs.trolltech.com/blogs/2008/06/02/bringing-web-inspector-to-linux/

• block-hierarchy like this: . Or tree map?

• Panel that shows the siblings of the currently selected object for each (or one) of its parents
• For chat panel:
  • Set a presence state on avatar when panel is open, clear when closed, change when minimized. E.g. set to "NA" when closed, and "Away" or "Busy" when minimized.
• Variants of list/tree views that find a "ui:workspace" child in the current object and shows its contents

• See this article for a few tips and examples about "dashboard" type applications: http://www.ddj.com/web-development/203101035

• Panel that shows children of the target object as sections, with their
  • children beneath either as property lists or trees. Sections can be expanded/collapsed (in a style similar to Outlook).

• Show objects in a "pivot graph". E.g. see http://www.visualcomplexity.com/vc/project.cfm?id=543 and http://www.research.ibm.com/visual/papers/pivotgraph.pdf

• Planar overview (top view) of the current 3D sector, with objects rendered as symbols, or as simpler orthographic projections. Makes it easier to quickly place objects roughly in a place, to either float at height 0, or fall until they hit a floor. Could also generalize to any planar slice of a space, perhaps with a bounding box restriction-- then you could temporarily create one at the appropriate angle to lay out one part of a 3D scene (e.g. to face a building's wall); or, e.g., a curved ring-shaped wall is flattened to facilitate layout on it.
• Panel for editing CSS or CSS-like stylesheet (in backend are vobects that can be exported to a stylesheet file; possible properties are known through OTDs)
  • Merge in properrties from different stylesheets (display current different from others; display specially if overidden by current).
• Cover View (Apple iPhone, iTunes) style display of objects, either properties containing images or objects with an image reperesentation given as metadata.
• General web browser (browse to any page with HTTP)
• General purpose "telnet" like panel (use next view type and run telnet in it?)
• Local system shell (embedded terminal emulator). Store a list of initial
  • commands to run and directory to change to in config.
• 3d view.
  • configurable to allow for a lo-rez 3d view for use with software opengl implementations, small
    • devices, etc. - skip shaders, downsample polygons and textures perhaps. automatically select some of these as defaults based on device detection.
• Voice chat controls. Automatically opened when someone talks to you or when you choose to talk to someone else. (Same with chat window).
• Grid of graphs tied to property values (editable)
• "Stack" or "block" view, where objects are grouped into blocks that are stacked on top of each other, or some next to each other. Object relations (or certain subset of relational links) are displayed. This view corresponds to a typical software system stack diagram.
• Options (minor modes?) on graph or Sankey view to format the display to show one "dependency" chain in a row (with branches below). Or, sort each node on a grid according to two properties or types of the items, with arrows or ribbons connecting. (Possible use: To Do list or Gantt chart).
• Sankey diagram
• Rolodex list (with alphabetical tabs)
• Filtering controller in own panel, creates new filtering objects with original object as input set, sets its output object as view group's target object. Also with controls to show current filter setup.
• Related-objects controller that displays "related" objects to current object (i.e. related data sets to current set), can change current view targets, or grab them to apply to other views (e.g. original input of filtered results being viewed? or automatically find the original input being used with current view?)

General features:

• Color picker for properties. Format output according to property data type (e.g. #RRGGBB for html colors, r.rr, g.gg, b.bb for float arrays)
• Save objects currently being shown in a view to a file (COD or XOD)

• Save different pane/GUI configurations to file (COD or XOD), including an initial "default" view. Associate configs with initial vobject type. Possible to load views from files and generally to express GUI panel setups and options using Vobjects, perhaps in the same site(s) as the data you're working on, even to load them remotely. (Not sure how to do this yet...) On start up, if there is no automatic default view set, show a launcher screen where you can start a new "project" with a certain configuration, including tutorial configurations. Also show this launcher window from something like File->New Configuration or New Project. Also in a menu. Also remember if a pane has a certain toolbar attached.

  • Include ability to skin/theme/style the look of the GUI
  • Include ability to selectively customize messages (error messages, labels, menus, etc.) [use toolkit's translation framework?]
• Publish GUI configuration, either directly from UI app itself over VOS, or uploading copies of configuration objects to a server; also "announce" published/uploaded configuration so that other avatars in the current world see them in a list. Publish using mdns.
• Options for how much/how little GUI controls to show. Perhaps also compile-time option (for making custom apps).

• Load plugins. Similar format to omnivos plugins and mesh plugins (so a plugin could provide features to all three applications; in the case of mesh plugins, and TerAngreal having an embedded mesh/console mode, it should invoke both aspects of the plugin.)

• Record a movie of a pane locally (and hide mouse cursor, unless you hold down

  • a key to make it visible with custom icon image). In the 3D view, use http://www.neopsis.com/projects/yukon/ to record from OpenGL rather than wx or X. (So need a mechanism for view modes to override record-movie action).

• Use revision control features of objects to make a branch, update from parent
  • branch, sync back to parent branch, etc.

• Controls to apply various tools from the VosToolbox to metaobjects (i.e. extend them with

  • the types defined therein). Also include controls to create local objects from the toolbox (e.g. filters/views)
• buttons to minimize panes to small icons that restore them. command to
  • hide/show all except a main or selected pane. Auto-unminimize panels on mouseover their visible edge.

• Use ObjectTypeDefinition to place placeholder slots in tree/list views where children specified by the type definition are missing. Click placeholder to create new object/property there. Drag other objects onto it to insert them there with that cname.

• Action that opens a new (tab|pane|window|mini-float-window) with some view of
  • the chosen object (e.g. from right-click menu)
• Firefox-style notification banner
• System-tray notification bubble
• Mini-float-windows
• Tabs
• toolbar buttons inside panes to set view types (frequently used types) or to open a certain object with a certain view
• Associate different pane background colors with different object types, or different tasks the user might use a pane for. OR different selection groups?
• Modify default style slightly for a more elegant UI, especially if we choose
  • a GUI toolkit that doesn't use the native look and feel (e.g. Qt, FLTK).
• Let the user have a personal "stylesheet" that customizes the look of the
  • application, and also include app. "stylesheets" in a saved configuration. (Note, Qt 4 has a "stylesheet" feature built in, but this could be implemented for other toolkits that let you customize widget styles)
• In most views: searching. Enter search query that searches object metadata and selects matching objects. In lists etc. filter the list. In graphical views, activate visual pointers of some kind on matching objects.
• In general on mouseover or selection of an object (in all views), display a popup or banner with basic metadata about the object.
• Buttons to dim out and black out the whole view until you push any key. This is useful when you are working in a meeting and want to stop using the tool temporarily.
• Make it easy to publish/unpublish vobjects, and browse published objects, using MDNS - a view pane can be used to drag and drop objects (or a minor mode of any view pane?). Also include a pane that lists saved GUI configurations to load or publish.

• Copy/paste in all views. Objects are copied as XOD. (Or COD for more compact?) Also accept various other pasted formats and invoke importers to load. (Should include interpreting structured plain text and HTML (esp. <table>s and lists)

List or icon views

• Simple list (one line per object); grid of icons; or list with little mini panels [blender-style] stacked to show several bits of information including icon, name, types, etc.
• Pane which is a "palette" of objects (When added to 3D view perhaps conjure objects as children of the avatar, then have command for reparenting those objects into the world?). Different modes in the palette to copy objects from it, link them, or move them. Palettes can be ui:palette type Vobjects, which could be discovered remotely or by loading files.
• Filter objects shown in a list view by keyword or other metadata

* Option to pop up a notification if some object changes.

• Command to split a property at the cursor into two sibling properties, or into three sibling properties using the currently selected text as the middle one.
• "Recycle bin" for deleting stuff
• Thing that listens to certain kinds of objects in a view or in all views for certain kinds of events (and dynamically includes new objects) and play sounds or display messages etc. Use this for notifications with the chat views, or just to add meta-modalities to the UI. (Think Star Trek computers that make little unobtrusive noises when various things happen.)
• Crash handler that logs debug info and stack trace, and offers to mail it to us or open a web
  • browser with the bug report or just display the info and stack trace for copy and paste. On Linux, catch signals and debug using gdb (trace for all threads), or backtrace() if

    errors running gdb. On Windows, set handler with SetUnhandledExceptionFilter and get stacktrace using MiniDumpWriteDump() in dbghelp.dll (analyze dump later with windbg or Visual Studio). For inspiration on features for error handling see also http://code.google.com/p/elmah/, http://www.codeproject.com/KB/applications/blackbox.aspx,

Web/hypervos editing tools

• Bring commonly used HTML XML types and attributes, as well as hypervos macros, and useful filters and other listeners to the top of the list
• A seperate view of hypervos templates that can be used, with large icons
• showing something about their layout or style, letting you drag template objects in to link them.
• Custom editing pane for hypervos templates
• A special/custom editing view for a CSS stylesheet. (hypervos needs to let you keep CSS rules as vobjects, probably)
• Special/custom editing view for certain HTML elements and hypervos macros (links, images, embedded objects, etc.)

Command Console view

• Embed Mesh library and interpreter [inprogress]
• also callable from other views, e.g. chat view

For 3D interface (i.e. TerAngreal):

• When connecting, pop up a firefox-style info bar at the top that shows (counts down) number of objects not yet downloaded, categorized by 3d objects, textures, etc. In general this could be applied to any view type (with different object types of interest.)

• Show list of A3dlViewpoints, in a menu and in a side pane

• Show list of Layer objects, in a menu and in a side pane
• Option to turn off gravity
• Option to turn off collision detection for your avatar
• Option to automatically help steer around obstacles. Especially if you keep trying to walk through a wall, walk around it or jump over it.
• Avatar animation triggers, in a menu and a side pane. Also recognize words in speech text, and special commands typed into speech text.

• Avatar presence status in menu and set away after idle time. Also link to other IM clients. See Chat, Presence and Messaging Ideas

• Click to start walking towards a place. Click to teleport to a place.
• Select an object to do stuff
• Controls to orbit around a selected object
• Fly mode
• Strafe (Translate) left/right/up/down controls
• Follow mode: as the selected object or moves, follow behind it and orient to look at it, or just look at it (watch mode)
• Command to easily create hypercards with some standard shape
• Command to create a portal
• Commands to delete selected object (if you are authorized to do so) or move it into your pallette (see below)
• Command to toggle indicators around objects in view with name and type (especially useful for tiny objects)
• Speech bubbles should have more rounded edges, and have a black outline around them to give them some definition.
  • In general we need a little kit of simple on-screen bubbles/tags. Maybe with mouse click callbacks to make hyperlinks and buttons:
    • Speech bubbles (maybe the user can choose from several kinds. Maybe site-provided scripts can set them temporarily too. Transparent/text only, translucent, big, small, border, no border, cartoonish)
    • Hyperlinks and/or text from hypercards
    • Object metadatata tags; avatar names and status tags, etc.
    • Important events about objects
    • Optionally have tags/messages/hyperlinks near objects, or alternatively stacked on top of the screen like the console (firefox style)
• Need good UI for selecting and moving objects. Should use 3D effects like halo around objects, lighting, etc. This could also be temporarily set by world-provided scripts to work better with the feel and style of the world.
• Headlight

• Integrate CrystalZilla

• Doom style sprites
  • essentially 2d, but with a different image for various angles.
    • use 2, 4, 8 or 10 faces (front, back, left, right, front-left, front-right, back-left, back-right, top, bottom
  • A novel implementation would be to use OpenCV from Intel to
    • interpolate between the sprite faces.

      http://www.intel.com/research/mrl/research/opencv

  • action queue: store actions in a list or mask, and apply them when finished.
    • or store them as a macro. or queue up actions, that are applied to an object that is finally selected. will need to add some more abstraction around actions, with a "do" function, passed an action function pointer (perhaps abstracted with typedef and a bunch of #define'd symbols).
• select multiple objects (with one "primary" selected object)
• Hold down a key to create an arrow or pointer connected to your avatar that
  • others can see. Also do this for shared 2D spaces.
• Apply overall "filters" to graphics/lighting of a sector: affect amount of color/saturation; overall brightness, contrast, etc. color filters? (For software rendering, could specify a function to apply to each pixel mbefore drawing? maybe somethnig similar for hardware renderer too?) See A3dlViewRasterFIlters
• "distance dimming"

• For scripting the user interface?? - Google Web Toolkit - Build AJAX apps in the Java language ... (Builds a full, Swing-style application in Java that runs in JavaScript) ( http://code.google.com/webtoolkit/ )

• Specific commands for selecting objects out of your bag/inventory/palette to create and attach to your avatar, and trigger an avatar animation (e.g. a gesture of offering the object to someone, or holding it up to display, etc.). Also maybe formalize offering objects to other people (including hypercards) as part of talk messages along with this instantiation and offering (like sl).
• Allow multiple avatars in one client (shared client). Then you can place a
  • system in a common space like a conference room or living room. Or two people can use the same computer (great for kids playing together!)
  • Detect presense of people using rfid, bluetooth, or manual control.
• Avatar presence management, auto-away when idle. If idle long enough, stop rendering and turn screen black (with a message/graphic on it to indicate why). Allow computer screen to turn off or other screensaver or other power management stuff to occur. Resume on any interaction.
• Seperate eye/head/body animations
• Automatically detect when other avatars are in your view, when your avatar's gaze meets another's, and when anoter avatar's body is facing yours, when two other avatars' gazes intersect, and provide a subtle indication to user. I.e. interpret body language to help user (esp. useful if afk or lurking). Or, send explicit signal to other avatars when you select them, etc. On the other side, include controls to automate animating and moving your avatar as you talk (and address other avatars by name), or click with the mouse to select others, etc, detect clusters of avatars and indicate groupings visually in the UI, with controls attached to e.g. join the cluster in a natural way.
• Notification when addressed by name in chat
• Side pane to view and post comments/notes about the world (publically viewable)

• On very first run, TerAngreal joins an Interreality-hosted world (maybe a random sector/neighborhood within a city). This could help people form communities in neighborhoods?

• Allow objects to have position in more than 3 dimensions. Include a control to map those dimensions to the 3 dimensions of the display, or to cycle through combinations of the 3 dimensions [e.g. if an object has position in (X, Y, Z, W, V), then cycle through (X, Y, Z), (Y, Z, W), (Z, W, V), (W, V, X), (V, X, Y). Or through all permutations perhaps.]

• Package graph-layout code in plugins that can be loaded either by omnivos or TerAngreal. It should work through listeners attached to original objects, and laying out 2D or 3D objects to illustrate the graph structure of those original objects.

• Mode where either camera or avatar or both automatically tracks around objects (if possible) as you move (to avoid going "through" objects as you move).
• Mode where you walk or fly "along" an object, i.e. walk or fly along its surface as the "ground".
• HTML (or other display type-- PS/PDF/etc) objects that get displayed in a grouped side panel when you get near them
• Object layers/groups, with a list in a side pane to enable/disable, and override properties such as alpha, apply position offsets, etc. Should layer groups be orthogonal to main scene, or integrated within it?
• Menu of viewpoints in a side pane.
• Controls for adjusting access control, and for asking an avatar inside a room (object) to grant you (temporary?) access ("knock on the door").

For graph node/link layout view mode

• Represent objects as little crystals, and animate them rotating to show different faces showing different aspects of them (with different colors), each facet showing one of its metaobject types, or a subset of its metaobject types.
• See inspiration.com for an example of a really good visual graph-like interface

Operating Forms or Configurations

Forms that the program or the program's window frames can take:

• Normal app with windows
• Full screen (kiosk)
• Side pane applet (in Vista or Gnome Panel)
• Minimize with only a statusbar icon
• Shut down GUI and fork into a daemon (but warn users adequately about what
  • they're about to do! maybe have a component on the root or site that lets an authenticated remote user quit the daemon?)

"Elsewhere" sectors

A useful concept for flexible user-interfaces is that of a "elsewhere" sector; from a low-level perspective, these might be considered alternative views of a sector. The user may bring up an elsewhere view in place of the world view, or as an "overlay" window, by pressing a key or clicking on something on the UI.

On a higher level, these sectors can be used in games to implement the player's inventory, for example, or to provide "tools" that can be used to interact with the "main" world (the 3dui equivalent of a menu).

(originally termed "sub-sites", which is rather incorrect. The term "elsewhere" is also rather vague - it's borrowed from my favorite RPG - but it's good enough for discussion.)

-Lalo

I fail to see the difference between a "sub-site" and what we can do with vobjects normally. A "site" has a very specific technical role in VOS as the bridge for network connections between Vobjects, which is not addressed in the idea above. Can we have a different name, "subsector" maybe?

-Peter

I think Lalo's new term "elsewhere" or alternative views works. Generally speaking, this is just another world-view, just like wxTerangreal can do now with tabs, but displayed next to your main view so you can compare them or copy or move objects between them.

Also, a 2d heads up display window could be implemented using MetaobjectsFor2dGraphics placed within the world, and logically grouped so that client actions (as tetron describes below) could turn them on and off.

-- ReedHedges

Client actions

(separate this into its own page if it becomes too big)

my long term ideas for VOS are to make ter'angreal into essentially the web browser for the 3D internet...

-- PeterAmstutz

Okay. We have hypercards and possibly portals, for walking between sites (and between 3d and 2d sites). But that's not all you'd expect the browser to do wrt interactivity; since the system is fully object-oriented, it's somewhat reasonable to expect the user to be able to send messages to the other side.

Usually, you'd do that by having keybindings, or "hot" areas (buttons) in the user interface.

I think it's important for this design to allow some basic leeway in client design; we have now Ter'angreal which is basically a 3d window on a 2d app (with standard GUI menus and whatnot); but it's conceivable that someone might want a "fullscreen", completely 3d browser, and of course there may be variations we haven't even imagined yet.

So this is a proposed design for these keybindings, buttons, voice commands, etc etc - which I'm generalizing under the umbrella term "client actions". It's loosely based on the way the crossfire client does keybindings, and on emacs.

The core of the idea is that the avatar object - in cases where the avatar is on the server side, which I believe is the norm - or a special proxy object - when the avatar is on the client site - handle a number of "client messages".

For the sake of example, let's consider three such messages - "fire", "jump", and "wield". This is a very simple kill-them game, as discussed in the mailing list. The "fire" and "jump" messages have no arguments, while "wield" can take either a string or a vObject. If passed an object, it has to be "in reach" (in the player's inventory, or near his avatar by some max distance), and it will be selected as the active weapon (putting any previously active weapon in the inventory). If it's not "in reach", an error is returned. If passed a string, the server searches all objects "in reach" to check if any of them answers by that appellation (the exact logic of this is left as an exercise to the reader - but we might have "type" and/or "name" fields, for example). If one is found, it is "wielded"; otherwise, an error is returned.

Now, the first step to make this useful, is to provide the browser with a console, where the user may type "commands" to be sent to the avatar. Since Ter'angreal is already a chat client, we can use the existing chat line, with the IRC convention of preceding commands by a slash, or we can repurpose the line for commands and adopt the crossfire convention of preceding chat with a quote mark, or with the "say" command. (For that matter, Talkative can be refactored to operate "on top of" this subsystem, which is specially convenient because it allows the user to program "canned" speech, like "help, I'm taking fire!")

Of course, the console only allows string arguments. For the sake of user-friendliness, hack-friendliness and script-friendliness, it would be a good idea for all messages to accept strings anyway.

Some convention has to be concocted to separate arguments. Whitespace, shell-style, seems like a poor choice, as it would make users quote pretty much everything. Either a comma or semicolon would be probably better; which one would depend on whether we want to allow the semicolon to separate multiple commands on the same line.

If you have a sub-site representing your inventory, an interactive "wield" may be implemented by dragging the weapon from the "carried" area of the inventory, to the corresponding wield slot. This doesn't actually send a "wield" message; the client is fully dumb about this, it just sends the drag message, and the logic is on the server side.

Next step is to allow the user to bind keys to commands; so you can bind "b" to "apply bow", " " to "fire", etc (or on a fullscreen client, bind "ctrl" to "fire" and " " to "jump"). As simple as that; this is how crossfire does bindings. These keybindings should be per-site; ideally, you should even be able to have different bindings for each "login" you have on a site.

Finally, you can also have a toolbar of buttons "bound" in just the same way.

This system is extremely simple from the browser side, and can be implemented without any client-side code downloaded from the site (so no worries about sandboxing); and it doesn't require very extensive changes to either server or browser.

-- Lalo

Service Directory

The infrastructure for bringing the service directory stuff back is still there, all we need is UI in TerAngreal for browsing service directories (both remote and from local LAN advertisements). This can be done as a simple list of worlds in the directory. However, TerAngreal also has a default local starting world now, loaded from a XOD file on startup. It would be cool to have it also place hypercards or portals in this local world for services and perhaps also your bookmarks. (Bookmarks list could also use the service directory to tell if a bookmarked world is online or not)

We also need a way to have a buddy list for sending private messages. This could be done as a set of service directories each corresponding to both personal buddy lists as well as other kinds of groups (like all your co-workers or all Interreality developers). (Then we need UI for people to specify which directories they want to put their avatar in when starting TerAngreal, creating the directories as needed, and choosing access control policies for that directory, and for their avatar to determine who can tell whether they are online, and also what worlds etc. they are currently viewing -- you might want your friends to know what world you are in so they can come find you but not any stranger; such an access control policy could use the buddy list service-directory to know who your friends are.)

-- ReedHedges

Application Messages and Logging

TerAngreal needs some kind of filtered log window system. You should be able to choose what log level and types to view. For example, you might want to monitor which remote objects are getting information about your avatar or what you are currently doing in-world. In general, since VOS is a very open distributed system, people need ways to monitor how other people are accessing their information in order to craft proper access control policies.

This UI might benefit from knowing ahead of time what the possible log channels are, so it can construct a pulldown list or something. This would require adding a function to VOS's logging API to "register" or add a channel name to a list, and a function to access that list. Or maybe not -- just add to the list when the first message is logged to that channel, and call a listener so the UI can be updated. Maybe the UI should just use the logctrl metaobject interface? (Rename to "sitelog" or "hostlog"?) Then you could have log windows for any number of sites, including local... would maybe also want a combined view as well?

wxTerAngreal also needs a way to display more or less unobtrusive notifications similarly to either firefox or Windows/Gnome system tray notifications: to display info that you may or may not be interested in without interrupting you like a dialog box does; also to indicate what kinds of information exists about the world or a person e.g. misc:metadata properties, perhaps by adding an icon to the status bar or the toolbar with the world URL.

It should also have a way to save views to a log file, or to the clipboard, or to compose an email to us with it attached.

See "Ring Buffer Logging" at http://www.visibleworkings.org for patterns to use in implementing logging such that it is useful to a user (and for tech support). VOS logging does some of that, except that it just sends log messages to output, it doesn't try to conserve space by keeping them in a ring buffer. It would be useful, also, to seperate out "session context" log messages from "event" log messages. The state would be more or less unchanging information about the session -- version numbers, plugins loaded features enabled/disabled, values of some configuration properties etc. These messages should be preserved, even as them ore transient "events" are in a ring buffer.

-Reed

Input and Controls

• Joystick and tracker support:
  • P5 glove support (linux driver/daemon is now available)
  • Make libtracking useful?
  • Extend joystick stuff in CS to N axes with labels, and giving "joysticks"
    • various tags?

  • Support in CS for 3dconnexion Space Navigator http://www.3dconnexion.com/ 3D controller

Extensions and interoperating with other software

• Plugins

• Scripting (see VosScripting ?)

• List of commands in menus that call out to external programs, providing info
  • about currently viewed and selected objects either as command-line arguments, or in environment variables, or piped as input. Optionally save files with current objects' data in some format (e.g. tab seperated or fixed column format) and give those file names to the program as input, and read output in a new view. (E.g. call out to graph visualization/layout programs, data analysis programs, programs that import/export data from other databases, etc.)

Low Priority but Useful

• Optionally collect data about users' systems (i.e. graphics capability) and how
  • they use the software and mail it back to us.
• Built in autoupdater.

Long Term

* Use VOS to represent all GUI elements (revive the old VOSGUI and metatronic code). Can display both local and remote GUIs. Can modify the GUI at runtime. (Save objects in a cod or xod file -- need tools for merging vobject structures to include user modifications and upstream modifications!)

Ideas and Resources

* http://terranova.blogs.com/terra_nova/2008/08/qa-with-google.html * http://clarkbw.net/blog/2008/06/04/activity-is-the-new-download/ * http://madhava.com/egotism/archive/005017.html * http://developer.yahoo.com/ypatterns/index.php * GraphicsInspiration * http://visiblecertainty.com/data_sets

Some Discussion from a long time ago

 (reed_)  you want to preserve the way keys and commands are done in s3?  
 (tetron)  right, I could bring in some of the s3 functionality  
 (reed_)  i think we should use CS-style config files for everything   
 (reed_)  one thing i'd like to do is automate the way settings are done...  ideally the settings GUI could be generated automatically from the config file  tetron)  partly I think we need a vision... what do we want terangreal 1.0 to do?  reed_)  and vice versa
 (reed_)  (so you can actually *save* settings to the files!)   
 (tetron)  (yes) 
 (reed_)  key features: 
 (reed_)  - everything that s3 does 
 (reed_)  - better chat interface, including private IMs, setting personal status info, etc. (basics of IRC and IM clients) 
 (reed_)  - avatar keyframe animation 
 (reed_)  - easy access to companion programs like the GUI editor, etc.

 (reed_)  - portals 
 (reed_)  - proper caching of data, and online caching of lightmaps 
 (reed_)  - content libraries (same mechanism as caching) 
 (reed_)  - keep a little "palette" of objects and object-factories 
 (reed_)  - misc. improvements to graphics and navigation 
 (reed_)  - comprehensive settings/config GUI 
 (tetron)  I agree with all that, what I'm concerned with is the "big picture", eg take one step back from specific features and say "what do we want this to do" 
 (reed_)  well now that we have csa3dl, terangreal is the swiss army knife 
 (tetron)  and "3D chat that doesn't suck" is an acceptable goal :)

 (reed_)  you just use it to explore worlds and socialize 
 (reed_)  and help build world 
 (tetron)  yes 
 (reed_)  (though not to compete with better creation tools like blender, or level editors, etc.) 
 (tetron)  well, being able to pop in premade objects in a far cry from a real modeling program 
 (reed_)  it should be useful as an interface to games and such that don't require special logic or control (e.g. online checkers) 
 (reed_)  and data visualization 
 (reed_)  since csa3dl exists' specialized clients can be made like the prophecy 
 (tetron)  yep 
 (tetron)  oh, I think that we should try and focus on being able to load existing content, too
 (tetron)  VRML being the obvious one, but maybe also Quake maps and such 
 (tetron)  since we don't yet have a huge base on artists banging out door down... we can go steal stuff :)
 (reed_)  we need to determine what the best/most popular quake/doom/unreal editor(s) is(are)
 (tetron)  probably Valve Hammer
 (reed_)  those editors are pretty easy to learn 
 (tetron)  there's a library out there that allegedly reads half life maps 
 (reed_)  so talking about goals of terangreal 
 (tetron)  what we really need are some really cool demo worlds, and step-by-step instructions on how to create your own worlds 
 (tetron)  and how to host them, or upload them 
 (reed_)  terangreal needs to be the interface through which we can play with ideas 
 (reed_)  so it also needs to do stuff like display remote GUI controls (trivial)
 (reed_)  maybe do some 2d graphics 
 (tetron)  we'll be able to display web documents as textures 
 (tetron)  I'm just trying to envision what ter'angreal needs to do (and not do) that I could comfortably have it on my desktop all the time 
 (tetron)  the 3 apps that I most interact with are IRC, a terminal window, and a web browser
 (reed_)  well how can you run it with 3d graphics but not use too many resources? 
 (tetron)  the resource usage issue is the most annoying problem
 (reed_)  that's why i mentioned component architectures, or reusing code in libraries vs. apps the other day at your house 
 (reed_)  one idea is to just "detach" the chat/buddy-list GUI from terangreal 
 (reed_)  and close the graphics part.  I guess you could iconify just that window and pause it.... 
 (tetron)  one possibility would simply be to throttle CPU usage 
 (tetron)  one could iconify the 3D window but still use other windows part of the same uber-app
 (tetron) (although along those lines people also don't like uber-apps that do everything...) 
 (tetron)  (witness the fact that mozilla is being split into several individual apps) 
 (reed_)  but you also might want to have your chat interface on your PDA, and your 3D stuff on your desktop or something 
 (reed_)  we will need to think a bit more about replication, so your avatar/talk objects could really be persistant and never die 
 (reed_)  and you could transfer them from one machine to another 
 (reed_)  (identity too) 
 (tetron)  so we need to think about ways we can have the 3D chatroom add to IRC 
 (reed_)  one thing is an irc bridge that replicates individual users 
 (reed_)  another is something like an infobot or agent 
 (reed_)  so when people come online to an otherwise deserted server, there is *something* to talk to :) 
 (tetron)  the stuff that 3D is going to add will be facial expressions, gestures... 
 (tetron)  that's what "There" is good at 
 (tetron)  that's going to require some character animators to contribute 
 (tetron)  perhaps we can cull some stuff from other projects 
 (reed_)  well you can start with some game models 
 (reed_)  though there your expressions are "standing", "running", "attacking", "dead" 
 (tetron)  but, y'know, for 3D to be interesting I'd like to see your avatar doing something when you type a message, something more interesting than their standing loop 
 (reed_)  like talking 
 (tetron)  I think that's what we need to strive for, is how we can add to interpersonal
interaction 
 (reed_)  you could have two things 
 (reed_)  a "do this while i'm typing" action 
 (reed_)  and actions triggered by patterns in the text 
 (reed_)  like ! 
 (reed_)  :) 
 (reed_)  hmm... 
 (tetron)  another thing is for two users to able to see the same thing, such as a web page or terminal window 
 (reed_)  uh...

 (reed_)  lol 
 (tetron)  syb was talking about how much easier it would be to debug compile problems if you were standing in the virtual world and you could call someone over to look at your terminal :) 
 (reed_)  part of your object creation library in terangreal should be creating hypercards to any data 
 (tetron)  so we have to think about ways of fully takin advantage of user presence in a space 
 (reed_)  or creating visible artifacts by cut and paste or drag and drop 
 (reed_)  ie, a plane with some text on it 
 (tetron)  post-it notes :) 
 (tetron)  these are the sorts of ideas we need to be thinking about... not features, uses