------------------------------- Interreality/VOS Long-term Ideas, Wish list ------------------------------- For near-term bugs, missing features, new features, see the online bugzilla database at . THINGS WE COULD DO NOW OR SOON: ------------------------------- * Plugin for GIMP to edit an image in a Property * Get URL of target object(s) from env, or prompt with a dialog * then import existing image metaobjects as layers (one layer if target is an image not a frame) in a new image in gimp * 2 modes: synch periodically (choose rate or after every change) or when a button is pressed (with key shoutcut if possible) * Plugin or script in Blender to edit a scene in real time * CrystalZilla ready for use in TerAngreal. (See http://interreality.org/projects/crystalzilla) * VOS-based shared GUI system, with wxWidgets client. See VOSGUI, Metatronic, Metalurgy from pre-0.21 CVS, plus add: * Float/Formatted sliders and spin controls in wx * scrolling containers * flag input boxes as written/not written ? * drag and drop stuff * HTML front end * Remember window positions (key on object site name?) * Object Type Definitions (OTD): RDF documents describing MetaObject types (currently partially designed) * tools for applications to recover remote child objects after quit and resume. (i.e. save remote child urls to disk in sigkill handler and reading them back, finding them, and re-inserting them at restart, could just be implemented by a pair of helpful functions; Or, children to recover parents that vanish and come back (due to some failure on a server -- probably this one) * We should also more completely implement a policy of disconnecting any peer sites that violate the protocol, by (for example) lying about their length fields. * Dev C++, update project builder files. * For mesh: * run system commands, with environment set (like a real shell), and to pipe input/output. e.g. "property read | !grep foo" * pipe any command to any other? (use string streams) * merge command (preserve all objects) command reed? * overlay command (ignore children from first object with same name as children in second object) reed? * A3DL Converters and/or CS Importers and/or COD export from native tools for these formats: * Alice3D (http://www.alice.org) * Unreal Editor * Quark * Valve * GTKradiant?? * Maya? * AC3D? * Lightwave? * Hash Animation Master? * FBX (http://www.alias.com/eng/products-services/fbx/index.shtml) * Terangreal: * "flashlight"/"headlight" (useful for poorly lit worlds) A BIT LONGER TERM IDEAS ----------------------- * Tools for hooking together common object-linking actions triggered by metadata search queries (and future listener updates added by the queries). So you can search for objects, and keep the results as children of a container object, updated whenever a new object is added somewhere else in the scope of the search to add a new link in the container if it matches the search query criteria. * For mesh: * run all commands in a scripting language interpreter (perl, python, ruby), also providing the full vos api * GUI for a3dlconverter, omnivos, etc. if they are invoked with no parameters (for a3dlconverter, just a load then a save dialog; for omnivos, a more complex configuration dialog, plus a log window: we will need a little bridge class that is a stringstream to accept LOG output and execute wxWindows logging functions) * "details" subobjects: objects that are loaded/revealed when the user gets close to the object (by some specified distance, in a 3D app), or otherwise asks/ wants the details. in gui and text, this could be a little "expand/contract" ("show/hide") thing (arrow or lever). in 3D use LOD: see terangreal/TODO * Log talkative messages in a nice interchange format. For example, see http://www.threadsml.org * Improve Chat GUI: display chat as threads with a tree control, more tools to edit your avatar "profile": add "info" pages. (pictures, urls, url-to-content-with-typechain, FOAF form, misc. things like "mood"); locate other avatars on LAN; create chat rooms (and advertise as service); some way to work with other IM services/ clients? * idea for talkative: instead of a special talk message, store "last message" as a property that senders can set and multiple talkatives can share. needs a special format to include the diff. components of a talk message, though. Probably this idea will not be implemented. * Message forwarding: filter messages, include max-hops and hop count fields, etc, or forward only on some condition. * Propagating P2P Messages/Queries/Searches: * recursive p2p query. two children: "associates" and "data" match query expressions against data with selected c-name 
        if match (all or any, depending on match-mode flag): 
            process message
            if end-on-match flag: end
        else if end-on-no-match flag: end
        maxhops--
        if maxhops == 0: end
        for each child of "associates":
             send query to child
* New object type: friendly * list of friends, each with: * GUID (FOAF email hash) * URL to static FOAF file * URL to static HTML info (generated from FOAF/this object) * avatar object * directory object to search if friend is offline (avatar object missing) * anything from FOAF etc. missing from avatar type * diary/blog entries or URL of * comments to others' blog entries or URLs of * other properties, like interests, etc. (queryable) * identities * utility functions to initiate queries of your friend network * local UI (with talk interface included). integrate into terangreal? how? * Display graph visually (3D?) * shared library (libvosfriendgui)? plugins for chatgui, friendgui, other future applibs? * Metadata type: * add "total-data-size" property for grouping objects (like a3dl:sector) * add "expires" date (maybe with a comment here too) * add property for confidentiality and other "use" tags * Sound * Should sound store current state in a property, or send special play, pause, etc. messages? How is current time point stored? How can new clients connect? * Maybe a hybrid: use messsages for play, pause, etc. but new clients can query current state (and recieve e.g. play message containing starting time point in the stream). * One question, is should we assume that VOS property updates are delivered to the listener ASAP? Or should we say that messages are always delivered ASAP, but property updates may take some time? Does VOS need priority numbers on messages? * speech comm. using http://www.speex.org/ or something similiar. * Property: * Math expressions in extrapolated property, with other values available as well as current value and time step. * Set max and min for extrapolated property values, and what to do when the min or max are reached (loop, stop, ignore, reverse: or should this just be a math expression?) * include an md5 hash or some other fingerprint for quick comparison of contents; property's methods can be conditional on matching a given fingerprint. * Subclasses of property that: * initially is just a proxy to another property until you tell it otherwise, in which case it duplicates the other property's value, and replaces it in certain parents. * is a "copy on write" property. also need to have a particular parent in mind for the copy/replace. * 2D images: * Metaobject types for layers of 2d images: bitmaps mostly, also some primitives * "frames" define coordinate systems (containers), then bitmap, primitives, text, invisible guide lines, etc. * Design Based on SVG in part, and Flash, SMIL, etc. * Plugin or add-on for Inkscape to use Vobject backend * SVG, Flash, SMIL support for images (as well as bitmaps) * HyperVOS: * A mode for the hypervos web server that provides forms and cgi on Vobject web pages for manipulating the objects, and editing property contents. * hook "filter" scripts onto properties (post-write or pre-read). e.g, a script which closes all open tags in the property data. these scripts are e.g. in python or perl and are stored in other properties with appropriate access control. (or probably this can be a general omnivos plugin: script_hooks) * Generate RSS * Create a CD (a small 185 MB cd?) with VOS installers, source code, documentation, and whatever else will fit, like: mailing list archives, CrystalSpace source code and demos, wxWindows source code & docs, Blender binaries, docs & source code, Wings3D, Boost source code & docs, etc. And beyond that, modify a Gentoo boot CD to include VOS and terangreal, perhaps to run at boot, for a real slick demo. Pass them out to everyone, through the mail, conferences, etc. RATHER LONG TERM/BLUE SKY STUFF: -------------------------------- * XML! Everyone uses XML.... * Simple tools: 1. fetch: given some optinos, make a search query or manually walk some subset of vobjects and output an XML file for it. 2. publish: parse an xml file and create objects, or modify existing objecs, remove missing ones, etc. * Need to look at DTD or schema to know if a property should be an attribute or element. * Need to look at OTD to know what an attribute or element's type should be. * Live editor: write a tool or add to an existing one the ability to fetch and publish online, while presenting "unfolding" XML nodes/attributes/cdata content * Now you can use XML types as a base for metaobject types (and vice versa). * Use RDF * ?? Plugins for Eclipse IDE (http:www.eclipse.org): * build code using GNU autotools, libtool, and make on Linux and cygwin * Edit and create OTDs, and create boilerplate code for VOS * Edit GUIs and live object trees. * Need Java bindings! * Gnu Enterprise (http://www.gnuenterprise.org) * Use VOS as a datastore in the "Common" library (used by the AppServer) * need Database like interfaces, and property "view" remapping. * Use Forms tool to create Metatronic GUIs * Run a GUI server importing different file formats from interface design tools, e.g. wxDesigner, Glade (GTK), QT Designer, Apple Interface Builder, MS Developer Studio, etc. * VOS plugins for matlab and/or mathmatica and/or other systems where shared 3d (or 2d) visualizations are useful * Write a plugin to these programs so that they can run a VOS server in real time: * Blender * Alice (http://www.alice.org) * Verse (Quel Solar) * Maya?? Possible? * Animation Master? * 3D/VR Apps * general robustness when faced with malformed world property structures. * Load a COD as avatar * Shader programs (new CS renderer will support) * Also attach shaders to sector to be used by all objects in it? * Turn on/off shadows on genmeshes (maybe have a property for it?) * Texture layers: * If a texture layer is a solid color (float-vector), then make a solid color texture to go there. * more options for masks and filters? * optional property to manually set render priority (layers)? * Snow: select lighting on/off ? * Canned proctex layers (fire, water, plasma, different kinds of noise), incl. animation. * animated/movie textures (see Ogg Theoraa and Helix). * SVG Textures (see http://librsvg.sourceforge.net/ and http://www.w3.org/Graphics/SVG/) (write as a CS plugin, or maybe handled by pete's mozilla plugin. probably should be a proctex for adaptive rasterizing.) reed, maybe * new layer blend mode "complement" (need to modify CS) * edge effect (shader program can do this?) * Vertex Colors * flat shading flag * Doom style sprites * essentially 2d, but with a different image for various angles. use 2, 4, 8 or 10 faces (front, back, left, right, front-left, front-right, back-left, back-right, top, bottom * A novel implementation would be to use OpenCV from Intel to interpolate between the sprite faces. http://www.intel.com/research/mrl/research/opencv Treatz * Splines/Nurbs * Subsurface polys * flag an object to follow the camera around rather than be rendered at a specific position? (use CS_ENTITY_CAMERA in crystalspace; but also set correct rendering priority as well...) * fractal objects (see the IFS plugin in the Gimp for 2D) * LOD (level of detail). use CS's "static LOD" stuff -- see manual see also the "details" idea in the main vos todo * Set render priority ("layer") * Light lists * Terangreal: * Joystick and tracker support: * P5 glove support (linux driver/daemon is now available) * Make libtracking useful? * Extend joystick stuff in CS to N axes with labels, and giving "joysticks" various tags? Use Add/Find methods in iEvent instead of the old joystick struct? * action queue: store actions in a list or mask, and apply them when finished. or store them as a macro. or queue up actions, that are applied to an object that is finally selected. will need to add some more abstraction around actions, with a "do" function, passed an action function pointer (perhaps abstracted with typedef and a bunch of #define'd symbols). * Keys to "Orbit" selected object reed * select multiple objects (with one "primary" selected object) * Keys/GUI to store "palette" of objects (perhaps conjure objects as children of the avatar, then have command for reparenting those objects into the world?) * Customizable menu of "tools": commands to run with vos environment variables (with some defaults like gui editor, xterm -e mesh, etc.) reed? * ?? Python "startup script" that sets up some variables from the env vars: terangreal could put the path to this script in the PYTHONSTARTUP env var before running a "tool". If one of those tools is "xterm -e python" then it will run before entering interactive mode. * Mesh library; terangreal can send commands to it? 
            <dfryer> say, can you access mesh from within terangreal?
            <reed> sort of
            <reed> why?
            <reed> i mean, you can use the /system command to run it in an xterm
            <reed> i think "/system xterm -e mesh" will work
            <dfryer> it would make sense to be able to manipulate a VOS environment through
            one channel
            <dfryer> well, thinking of the future, if mesh could be extended to understand
            different object types, or something like that, it would make sense to be able
            to access that from terangreal
            <reed> but you mean, have a mesh-like command line inside terangreal?
            <dfryer> yeah, basically
* Apply overall "filters" to graphics/lighting of a sector: affect amount of color/saturation; overall brightness, contrast, etc. color filters? (For software rendering, could specify a function to apply to each pixel mbefore drawing? maybe somethnig similar for hardware renderer too?) * "distance dimming" Server tools, extensions and demos * 3D proximity sensor, etc. (make a library of "sensors") * live weather world Planned RH * Self-editing world: includes GUI component to edit its own (perl) code, then run the new world with a hypercard in the old one. * maitre'd. simple agent that stores messages for people, welcome messages, etc.

Caching:

* Better caching: store between sessions and don't download property if a value is the cache and the values match (need a compact "fingerprint" on the property) * Also check "permanent" caches for user and on system. (Then you could have "libraries" stored locally for speed) * should cached property values be stored by URL or "fingerprint" (the latter supports copying contents) * if a binary search by url is faster, then store in both schemes, and fall back on the "fingerprint" lookup. otherwise, just store in a hash of "fingerprints". * Save lightmap on server neil 2:23:<reed_> you want to preserve the way keys and commands are done in s3? 2:23:<tetron> right, I could bring in some of the s3 functionality 2:24:<reed_> i think we should use CS-style config files for everything 2:24:<reed_> one thing i'd like to do is automate the way settings are done... ideally the settings GUI could be generated automatically from the config file 2:24:<tetron> partly I think we need a vision... what do we want terangreal 1.0 to do? 2:24:<reed_> and vice versa 2:25:<reed_> (so you can actually *save* settings to the files!) 2:25:<tetron> (yes) 2:25:<reed_> key features: 2:25:<reed_> - everything that s3 does 2:26:<reed_> - better chat interface, including private IMs, setting personal status info, etc. (basics of IRC and IM clients) 2:26:<reed_> - avatar keyframe animation 2:26:<reed_> - easy access to companion programs like the GUI editor, etc. 2:26:<reed_> - portals 2:27:<reed_> - proper caching of data, and online caching of lightmaps 2:27:<reed_> - content libraries (same mechanism as caching) 2:27:<reed_> - keep a little "palette" of objects and object-factories 2:27:<reed_> - misc. improvements to graphics and navigation 2:28:<reed_> - comprehensive settings/config GUI 2:28:<tetron> I agree with all that, what I'm concerned with is the "big picture", eg take one step back from specific features and say "what do we want this to do" 2:28:<reed_> well now that we have csa3dl, terangreal is the swiss army knife 2:29:<tetron> and "3D chat that doesn't suck" is an acceptable goal :-) 2:29:<reed_> you just use it to explore worlds and socialize 2:29:<reed_> and help build world 2:29:<tetron> yes 2:29:<reed_> (though not to compete with better creation tools like blender, or level editors, etc.) 2:30:<tetron> well, being able to pop in premade objects in a far cry from a real modeling program 2:30:<reed_> it should be useful as an interface to games and such that don't require special logic or control (e.g. online checkers) 2:30:<reed_> and data visualization 2:30:<reed_> since csa3dl exists' specialized clients can be made like the prophecy 2:30:<tetron> yep 2:34:<tetron> oh, I think that we should try and focus on being able to load existing content, too 2:35:<tetron> VRML being the obvious one, but maybe also Quake maps and such 2:35:<tetron> since we don't yet have a huge base on artists banging out door down... we can go steal stuff :-) 2:35:<reed_> we need to determine what the best/most popular quake/doom/unreal editor(s) is(are) 2:35:<tetron> probably Valve Hammer 2:35:<reed_> those editors are pretty easy to learn 2:36:<tetron> there's a library out there that allegedly reads half life maps 2:37:<reed_> so talking about goals of terangreal 2:37:<tetron> what we really need are some really cool demo worlds, and step-by-step instructions on how to create your own worlds 2:37:<tetron> and how to host them, or upload them 2:37:<reed_> terangreal needs to be the interface through which we can play with ideas like this 2:38:<reed_> so it also needs to do stuff like display remote GUI controls (trivial) 2:38:<reed_> maybe do some 2d graphics 2:38:<tetron> we'll be able to display web documents as textures 2:39:<tetron> I'm just trying to envision what ter'angreal needs to do (and not do) that I could comfortably have it on my desktop all the time 2:41:<tetron> the 3 apps that I most interact with are IRC, a terminal window, and a web browser 2:41:<reed_> well how can you run it with 3d graphics but not use too many resources? 2:41:<tetron> the resource usage issue is the most annoying problem 2:41:<reed_> that's why i mentioned component architectures, or reusing code in libraries vs. apps the other day at your house 2:42:<reed_> one idea is to just "detach" the chat/buddy-list GUI from terangreal 2:42:<reed_> and close the graphics part. I guess you could iconify just that window and pause it.... 2:42:<tetron> one possibility would simply be to throttle CPU usage 2:42:<tetron> one could iconify the 3D window but still use other windows part of the same uber-app 2:43:<tetron> (although along those lines people also don't like uber-apps that do everything...) 2:43:<tetron> (witness the fact that mozilla is being split into several individual apps) 2:43:<reed_> but you also might want to have your chat interface on your PDA, and your 3D stuff on your desktop or something 2:44:<reed_> we will need to think a bit more about replication, so your avatar/talk objects could really be persistant and never die 2:44:<reed_> and you could transfer them from one machine to another 2:44:<reed_> (identity too) 2:47:<tetron> so we need to think about ways we can have the 3D chatroom add to IRC 2:47:<reed_> one thing is an irc bridge that replicates individual users 2:47:<reed_> another is something like an infobot or agent 2:48:<reed_> so when people come online to an otherwise deserted server, there is *something* to talk to :) 2:48:<tetron> the stuff that 3D is going to add will be facial expressions, gestures... 2:48:<tetron> that's what "There" is good at 2:49:<tetron> that's going to require some character animators to contribute 2:49:<tetron> perhaps we can cull some stuff from other projects 2:49:<reed_> well you can start with some game models 2:49:<reed_> though there your expressions are "standing", "running", "attacking", "dead" 2:50:<tetron> but, y'know, for 3D to be interesting I'd like to see your avatar doing something when you type a message, something more interesting than their standing loop 2:50:<reed_> like talking 2:50:<tetron> I think that's what we need to strive for, is how we can add to interpersonal interaction 2:50:<reed_> you could have two things 2:51:<reed_> a "do this while i'm typing" action 2:51:<reed_> and actions triggered by patterns in the text 2:51:<reed_> like ! 2:51:<reed_> :) 2:51:<reed_> hmm... 2:51:<tetron> another thing is for two users to able to see the same thing, such as a web page or terminal window 2:51:<reed_> uh... 2:52:<reed_> lol 2:52:<tetron> syb was talking about how much easier it would be to debug compile problems if you were standing in the virtual world and you could call someone over to look at your terminal :-) 2:52:<reed_> part of your object creation library in terangreal should be creating hypercards to any data 2:52:<tetron> so we have to think about ways of fully takin advantage of user presence in a space 2:53:<reed_> or creating visible artifacts by cut & paste or drag and drop 2:53:<reed_> ie, a plane with some text on it 2:53:<tetron> post-it notes :-) 2:53:<tetron> these are the sorts of ideas we need to be thinking about... not features, uses - Listener priority. When becoming a listener, the new listener can set it's own priority. When notifying, high priority listeners go first. Change priorities dynamically as well. - Sites could form a single Borg-site by host-aliasing each other, and authenticating to each other, and talking to each other. (for example, lots of local sites talking with shared memory or domain sockets VOP sessions (VOS does not need to operate over network streams, remember, and also being authentic host aliases of each other). See next node about replication. - Object replication/relocation. There is some of this in the "policy" metaobject. Tetron has also done some work on this (see the "renaldo" game in the CVS tree). One useful feature is to create objects locally, then relocate to another site when you quit. Or, when a site is heavily loaded or detects bandwidth saturation, to relocate objects to other sites. Need some protocol for negotiating with sites for relocation, and for recovering from the site you just sent your objects to suddenly dissapearing (so all "relocation" should really be "replication, then hide the original but keep it up to date?). This would only work for passive data objects, not "active" objects. But imagine a low bandwidth site on a modem which is the "originator" of the world, and as more people join, the objects start spreading out to the people with more bandwidth. Then as people leave, they merge back into earlier "authoritative" origination sites. (this can be represented by a hierarchical graph showing the replication history, followed back up to merge back to the prime state) - Related is the idea of simply doing redundent "gratuitous" replication and referring users to the duplicates randomly. (See previous Borg note) - Users with reliable broadband connections could run helper servers that accept replications from other servers (friend's servers, favorite worlds, anything on the interreality.org service list, etc.) to balance loads - Adapt to bandwidth and latency problems by indirect object addressing: if some user has less latency than others (WRT you), send all messages through them, and let them forward messages to others (though replies go back to you directly). Access control problems here? Not sure if this is a good idea or not. Would also need to be pretty smart about choosing a user to forward through (and deal with him ignoring your messages). Might require some infrastructure changes. - Micropayments and subscription support. It would be a good experiment to try and implement a micropayment system in VOS, a'la Xanadu. It would probably be an extension to Property, with subobjects to the property containing price and certificate information, as well as descriptions of what you're buying, a "free" version, etc. Secure (TLS/SSL) transactions would of course be required. On the other hand, subscriptions have been more successful on the net so far, so some support for access control based on looking up TLS certs in a database of subscribers would be good too. Note that Paypal and even Bitpass are'n't really micropayments, just very small payments. You need and intertwingled medium for Xanadu-style mircropayments to be possible (and you need to make the payments REALLY small). Whether or not people will actually accept uPayments is another story entirely. Clay Shirky argues againt micro/mini payments here, not a bad article: http://shirky.com/writings/fame_vs_fortune.html Here is an interesting new company as well: http://www.peppercoin.com - we need to network plants with water sensors and avatars that make goofy speach bubbles for being watered. and various other cheap sensor arrays that display in a virtual world. X10 interface ? :) :) * Use parapin or this: <http://www.makingthings.com/products/products.htm> * Set up a config file to map sensor readings or pin states to property values Some links, for information 3D Renderering Libraries: http://plib.sf.net http://opensg.org (not to be confused with openscenegraph) http://openscenegraph.org (not to be confused with opensg) http://ogre.sf.net Libraries for tracking devices: VR Juggler OpenTracker