
 0. Preface
 ==========

  0.1. Some General Remarks
  =========================

   First of all, I have to apologize myself for my somewhat limited
   language skills, which were even worsen for parts of this document
   due to the nightly hours of my "session".

   This challange will require some time, but will be managed at the
   end of the day, at least to an acceptable degree.

   Some chapters still has to be completed, but are already present as
   reminder. This document is used as an permanent "realtime" and
   iterative Concept-Design-Development-Test guide, though the
   chapters identified as the most important for that cyclic development
   process are written first and are already present.

   The target of this description was to reach a level, which excludes
   the described new application as well as any additional basic
   functionality from "beeing claimed as a patent" to be used as a
   blocking point. For "selfless reasons only", of course.

   As far as I can see for now, this target is reached.

   The given description should provide a detailed and easy to proof
   base for any defensive act on patents related to nested stacks of
   PMs and VMs - Physical and Virtual Machines - and any additional
   basic application to manage them, filed from now on. The same should
   be true for the application of complete VMs as an ordinary part of
   a Software System, an extended SW-component, which itself could be
   extended by a variety of means. The Management of relocation of
   services for reasons of energy efficiency is included in this
   statement.

   This is different to the common view of a so called "Virtual
   Appliance", which is defined and unsderstood as a complete Software
   Application, and due to it's closed structure cannot host any
   embedded - nested - virtual machine itself, not only to technical
   reasons, but to definition of concept too. Particularly, it is not
   foreseen for a "Virtual Appliance" to be managed and extended in a
   modularized manner by the end-user, which became actually a
   "software-mechanics" redesigning it's services almost
   independently. 

   One example for this is a therefore designed simulation
   software-system, which is modularised by grouped VMs, which offer
   an online-reconfigurable software application. The components of a
   thoroughly designed system could be relocated on the level of
   submodules without any interruption of their process, thus without
   any dataloss. Thus in case of a requird maintenance within an group
   of PMs a longrunning job could be continued without any losses.

   The common view for now is the state-of-the-art application of VMs
   for a single flat-level of consolidation, without the design
   view as a componet replacing a process, but as a complete
   application.

   So, astonishing, but it seems though, that this document and
   software is the first one which implements this new view
   at all, or at least systematically, including a solution of the
   namebinding for "hidden" VMs within another.

   The application for standardisation of some keyparts will follow
   closely.

   Therefore any given statement in this document is a design
   description of an implemented and tested feature within the
   "UnifiedSessionsManager" as could be reviewed and tested with these
   distributed software. The software itself is under daily operations
   with utilization and ongoing development of any of it's features.

   The nested VMs are based on Xen, QEMU, and VMware. KQEMU is for now
   not used within the stack. Support for OpenVZ and VirtualBox seems
   to be the next. The implementation is for now completely based on
   bash and the well known shell utilities. Thus they might be easily
   adaptable by almost anyone - with knowledge of at least medium
   level of bash-shell-programming.

   So, no sophisticated environment is required, the  most of the
   development and tests are performed on outdated CPUs without
   support of HVM.

   As stated later, any invention described or just implemented within
   this document is provided on the base of GPL3.

   Because it is in nowadays impossible to check any filed new
   feature, there might be already someone who had filed any of the
   implemented features. Therefore the recipient and user of this
   software is responsible for the usage as stated within the GPL3, no
   guarantees will be given by the author.

   And of course, this artifact is not the only one on the stack.


  0.2. Legal
  ==========



       All mentioned Red Hat products and their registered names are
       Trademarks of the Company Red Hat, Inc.

       All mentioned SuSE products and their registered names are
       Trademarks of the Company Novell, Inc.

       All mentioned VMware products and their registered names are
       Trademarks of the Company VMware, Inc.

       All mentioned Google products and their registered names are
       Trademarks of the Google, Inc.

       All mentioned Sun products and their registered names are
       Trademarks of the Sun Microsystems, Inc.

       All mentioned Microsoft products and their registered names are
       Trademarks of the Company Microsoft, Inc.

       All mentioned Intel products and their registered names are
       Trademarks of the Company Intel, Inc.

       All mentioned AMD products and their registered names are
       Trademarks of the Company Advanced Micro Devices, Inc.

       Xen is a trademark of XenSource Inc.

       QEMU is a trademark of Fabrice Bellard.

       i4p is a trademark of Ingenieurbuero Arno-Can Uestuensoez.


       If some is forgotten, it will be added immediately.


  0.3. Some personal AD
  =====================

   0.3.1. Motivation
   =================

     The initial motivation was the personal reuqirement to a smart
     and enhanced work environment for personal use only, of course.

     But when putting things together the vision of the advantage
     could be reached uprises very soon. So the technical motivation
     was to solve the requirements of the very soon future.

     The second, but much more relevant idea was to give something
     back to the community of the countless unknown donators, from
     which I benefit meanwhile so much, that I felt something has to
     be done.

     Therefore I decided to bring the internal, somewhat
     "genious-required-for-operations" toolset into a state, where it
     could be distributed widely and (hopefully) used by anyone
     with a little introduction.

     I would be glad, if reality would proof, that I cope that
     reqirement at least basically. 

     But of course, business is never absent, same for me. So calling
     this chapter "Some personal AD".



   0.3.2. Professional Services
   ============================

     My personal background on commercial software development begins
     at 1986 and includes major projects as well as one-man projects.

     The offered services include:

       - Software development for distributed and networked systems,
         mostly on UNIX basis within multi-threaded environments. 

         The applied programming languages are not only bash, but
         mainly C/C++ and Java on a more advanced level.

       - Design, development, and operation of large-scale and highly
         sophisticated networked environments with a longterm standing
         experience as might be reflected within ctys.



   0.3.3. Open Source-Projects
   ===========================

     This project is the first distributed to the public. 

     One already at sourceforge registered project is "zamklibant",
     which meanwhile became an extended appliance builder for a
     unified and distributed environment. This is planned to be
     distributed next.

     Several security and networking related projects might probably
     be distributed too. These range from the usage of standard PCs to
     embedded systems on basis of standard components and some
     classical controller based boards, preferably on ARM basis.



   0.3.4. Source-Distribution Policies
   ===================================

     The source code is distributed in general on basis of GPL3.

     The main distribution site will be sourceforge, with probably
     additional sites such as berliOS.

     In addition some pre-releases and temporarily documentation still
     based on commercial tools will be distributed by the site
     "UnifiedSessionsManager.org".



   0.3.5. Contact
   ==============

     - By Mail

      - For maintenance: acue_sf1@sourceforge.net

     For any additional information and support the seach string
     "Uestuensoez Arno-Can" might match the projects with my obviously
     world wide and all-time unique personal ID, natively written with
     "Umlauts".




  0.4. Acknowledgements
  =====================

       And, of course, I want to thank VMware for supporting their
       excellent VMware-Server and VMware-Player for free.
       The VMware-Workstation product initiated to my mind a major 
       step of change and inspired a lot how software is commonly 
       used and developed.


       Many Thanks to Mr. Fabrice Bellard for his QEMU, which is the
       only and one test base for me to demonstrate a nested stack of
       VMs and it's integrated addressing including state propagation
       algorithms for now.


       Great thank to the inventors of Xen athe the university of
       Cambridge. UK, for their efficient VM.


       And, of course, I would have probably no chance without 
       "googling", so, even though it has to do something with 
       business, many thanks for bringing the information of the 
       whole world - and as soon as contacted of the remaining 
       universe for sure - to my dektop. Hopefully I cope the current 
       amount before the remaining universe comes into the scene.
       

       I am meanwhile an enthusiastic user of CentOS/RHEL and OpenBSD, 
       so I am glad having the opportunity to express my thank this 
       way to all supporting persons an companies. Particularly RedHat
       Inc. for their actual open minded distribution policy and the
       CentOS team for their great work, and the OpenBSD team for
       their ongoing support for a base of real security.


       And, last but not least, I want to thank very, very much to all
       the countless contributors for the numerous excellent Open- and
       Free-Software I use. Hopefully I can express my commitment and
       gratefullness with this piece of software, and my next following
       projects.


       And finally I would like to express my thank to my friend Dirk
       and his wife Gisela, for their patience and enduring support.
       Their support at all enabled me reaching this milestone, despite 
       of all the various and countless challanges and throwbacks to 
       be managed.



       Arno-Can Uestuensoz
       Munich, Germany
       January 2008

 1. Introduction
 ===============
  1.1. The Big Picture 
  ====================

   1.1.1. Motivation
   =================

    The origins of the development are set up when handling a group of
    machines with an array of monitors. This became somewaht urgent
    when the first Virtualization product came into the scene and the
    management of the monitor array with 7 monitors and multiple
    sessions for developing and testing became somewhat cumbersome.

    The different interfaces for the session utilities, which range
    from a simple SSH-CLI session, a VNC desktop, to proprietary
    WMware consoles. Where the usage of VNCviewer for VNCserver, Xen
    Domains, and VMware WS6, each slightly different, grew to be
    almost out of order. The common usage of configuration file paths
    as a unique ID for virtual machines, but a dynamic numerical ID
    for X11, and DISPLAY or ports for VNC, did not eas the seamless
    handling. Not to speak about variants within VNC itself.

    The bunch of security mechanisms applicable did not help to
    concentrate of the application either.

    Last but not least the openning of a sessions window on the users
    desktop, the pre-configuration of login-desktops with automated
    logins to multiple physical and/or virtual machines, including
    their automated on demand start-up was missed bitterly too.

    And of course, resulting some little extras like integrated
    on-demand wake-up by standard Wake-On-Lan in the green era,
    and the recognition of nested phsical and virtual machines was
    desired soon.

    And, of course, it should be easily adaptable and expandable by
    just basic admin skills, such as the usage of bash only.

    Not to mention and hopefully almost trivial user interface,
    lacking any sophisticated pre-training, has to be there.






    --------------------------------------------------------------

         So, a solution is now presented here, that solves 
         all these very natural and obvious requirements 
         and much more.


         The "UnifiedSessionsManager" a.k.a. "ctys" is 
         targeted to be:



                 "ctys" the "Swiss-Army-Knife"


           for any kind of remote sessions from and to 



        physical and virtual machines in a  nested VM stack.



          Including stack and basic facility management.



         It fits for small environments, scaling from one 
         machine with one monitor and a few sessions, as 
         well as to the proven sophisticated usage on 
         environments with hundred and more active sessions 
         handeled on monitor arrays with almost a dozen or 
         more screens for each user.


    --------------------------------------------------------------


   1.1.2. Abstract
   ===============

    "ctys" is a unified and simplified shell-interface for usage and
    management of remote sessions and batch-setup of distributed
    desktops.



    "ctys" supports management of sessions

    - on local and remote, physical and virtual machines

    - client processes, which could be executed on local and/or remote
      machines by ConnectionForwarding and/or DisplayForwarding 

    - client windows and desktops by handling positions and sizes on
      multiple displays and multiple desktops

    - secure communications using SSH only by transparently
      configuring X-forwarding or openning locally forwarded ports 



    The standard distribution contains by default plugins for X11,
    VNC, Xen, QEMU, and VMware-Workstation/Player/Server.


    Extended online help is embedded, by providing advanced context
    specific help on call interface and supporting the actual manual
    as printed document from standard command line by "ctys -H print".


    The architecture of ctys is targeted for open and easy extension
    by writing plugings as bash modules. Therefore ctys defines - and 
    is implemented completely based on - dynamic loadable and self-
    configuring bash-plugins, which serve as thoroughly documented
    patterns.

    ctys itself is assembled by supported common generic libraires, 
    CORE modules and bridging modules a.k.a plugins. 


    Specific help for developers is provided by embedded help for any
    call interface from standard call. A comprehensive debugging
    feature with control of various severity levels and code-ranges is
    supported, the output for clients and server components could be
    selected seperately.


   1.1.3. Feature-Sum-Up
   =====================

    This release:
    -------------

     TARGET_OS      = Linux: CentOS/RHEL(5+), SuSE-Professional 9.3
                      OpenBSD(4+)

     TARGET_VM      = VMware:WS+Server+Player, Xen, QEMU

     TARGET_WM      = X11/Gnome

     NAMESERVICE    = Integrated Multi-Stack-Addressing by CONF-File,
                      PATH, TCP/IP, MAC, UUID, LABEL

                      NFS-Based load distribution by extendable COST
                      calculations

                      Tree-Based "Attribute-Value-Assertion" for
                      VM-Conf searches, ENUMERATION

                      Enhanced awk-regexpr for addresses

                      Performance-Enhancements and Configurable
                      Scopes: FileDB(fdb) + CacheDB(cdb)

                      Preconfigured Access-Groups

     SECURITY       = OpenSSH-Dynamic-Forwarding+Tunnels,
                      EncryptedConnectionsOnly
 
                      Nested-Encryption,
                      OpenSSH-Static/Dynamic-VCircuits

                      PKI-CAExt ctys certificate generator/distributor

                      IPSec

     UI_CLI           bash, Remote-Consoles

     UI_GUI           X11-XClients/XServer/Desktops/ 
                      Workspaces-bash-NameBinding

                      DISPLAY and CONNECTION-Forwarding

                      Desktop/Workspace-Management

                      Display-Arrays/Xinerama

                      geometryExt-Logical Screens, Windows and
                      Workspace/Desktop Addressing


    Next releases:
    --------------

     TARGET_OS      = Ubuntu, OpenSuSE, FreeBSD, NetBSD, Solaris
                      (CPU-QEMU:ARM+PPC)
                      (uCLinux(QEMU:ARM+PPC), eCOS(QEMU:ARM+PPC))

     TARGET_VM      = QEMU, (OpenVX)

     TARGET_WM      = fvwm, xfce, KDE

     NAMESERVICE    = LDAP-DB/Access+Distribution,
                      Nested-VM-Stack-Addressing
 

     SECURITY       = Nested-Encryption,
                      OpenSSH-Static/Dynamic-VCircuits

                      PKI-CAExt ctys certificate generator/distributor

                      IPSec

     UI_CLI         = Remote RS232-Console-Server
                      Perl

     UI_GUI         = Sub-GRID-Management, Encrypted-Remote-DISPLAY-Redirection
                      Nagios-Integration






   1.1.4. The Vision
   =================

    The vision is to offer the user a seamless environment, where
    he/she can utilize a distributed environement in a comprehensive
    and sophisticated manner for it's business. 

    Without the neccessity of more than basic handling or recognition
    of the used workenvironment.

    The end of the story would be the integration of complete
    virtualized appliances as common part of an ordinary standard
    software, which itself might be an virtual appliance too.

    So a virtual appliance will be to my mind nothing else than an
    extended software component, offering a much more convenient
    encapsulation and design facility, than now.

    This might become obvious imediately, when you do a forcast on the
    number of CPU cores beeing provided by each CPU in the year 2010,
    2015, or 2027. Just compare for the latter the performance of a
    6502/Z80+64kRam wih 144k FDD from 1982 with a Quad-Core on a Dual
    or Quad-CPU board with about 32GByte RAM and some
    Raptor-RAID0+RAID5-Arrays.

    The "UnifiedSessionsManager" is, even though it is designed for
    and used excessively within daily business, an advanced
    proof-of-concept for handling nested virtual machines.

    Therefore some standards are soon going to be filed for
    application. 


   1.2. Claimed Inventions
   =======================

    Related to IP/SW-Patents I would like to make clear, that this
    software is owned solely by myself and donated to the public
    worldwide based on GPL3, so no later claims for ownership of any
    first-time-invention may be applicable, nor will be accepted.

    Therefore this software is distributed widely and once to a wide
    range of recipients, in order of proof of uniqueness at the time 
    of distribution, and the loss of the attrribute of "new invention"
    for claimed items from on this time.

    I apologise the audience receiving this software in first
    "bulk-wave" and aks for their "passive-support" as witnesses in
    order assure the to claim of the implemented principles for the
    community.

    The terms of GPL3 particularly comprise all concepts and design
    principles for the whole project "Unified Sessions Manager" as
    listed here non comprehensively. 


      ---------------------------------------------------------------

      - VM-Stacks
 
        The basic building block for stacked VM handling, including
        some of the advanced multi-ISO-Layer-address-handling and the
        derived technologies within this software.

        The whole theory and technology, as well as the concepts of
        the designed and implemented, and as upcoming described
        feature previews.


      - Management of nested stacks of Virtual-Machines

        The concepts, design, and implementation of the escalation of
        dependent and remapped state change actions for virtual and/or
        physical machines, when additional single or nested virtual
        instances are operational. The state change propagation by
        remapping to appropriate states when propagating into upper
        layer.


      - Appliances as ordinary SW-Components

        The concepts, design, and implementation of the usage of a
        huge amount of virtual machines on bulk-core-CPUs with
        hundreds or even thousands of cores, where due to expected
        future processing power enhancement this heavily seem to
        become to be used in a very ordinary manner. 

        Thus the potential is even the replacement or better extension
        of ordinary system processes by virtual appliances. Offering a
        much more flexible design and operational base for network
        relocation, component based availability enhancements, and
        encapsulation.



      - Integrated Management Interface for self-reconfiguring Network
        Management Systems

        The concepts, design, and implementation of the usage of an
        attached functionality to ordinary graphical icons, but
        positioned on the screen in order to represent the physical or
        logical structure of a networked environment. Therefore an
        machine interface is defined, as a case study based on Nagios,
        where a via CLI started managed entity, which could be a
        virtual machine and/or a physical machine, registers itself by
        usage of a specific differentiated icon as a dynamically
        attached entitiy.

        This will provide multiple views, particularly a view
        representing the nested containement-like execution stack in
        various views itself. Therefore a tree-view, a nested
        boxed-view, and a staple of bricks-view is defined.

        Independently form this additional VM-Stack-View, any
        networked entity is represented by it's secondary logical
        nature as an ordinary networked device, which transparently
        covers the primary characteristic as a virtual entitiy.


        
      - Load distribution of stacked virtual machines 

        The concepts, design, and implementation of the usage of an
        system in order of evaluation COST values for load
        distribution within nested entities.

        Even though the overall CPU could be measured by monitoring the
        lowest container within the stack, this is not necessarily
        true for management of resources accessed by software
        applications from within virtual entities on higher levels of
        the stack only. This is frequently true, when a common network
        platform e.g. by Linux OS is defined with additional virtual
        instances as "worker-entities" only. When e.g. scanners,
        printers, or other devices are supported as a fabrication
        group with load distribution, the overall load of the stack
        base has no relevance to the FIFO character of the required
        resources. Thus any stacked element could have it's own
        constraints of several types, which is not visible outside the
        entitiy itself. This particularly could appear when
        e.g. embedded systems are simulated by usage QEMU for CPU
        simulation and execution of eCos or uCLinux, whereas the QEMU
        instance itself is executed within a so called DomU of Xen. 



      - Performance Enhancements for address resolution

        The applied technologies for replacement of highly
        sophisticated address resolution by so called
        Attribute-Value-Assertion as commonly used within
        CMISE/CMIP/Q3 and SNMP environments for structured access to
        data, are claimed to be independently invented by myself and
        donated to the public by GPL3 too.

        This technology comprises the pre-assembled generation of
        specific table entries with seperated fields, but for cascaded
        application of reqular expression based simple matching
        filters on the whole flat record leading to same data-results
        as huge ASN.1 based approaches. 

        The advance uprises from the combined handling of the overall
        record for flat-matching by regexpr and the read-out of data
        based on structured records by fields. Particularly the
        opportunity of chained filter application on the intermediate
        sets of results leads to quite good matching results with more
        than satisfying overall performance.

        The practical application advantage is that the almost
        "trivial" framework of common UNIX base-tools are required
        only and the average access time still remains within milliseconds
        even when using bulk PC components and implementing it as part
        of this software, by a simple awk-script.


      - Component-Oriented bash usage

        The consequent utilization of components as dynamically loaded
        components by "sourcing" seems to me to be a new first-time
        approach. 



      Arno-Can Uestuensoez
      Munich, Germany
      December 2007


      ---------------------------------------------------------------


  1.3. Basic Terminology 
  ======================

    The terminology, therefore the associations of terms are targeted
    to the dynamic characteristics of working sessions. This should be
    recognized when coming to the Interface Design of methods such as
    CREATE and it's suboptions.

    A static view to sessions might rise expectation, that the
    session entity itself, which is in case of VM - Virtual Machine -
    the VM image and some parts of it's configuration files, will be
    created. This is not the case, even though this might change in
    future, and for now some tools as "ctys-genmconf" support the
    generation of specific management data.

    The current scope of UnifiedSessionsManager is the management of
    the activation of stored sessions and the establishment of user
    connections to the running entity. So the scope of
    UnifiedSessionsManager, with it's main component ctys - "Commutate
    To Your Session" - is almost exclusively focused on dynamic
    runtime instances. Some additional static aspects raise from
    on-the-fly relocation of services, which mainly handle complete
    entities with some minor re-configuration as required. This is due
    to restart commonly mounted entites on different nodes, and/or to
    move session entities between nodes with more ore less support by
    the specific session.

    Thus the action CREATE creates a users session, which could be an
    interactive dialogue or a batch-mode session. It does not create
    the sessions entity (image and/or configuration), which it
    requires as a pre-requisite. 

    Additional actions like ENUMERATE and LIST handle the available
    off-line sessions and the active running sessions.

    The actions SHOW and INFO inform about dynamic and static aspects
    of running sessions.

    The method CANCEL cancels active sessions, but does not remove the
    stored sessions entities.

    The creation and installation of sessions has to be managed by the
    specific session types own tools.

    One another point to be mentioned is the assembly of options and
    suboptions to specific commands. This is smoothly related to the
    view of a session as an dynamic entity. Therefore the CREATE
    method could be seen in some minor variation as creation with an
    initial start of the requested session-entity, or a REUSE of a
    running session if already started. It could be seen as a simple
    CONNECT to a pre-required running session, where a "share"
    attribute just adds an additional client. It could be seen also as
    a RECONNECT, where in accordance to the unified interface, a fresh
    new connection is established, after cancelling ALL previous
    client sessions. It could be seen as a session which previously
    was frozen by CANCEL=SUSPEND, and now will be activated again with
    RESUME. 

    So the design of the combined CREATE action with it's further
    specialization by one of REUSE,CONNECT,RECONNECT, or RESUME
    results in a lean first-level interface definition. Grouping
    together a reasonable set of second-level suboptions. Additionally
    this has a positive influence on the Design and Code of the
    modules.

    Same principle is applied to the remaining actions.
    
    Another defition is the usage of the term "ACTION" for any method
    that is executed on a specific type or a group of types. This is
    similiar to a method performed on an instance, but more close to
    handle a generic action on a node as decribed within the ITU-T
    TMN related standards or within the SNMP-world. The standard
    methods GET and SET are found not to be really appropriate, due to
    the fact, that almost no attribute of a stored or an activated
    entity is set. So the generic type ACTION with it's own
    sub-definitions - a.k.a sub-options is choosen.

