Inventory Aggregate Manager

The Inventory Aggregate Manager stores node configuration information in database with a custom ​database schema. It can be interfaced via a rest API and is depended up by several services to answer questions like:

• which disk should we boot off?
• what is the name of the control interface?
• which kernel and initramfs should the node boot during pxe phases?
• what kind of network cards does this node have?

In the Inventory AM, INF_ prefixed configuration data is static and configured during the installation process (e.g. the control interface of a node). In contrast some INV_ prefixed configuration information is gathered via a live inventory collection process where the node is booted into a PRE-OS (pxelinux). While in the PRE-OS scripts on the node collection information about the nodes current configuration (e.g installed disk size, types of wireless cards).

The current attribute list is:

• CM_ip - Ip address of the Chassis Manager
• CM_port - Support for CMs that control multiple nodes/devices, for example MIMO Rack or VMs
• CM_type - Version number of the cmc associated with this node
• INF_control_ip - IP address of the control Interface
• INF_control_mac - MAC address of the control Interface, set manually on node installation
• INF_data_switch_port_id - Switch port number for the data interface
• INF_default_disk - Disk path that node boots off
• INF_max_frisbee_rate - maximum rate allowed for pushing images onto the disk
• INF_pxe_image - Name of the boot file that tells pxelinux.bin which kernel and initramfs to download
• INF_reset_timeout - time before we assumed the node failed to boot and then attempt a recovery by rebooting
• INF_x_coor - X coordinate of the node (domain specific)
• INF_y_coor - Y coordinate of the node (domain specific)
• INV_control_mac - MAC address of the control Interface, populated by the inventory process
• INV_check_in - Last time the node successfully completed an inventory
• INV_cpu_bench - rudimentary CPU benchmark (prime factorization) for comparing node capabilities
• INV_cpu_hz - CPU clock speed for identification
• INV_cpu_type - Vendor identification string that name the CPU architecture
• INV_fw_ping - Connectivity test of the control interface (pings the outbound gateway)
• INV_hd_model - Vendor string that identifies the hard disk
• INV_hd_size - Disk size reported by OS
• INV_hd_sn - Disk Serial number as reported by OS (for identification purposes)
• INV_mb_sn - Mother board Serial number as reported by OS (for identification purposes)
• INV_memory - Node Memory Capacity as reported by OS
• name - Fully qualified domain name of the node (this is the principal identifier)

Resources and Relationships

Resources are physical entities in a testbed that are assumed to be named with fully qualified domain name (fqdn) (for resources that can have one) or with arbitrary string (for resources that are not IP addressable)and can have any number of attributes. Resource can be in relation with any other resource with, so far, 2 types of relationships:

Type	Relationship	Description
1	belong_to	The subject resource belongs to the object resource
2	installed_in	The subject resource is installed in the object resource

Typically, type 1 is used to designate that the node belongs to a testbed, and type 2 for devices that are installed into a node (at the moment type of relationship is not used for any particular function by the inventory).

To facilitate better classification, inventory services are using string based resource type. So far, defined resources types are: testbed, node and device (these types are only used by other services).

Attributes

Attributes are name-value pairs of strings. Attribute names are prefixed to support globbing.

Attribute prefixes:

• CM_ - Attributes used by the CMC AM ( CM_TYPE, CM_IP, CM_PORT, CM_USER_NAME, CM_PASSWORD )
• INF_ - Core infrastructure attributes used by the PXE and Frisbee service ( INF_CONTROL_IP )
• INV_ - Attributes gathered by the inventory process

Inventory Service Group

<services>
  <serviceGroup name="inventory" prefix="/inventory">
    <info>Service to retrieve information about nodes or testbeds from the Inventory Database</info>
    <service name="attribute_add"></service>
    <service name="attribute_delete"></service>
    <service name="attribute_list"></service>
    <service name="attribute_listChildren"></service>
    <service name="attribute_list_full"></service>
    <service name="attribute_modify"></service>
    <service name="attribute_swap"></service>
    <service name="domain_nodes"></service>
    <service name="parent_list"></service>
    <service name="relation_add"></service>
    <service name="resource_add"></service>
    <service name="resource_delete"></service>
    <service name="resource_list"></service>
    <service name="resource_rename"></service>
    <service name="resource_swap"></service>
    <service name="topology_add"></service>
    <service name="topology_delete"></service>
    <service name="topology_list"></service>
    <service name="topology_nodes"></service>
    <service name="topology_show"></service>
    <service name="update_cache"></service>
  </serviceGroup>
</services>

Attribute Services

attribute_add - Add an attribute

<service name="attribute_add">
  <info>Add attribute to resource</info>
  <args>
    <arg isRequired="true" name="name" value="name">
      <info>Resource name</info>
    </arg>
    <arg isRequired="true" name="attribute" value="attribute">
      <info>Name of attribute</info>
    </arg>
    <arg isRequired="true" name="value" value="value">
      <info>Value of attribute </info>
    </arg>
  </args>
</service>

attribute_delete - Delete an attribute

<service name="attribute_delete">
  <info>Delete all attributes matching attribute pattern for all nodes matching node name pattern (rn)</info>
  <args>
    <arg isRequired="true" name="name" value="name"
      <info>Resource name or resource name pattern</info>
    </arg>
    <arg isRequired="true" name="attribute" value="attribute">
      <info>Attribute pattern</info>
    </arg>
  </args>
</service>

attribute_list - List attributes

<service name="attribute_list">
  <info>Get resource with names matching name pattern with, attributes matching attribute name pattern and value matching attribute value pattern</info>
  <args>
    <arg isRequired="true" name="set" value="set">
      <info>Set of resource names or resource name pattern</info>
    </arg>
    <arg isRequired="false" name="attribute" value="[attribute]">
      <info>Attribute name pattern</info>
    </arg>
    <arg isRequired="false" name="value" value="[value]">
      <info>Attribute value pattern</info>
    </arg>
  </args>
</service>

attribute_listChildren - List attributes of children

<service name="attribute_listChildren">
  <info>Get node with names matching name pattern with child (device) attributes matching attribute name pattern and value matching attribute value pattern</info>
  <args>
    <arg isRequired="true" name="set" value="set">
      <info>Set of resource names or resource name pattern</info>
    </arg>
    <arg isRequired="false" name="attribute" value="[attribute]">
      <info>Child attribute name pattern</info></arg>
    <arg isRequired="false" name="value" value="[value]">
      <info>Child attribute value pattern</info>
    </arg>
  </args>
</service>

attribute_list_full - List node names with attributes

<service name="attribute_list_full">
  <info>Get node with names matching name pattern with attributes matching attribute name pattern and value matching attribute value pattern</info>
  <args>
    <arg isRequired="true" name="set" value="set">
      <info>Set of resource names or resource name pattern</info>
    </arg>
    <arg isRequired="false" name="attribute" value="[attribute]">
      <info>Attribute name pattern</info>
    </arg>
    <arg isRequired="false" name="value" value="[value]">
      <info>Attribute value pattern</info>
    </arg>
  </args>
</service>

attribute_modify - Modify an attribute value

<service name="attribute_modify">
  <info>Modify attribute's value</info>
  <args>
    <arg isRequired="true" name="name" value="name">
      <info>Resource name</info>
    </arg>
    <arg isRequired="true" name="attribute" value="attribute">
      <info>Attribute name</info>
    </arg>
    <arg isRequired="true" name="value" value="value">
      <info>New value of attribute </info>
    </arg>
  </args>
</service>

attribute_swap - Swap attribute values between two resources

<service name="attribute_swap">
  <info>Swap attribute of two resources</info>
  <args>
    <arg isRequired="true" name="attribute" value="atribute">
      <info>Attribute to be swapped</info>
    </arg>
    <arg isRequired="true" name="resource1" value="resource1">
      <info>First Resource name</info>
    </arg><arg isRequired="true" name="resource2" value="resource2">
      <info>Second Resource name</info>
    </arg>
  </args>
</service>

Resource Services

Sub-group for resource management. Resources are: topology,node,domain,infrastructure

domain_nodes - Show all nodes in a domain

<service name="domain_nodes">
  <info>Get domain nodes</info>
  <args>
    <arg isRequired="true" name="domain" value="domain">
      <info>domain/testbed name</info>
    </arg>
  </args>
</service>

parent_list - Show all parent resources

<service name="parent_list">
   <info>Get all resources that given resourse belongs to</info>
   <args>
     <arg isRequired="false" name="name" value="[name]">
       <info>Fully qualified resource name</info>
     </arg>
   </args>
</service>

relation_add - Add Relationship Between Resources

<service name="relation_add">
  <info>Add relatin between two resources</info>
  <args>
    <arg isRequired="true" name="parent" value="rn">
      <info>parent resource name</info>
    </arg>
    <arg isRequired="true" name="child" value="rn">
      <info>child resource name</info>
    </arg>
  </args>
</service>

resource_add - Add Resource

<service name="resource_add">
  <info>Add resource</info>
  <args>
    <arg isRequired="true" name="name" value="name">
      <info>name of the resource or fully qualified resource name</info>
    </arg> 
    <arg isRequired="true" name="type" value="type">
      <info>type of the resource</info>
    </arg>
    <arg isRequired="false" name="xml" value="[xml]">
      <info>xml-encoded hash of resource parameters</info>
    </arg>
  </args>
</service>

resource_delete - Delete Resource

<service name="resource_delete">
  <info>Remove set of resources</info>
  <args>
    <arg isRequired="true" name="set" value="set">
      <info>set of resource names</info>
    </arg>
  </args>
</service>

resource_list - List Resources

<service name="resource_list">
  <info>Get all available resources of given type that belongs to given parent resource with all attributes</info>
  <args>
    <arg isRequired="false" name="parent" value="[parent]">
      <info>Fully qualified resource name</info>
    </arg>
    <arg isRequired="false" name="type" value="[type]">
      <info>Type name, currently suported testbed, node and device</info>
    </arg>
  </args>
</service>

resource_rename - Change resource name

<service name="resource_rename">
  <info>Rename resource</info>
  <args>
    <arg isRequired="true" name="oldname" value="oldname">
      <info>Old resource name</info>
    </arg>
    <arg isRequired="true" name="newname" value="newname">
      <info>New resource name</info>
    </arg>
  </args>
</service>

resource_swap - Swap two resources (names)

<service name="resource_swap">
<info>Swap two resources</info><args>
<arg isRequired="true" name="resource1" value="resource1">
<info>First Resource name</info></arg><arg isRequired="true" name="resource2" value="resource2"><info>Second Resource name</info>
</arg>
</args>
</service>

Topology Services

Topologies are collections of resources that are grouped based on certain characteristics or attributes. Examples are: topology with the name: TOPO_CPU_grid.orbit-lab.org_2.10GHz that contains list of nodes in the grid with a CPU with2.1 GHz clock.

topology_add - Service to create new topology

<service name="topology_add">
  <info>Add topology</info>
  <args>
    <arg isRequired="true" name="name" value="name">
      <info>Toplogy name</info>
    </arg>
    <arg isRequired="true" name="set" value="set">
      <info>comma separeted node names</info>
    </arg>
  </args>
</service>

topology_add - Service to delete existing topology

<service name="topology_delete">
<info>Remove set of topologies</info>
<args>
<arg isRequired="true" name="set" value="set">
<info>Set of toplogy names</info>
</arg>
</args>
</service>

<service name="topology_list">
<info>Get all defined topologies</info>
</service>

<service name="topology_nodes">
<info>Get topology nodes</info>
<args>
<arg isRequired="true" name="shortname" value="shortname">
<info>topology shortname</info>
</arg>
<arg isRequired="true" name="domain" value="domain">
<info>domain/testbed name</info>
</arg>
</args>
</service>

<service name="topology_show">
<info>Show list of nodes that belong to given topology name</info
<args>
<arg isRequired="true" name="name" value="name">
<info>topology name</info
</arg>
</args>
</service>

Maintenance Services

update_cache - Service to update internal cache

This service should be called after any changes to the inventory.

<service name="update_cache">
<info>Update cache</info>
</service>

Usage Examples

Create testbed, add node to it and add basic attributes

# Define the testbed
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/resource_add?name=sb10.orbit-lab.org&type=testbed'
# Add a node
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/resource_add?name=node1-1.sb10.orbit-lab.org&type=node'
# Create the parent/child relationship between the testbed and the node
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/relation_add?parent=sb10.orbit-lab.org&child=node1-1.sb10.orbit-lab.org'
# Add basic attributes
# CM is CM3 (mandatory)
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org&attribute=CM_type&value=3'
# IP Address for this CM (mandatory)
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org&attribute=CM_ip&value=10.22.1.1'
# IP Port for this CM (mandatory)
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org&attribute=CM_port&value=1'
# Disk from the node boots (for imaging)
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org&attribute=INF_default_disk&value=/dev/sda'
  ...

Replace inventory information

# First, delete all devices that belong to the node
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/resource_delete?set=node1-1.sb10.orbit-lab.org-*'
# Then delete all node level attributes collected by the inventory process
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_delete?name=node1-1.sb10.orbit-lab.org&attribute=INV_*'
# Add node level attribute
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org&attribute=INV_MB_ID&value=i7'
# Add first device named "node1-1.sb10.orbit-lab.org-e0" (and add it to the node)
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/resource_add?name=node1-1.sb10.orbit-lab.org-e0&type=device'
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/relation_add?parent=node1-1.sb10.orbit-lab.org&child=node1-1.sb10.orbit-lab.org-e0'
# Add attributes for the new device
  wget -qO- 'http://new.orbit-lab.org:5054/inventory/attribute_add?name=node1-1.sb10.orbit-lab.org-e0&attribute=INV_MAC_0&value=00:01:02:03:04:05'

Installation and Configuration

The Frisbee AM (tools and services) is packaged in the omf-aggmgr-frisbee-5.4 package. It can be installed form the package repository with

apt-get install omf-aggmgr-inventory-5.4

Instructions on how to add the Orbit software repository to your list of apt-sources can be found here.

Configruation

Te inventory service is configuraion is in /etc/omf-aggmgr-5.4/enabled/inventory.yaml YAML file. Mandatory configuration paramter is the database: which specifies inventory service database (actual database access is configured in global OMF AM configuration file). Otopnal :allowedIPs: parameter is used to restrict access to individual add/delete/modify services (in this case based on client IP addressa) with a default value set to 'localhost'.

Example configuration file:

# NOTE: use only 'spaces' to indent !
# ('tab' indents are not supported by the ruby yaml parser used to read this file)
#
# This is the Config file for the Inventory GridService on the NICTA platform
#
---
inventory:
  :database: 
    database: inventory54
  :allowedIPs:
        - 'localhost'