xm − Xen management user interface
xm <subcommand> [args]
The xm program is the main interface for managing Xen guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains, enable or pin VCPUs, and attach or detach virtual block devices.
The basic structure of every xm command is almost always:
xm <subcommand> <domain−id> [OPTIONS]
Where subcommand is one of the sub commands listed below, domain-id is the numeric domain id, or the domain name (which will be internally translated to domain id), and OPTIONS are sub command specific options. There are a few exceptions to this rule in the cases where the sub command in question acts on all domains, the entire machine, or directly on the xen hypervisor. Those exceptions will be clear for each of those sub commands.
All xm operations rely upon the Xen control daemon, aka xend. For any xm commands to run xend must also be running. For this reason you should start xend as a service when your system first boots using xen.
Most xm commands require root privileges to run due to the communications channels used to talk to the hypervisor. Running as non root will return an error.
Most xm commands act asynchronously, so just because the xm command returned, doesn’t mean the action is complete. This is important, as many operations on domains, like create and shutdown, can take considerable time (30 seconds or more) to bring the machine into a fully compliant state. If you want to know when one of these actions has finished you must poll through xm list periodically.
sub commands manipulate domains directly, as stated
previously most commands take domain-id as the first
Attach to domain domain−id’s console. If you’ve set up your Domains to have a traditional log in console this will look much like a normal text log in screen.
This uses the back end xenconsole service which currently only works for para-virtual domains.
The attached console will perform much like a standard serial console, so running curses based interfaces over the console is not advised. Vi tends to get very odd when using it over this interface.
create [−c] configfile [name=value]..
The create sub command requires a configfile and can optional take a series of name value pairs that add to or override variables defined in the config file. See xmdomain.cfg for full details of that file format, and possible options used in either the configfile or Name=Value combinations.
Configfile can either be an absolute path to a file, or a relative path to a file located in /etc/xen.
Create will return as soon as the domain is started. This does not mean the guest OS in the domain has actually booted, or is available for input.
Attache console to the domain as soon as it has started. This is useful for determining issues with crashing domains.
with config file
xm create Fedora4
This creates a domain with the file /etc/xen/Fedora4, and returns as soon as it is run.
without config file
xm create /dev/null ramdisk=initrd.img \ kernel=/boot/vmlinuz−18.104.22.168−xenU \ name=ramdisk nics=0 vcpus=1 \ memory=64 root=/dev/ram0
This creates the domain without using a config file (more specifically using /dev/null as an empty config file), kernel and ramdisk as specified, setting the name of the domain to "ramdisk", also disabling virtual networking. (This example comes from the xm-test test suite.)
Immediately terminate the domain domain−id. This doesn’t give the domain OS any chance to react, and it the equivalent of ripping the power cord out on a physical machine. In most cases you will want to use the shutdown command instead.
Converts a domain name to a domain id using xend’s internal mapping.
Converts a domain id to a domain name using xend’s internal mapping.
Displays the short help message (i.e. common commands).
The −−long option prints out the complete set of xm subcommands, grouped by function.
list [−−long | −−label] [domain−id, ...]
Prints information about one or more domains. If no domains are specified it prints out information about all domains.
An example format for the list is as follows:
Name ID Mem(MiB) VCPUs State Time(s) Domain−0 0 98 1 r−−−−− 5068.6 Fedora3 164 128 1 r−−−−− 7.6 Fedora4 165 128 1 −−−−−− 0.6 Mandrake2006 166 128 1 −b−−−− 3.6 Mandrake10.2 167 128 1 −−−−−− 2.5 Suse9.2 168 100 1 −−−−−− 1.8
Name is the name of the domain. ID the domain numeric id. Mem is the size of the memory allocated to the domain. VCPUS is the number of VCPUS allocated to domain. State is the run state (see below). Time is the total run time of the domain as accounted for by Xen.
The State field lists 6 states for a Xen Domain, and which ones the current Domain is in.
r − running
The domain is currently running on a CPU
b − blocked
The domain is blocked, and not running or runnable. This can be caused because the domain is waiting on IO (a traditional wait state) or has gone to sleep because there was nothing else for it to do.
p − paused
The domain has been paused, usually occurring through the administrator running xm pause. When in a paused state the domain will still consume allocated resources like memory, but will not be eligible for scheduling by the Xen hypervisor.
s − shutdown
FIXME: Why would you ever see this state?
c − crashed
The domain has crashed, which is always a violent ending. Usually this state can only occur if the domain has been configured not to restart on crash. See xmdomain.cfg for more info.
d − dying
The domain is in process of dying, but hasn’t completely shutdown or crashed.
FIXME: Is this right?
If −−long is specified, the output for xm list is not the table view shown above, but instead is an S−Expression representing all information known about all domains asked for. This is mostly only useful for external programs to parse the data.
Note: there is no stable guarantees on the format of this data. Use at your own risk.
If −−label is specified, the security labels are added to the output of xm list and the lines are sorted by the labels (ignoring case). The −−long option prints the labels by default and cannot be combined with −−label. See the ACCESS CONTROL SUBCOMMAND section of this man page for more information about labels.
The Time column is deceptive. Virtual IO (network and block devices) used by Domains requires coordination by Domain0, which means that Domain0 is actually charged for much of the time that a DomainU is doing IO . Use of this time value to determine relative utilizations by domains is thus very suspect, as a high IO workload may show as less utilized than a high CPU workload. Consider yourself warned.
mem-max domain-id mem
Specify the maximum amount of memory the Domain is able to use. Mem is specified in megabytes.
The mem-max value may not correspond to the actual memory used in the Domain, as it may balloon down it’s memory to give more back to the OS .
mem-set domain-id mem
Set the domain’s used memory using the balloon driver. Because this operation requires cooperation from the domain operating system, there is no guarantee that it will succeed.
Warning: there is no good way to know in advance how small of a mem-set will make a domain unstable and cause it to crash. Be very careful when using this command on running domains.
migrate domain-id host [options]
Migrate a domain to another Host machine. Xend must be running on other host machine, it must be running the same version of xen, it must have the migration TCP port open and accepting connections from the source host, and there must be sufficient resources for the domain to run (memory, disk, etc).
Migration is pretty complicated, and has many security implications, please read the Xen Users Guide to ensure you understand the ramifications and limitations on migration before attempting it in production.
Use live migration. This will migrate the domain between hosts without shutting down the domain. See the Xen Users Guide for more information.
−r, −−resource Mbs
Set maximum Mbs allowed for migrating the domain. This ensures that the network link is not saturated with migration traffic while attempting to do other useful work.
Pause a domain. When in a paused state the domain will still consume allocated resources such as memory, but will not be eligible for scheduling by the Xen hypervisor.
reboot [options] domain-id
Reboot a domain. This acts just as if the domain had the reboot command run from the console. The command returns as soon as it has executed the reboot action, which may be significantly before the domain actually reboots.
The behavior of what happens to a domain when it reboots is set by the on_reboot parameter of the xmdomain.cfg file when the domain was created.
Reboot all domains
Wait for reboot to complete before returning. This may take a while, as all services in the domain will have to be shut down cleanly.
Build a domain from an xm save state file. See save for more info.
save domain-id state-file
Saves a running domain to a state file so that it can be restored later. Once saved, the domain will no longer be running on the system, thus the memory allocated for the domain will be free for other domains to use. xm restore restores from this state file.
This is roughly equivalent to doing a hibernate on a running computer, with all the same limitations. Open network connections may be severed upon restore, as TCP timeouts may have expired.
shutdown [options] domain-id
Gracefully shuts down a domain. This coordinates with the domain OS to perform graceful shutdown, so there is no guarantee that it will succeed, and may take a variable length of time depending on what services must be shutdown in the domain. The command returns immediately after signally the domain unless that −w flag is used.
The behavior of what happens to a domain when it reboots is set by the on_shutdown parameter of the xmdomain.cfg file when the domain was created.
Shutdown all domains. Often used when doing a complete shutdown of a Xen system.
Wait for the domain to complete shutdown before returning.
sysrq domain-id letter
Send a Magic System Request signal to the domain. For more information on available magic sys req operations, see sysrq.txt in your Linux Kernel sources.
Moves a domain out of the paused state. This will allow a previously paused domain to now be eligible for scheduling by the Xen hypervisor.
vcpu-set domain-id vcpu-count
Enables the vcpu-count virtual CPUs for the domain in question. Like mem−set, this command can only allocate up to the maximum virtual CPU count configured at boot for the domain.
If the vcpu-count is smaller than the current number of active VCPUs, the highest number VCPUs will be hotplug removed. This may be important for pinning purposes.
Attempting to set the VCPUs to a number larger than the initially configured VCPU count is an error. Trying to set VCPUs to < 1 will be quietly ignored.
Lists VCPU information for a specific domain. If no domain is specified, VCPU information for all domains will be provided.
vcpu-pin domain-id vcpu cpus
Pins the the VCPU to only run on the specific CPUs. The keyword all can be used to apply the cpus list to all VCPUs in the domain.
Normally VCPUs can float between available CPUs whenever Xen deems a different run state is appropriate. Pinning can be used to restrict this, by ensuring certain VCPUs can only run on certain physical CPUs.
Reads the Xen message buffer, similar to dmesg on a Linux system. The buffer contains informational, warning, and error messages created during Xen’s boot process. If you are having problems with Xen, this is one of the first places to look as part of problem determination.
Clears Xen’s message buffer.
Print information about the Xen host in name : value format. When reporting a Xen bug, please provide this information as part of the bug report.
Sample xen domain info looks as follows (lines wrapped manually to make the man page more readable):
host : talon release : 22.214.171.124−xen0 version : #1 Mon Nov 14 14:26:26 EST 2005 machine : i686 nr_cpus : 2 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 1 threads_per_core : 1 cpu_mhz : 696 hw_caps : 0383fbff:00000000:00000000:00000040 total_memory : 767 free_memory : 37 xen_major : 3 xen_minor : 0 xen_extra : −devel xen_caps : xen−3.0−x86_32 xen_pagesize : 4096 platform_params : virt_start=0xfc000000 xen_changeset : Mon Nov 14 18:13:38 2005 +0100 7793:090e44133d40 cc_compiler : gcc version 3.4.3 (Mandrakelinux 10.2 3.4.3−7mdk) cc_compile_by : sdague cc_compile_domain : (none) cc_compile_date : Mon Nov 14 14:16:48 EST 2005 xend_config_format : 2
Not all fields will be explained here, but some of the less obvious ones deserve explanation:
A vector showing what hardware capabilities are supported by your processor. This is equivalent to, though more cryptic, the flags field in /proc/cpuinfo on a normal Linux machine.
Available memory (in MB ) not allocated to Xen, or any other Domains.
The xen version, architecture. Architecture values can be one of: x86_32, x86_32p (i.e. PAE enabled), x86_64, ia64.
The xen mercurial changeset id. Very useful for determining exactly what version of code your Xen system was built from.
Print out the xend log. This log file can be found in /var/log/xend.log.
Executes the xentop command, which provides real time monitoring of domains. Xentop is a curses interface, and reasonably self explanatory.
Xen ships with a number of domain schedulers, which can be set at boot time with the sched= parameter on the Xen command line. By default sedf is used for scheduling.
we really need a scheduler expert to write up this section.
sched-sedf period slice latency-hint extratime weight
Set Simple EDF (Earliest Deadline First) scheduler parameters. This scheduler provides weighted CPU sharing in an intuitive way and uses realtime-algorithms to ensure time guarantees. For more information see docs/misc/sedf_scheduler_mini−HOWTO.txt in the Xen distribution.
The normal EDF scheduling usage in nanoseconds
The normal EDF scheduling usage in nanoseconds
FIXME: these are lame, should explain more.
Scaled period if domain is doing heavy I/O.
Flag for allowing domain to run in extra time.
Another way of setting cpu slice.
normal EDF (20ms/5ms):
xm sched−sedf <dom−id> 20000000 5000000 0 0 0
best-effort domains (i.e. non−realtime):
xm sched−sedf <dom−id> 20000000 0 0 1 0 I<normal EDF (20ms/5ms) + share of extra−time:> xm sched−sedf <dom−id> 20000000 5000000 0 1 0
4 domains with weights 2:3:4:2
xm sched−sedf <d1> 0 0 0 0 2 xm sched−sedf <d2> 0 0 0 0 3 xm sched−sedf <d3> 0 0 0 0 4 xm sched−sedf <d4> 0 0 0 0 2
1 fully-specified (10ms/3ms) domain, 3 other domains share available rest in 2:7:3 ratio:
xm sched−sedf <d1> 10000000 3000000 0 0 0 xm sched−sedf <d2> 0 0 0 0 2 xm sched−sedf <d3> 0 0 0 0 7 xm sched−sedf <d4> 0 0 0 0 3
Most virtual devices can be added and removed while guests are running. The effect to the guest OS is much the same as any hotplug event.
block-attach domain-id be-dev fe-dev mode [bedomain−id]
Create a new virtual block device. This will trigger a hotplug event for the guest.
The domain id of the guest domain that the device will be attached to.
The device in the backend domain (usually domain 0) to be exported. This can be specified as a physical partition (phy:sda7) or as a file mounted as loopback (file://path/to/loop.iso).
How the device should be presented to the guest domain. It can be specified as either a symbolic name, such as /dev/hdc, for common devices, or by device id, such as 0x1400 (/dev/hdc device id in hex).
The access mode for the device from the guest domain. Supported modes are rw (read/write) or ro (read−only).
The back end domain hosting the device. This defaults to domain 0.
Mount an ISO as a Disk
xm block-attach guestdomain file://path/to/dsl−2.0RC2.iso /dev/hdc ro
This will mount the dsl iso as /dev/hdc in the guestdomain as a read only device. This will probably not be detected as a cdrom by the guest, but mounting /dev/hdc manually will work.
block-detach domain-id devid
Destroy a domain’s virtual block device. devid must be the device id given to the device by domain 0. You will need to run xm block-list to determine that number.
FIXME: this is currently broken. Even though a block device is removed from domU, it appears to still be allocated in the domain 0.
block-list [−l|−−long] domain-id
List virtual block devices for a domain. The returned output is formatted as a list or as an S−Expression if the ’−−long’ option was given.
network-attach domain-id [script=scriptname] [ip=ipaddr] [mac=macaddr]
Creates a new network device in the domain specified by domain−id. It takes the following optional options:
Use the specified script name to bring up the network. Defaults to the default setting in xend−config.sxp for vif-script.
Passes the specified IP Address to the adapter on creation.
FIXME: this currently appears to be broken. I’m not sure under what circumstances this should actually work.
The MAC address that the domain will see on its Ethernet device. If the device is not specified it will be randomly generated with the 00:16:3e vendor id prefix.
The name of the bridge to attach the vif to, in case you have more than one. This defaults to
The backend domain id. By default this is domain 0.
network-detach domain-id devid
Removes the network device from the domain specified by domain-id. devid is the virtual interface device number within the domain (i.e. the 3 in vif22.3).
FIXME: this is currently broken. Network devices aren’t completely removed from domain 0.
network-list [−l|−−long] domain-id
List virtual network interfaces for a domain. The returned output is formatted as a list or as an S−Expression if the ’−−long’ option was given.
vtpm-list [−l|−−long] domain-id
Show the virtual TPM device for a domain. The returned output is formatted as a list or as an S−Expression if the ’−−long’ option was given.
The Virtual Network interfaces for Xen.
This needs a lot more explanation, or it needs to be ripped
Create a vnet from a config file.
Delete a vnet.
Access Control in Xen consists of two components: (i) The Access Control Policy ( ACP ) defines security labels and access rules based on these labels. (ii) The Access Control Module ( ACM ) makes access control decisions by interpreting the policy when domains require to communicate or to access resources. The Xen access control has sufficient mechanisms in place to enforce the access decisions even against maliciously acting user domains (mandatory access control).
Access rights for domains in Xen are determined by the domain security label only and not based on the domain Name or ID . The ACP specifies security labels that can then be assigned to domains and resources. Every domain must be assigned exactly one security label, otherwise access control decisions could become indeterministic. ACPs are distinguished by their name, which is a parameter to most of the subcommands described below. Currently, the ACP specifies two ways to interpret labels:
(1) Simple Type Enforcement: Labels are interpreted to decide access of domains to comunication means and virtual or physical resources. Communication between domains as well as access to resources are forbidden by default and can only take place if they are explicitly allowed by the security policy. The proper assignment of labels to domains controls the sharing of information (directly through communication or indirectly through shared resources) between domains. This interpretation allows to control the overt (intended) communication channels in Xen.
(2) Chinese Wall: Labels are interpreted to decide which domains can co-exist (be run simultaneously) on the same system. This interpretation allows to prevent direct covert (unintended) channels and mitigates risks caused by imperfect core domain isolation (trade−off between security and other system requirements). For a short introduction to covert channels, please refer to http://www.multicians.org/timing−chn.html.
subcommands help you to manage security policies in Xen and
to assign security labels to domains. To enable access
control security in Xen, you must compile Xen with
ACM support enabled as described under
"Configuring Security" below. There, you will find
also examples of each subcommand described here.
Compiles the XML source representation of the security policy. It creates a mapping (.map) as well as a binary (.bin) version of the policy. The compiled policy can be loaded into Xen with the loadpolicy subcommand or can be configured to be loaded at boot time with the cfgbootpolicy subcommand.
policy is a dot-separated list of names. The last part is the file name pre-fix for the policy xml file. The preceding name parts are translated into the local path pointing to the policy xml file relative to the global policy root directory (/etc/xen/acm−security/policies). For example, example.chwall_ste.client_v1 denotes the policy file example/chwall_ste/client_v1−security_policy.xml relative to the global policy root directory.
Loads the binary representation of the policy into Xen. The binary representation can be created with the makepolicy subcommand.
cfgbootpolicy policy [kernelversion]
Configures policy as the boot policy for Xen. It copies the binary policy representation into the /boot directory and adds a module line specifying the binary policy to the /boot/grub/menu.lst file. If your boot configuration includes multiple Xen boot titles, then use the kernelversion parameter to select the proper title.
Prints the current security policy state information of Xen.
labels [policy] [type=dom|res|any]
Lists all labels of a type (domain, resource, or both) that are defined in the policy. Unless specified, the default policy is the currently enforced access control policy. The default for type is ’dom’. The labels are arranged in alphabetical order.
dom configfile [policy]
addlabel label res resource [policy]
Adds the security label with name label to a domain configfile (dom) or to the global resource label file for the given resource (res). Unless specified, the default policy is the currently enforced access control policy. This subcommand also verifies that the policy definition supports the specified label name.
rmlabel res resource
Works the same as the addlabel command (above), except that this command will remove the label from the domain configfile (dom) or the global resource label file (res).
getlabel res resource
Shows the label for the given configfile or resource
Lists all resources in the global resource label file. Each resource is listed with its associated label and policy name.
Determines if the specified configfile describes a domain with a valid security configuration for type enforcement. The test shows the policy decision made for each resource label against the domain label as well as the overall decision.
In xen_source_dir/Config.mk set the following parameters:
ACM_SECURITY ?= y ACM_DEFAULT_SECURITY_POLICY ?= \ ACM_CHINESE_WALL_AND_SIMPLE_TYPE_ENFORCEMENT_POLICY
Then recompile and install xen and the security tools and then reboot:
cd xen_source_dir/xen; make clean; make; cp xen.gz /boot; cd xen_source_dir/tools/security; make install; reboot into xen
COMPILING A SECURITY POLICY
This step creates client_v1.map and client_v1.bin files in /etc/xen/acm−security/policies/example/chwall_ste.
xm makepolicy example.chwall_ste.client_v1
LOADING A SECURITY POLICY
This step activates client_v1.bin as new security policy in Xen. You can use the dumppolicy subcommand before and afterwards to see the change in the Xen policy state.
xm loadpolicy example.chwall_ste.client_v1
CONFIGURING A BOOT SECURITY POLICY
This configures the boot loader to load client_v1.bin at boot time. During system start, the ACM configures Xen with this policy and Xen enforces this policy from then on.
xm cfgbootpolicy example.chwall_ste.client_v1
LISTING SECURITY LABELS
This subcommand shows all labels that are defined and which can be attached to domains.
xm labels example.chwall_ste.client_v1 type=dom
will print for our example policy:
dom_BoincClient dom_Fun dom_HomeBanking dom_NetworkDomain dom_StorageDomain dom_SystemManagement
ATTACHING A SECURITY LABEL TO A DOMAIN
The addlabel subcommand can attach a security label to a domain configuration file, here a HomeBanking label. The example policy ensures that this domain does not share information with other non-hombanking user domains (i.e., domains labeled as dom_Fun or dom_Boinc) and that it will not run simultaneously with domains labeled as dom_Fun.
We assume that the specified myconfig.xm configuration file actually instantiates a domain that runs workloads related to home−banking, probably just a browser environment for online−banking.
xm addlabel dom_HomeBanking dom myconfig.xm
The very simple configuration file might now look as printed below. The addlabel subcommand added the access_control entry at the end of the file, consisting of a label name and the policy that specifies this label name:
kernel = "/boot/vmlinuz−2.6.16−xen" ramdisk="/boot/U1_home_banking_ramdisk.img" memory = 164 name = "homebanking" vif = [ ’’ ] dhcp = "dhcp" access_control = [’policy=example.chwall_ste.client_v1, label=dom_HomeBanking’]
Security labels must be assigned to domain configurations because these labels are essential for making access control decisions as early as during the configuration phase of a newly instantiated domain. Consequently, a security-enabled Xen hypervisor will only start domains that have a security label configured and whose security label is consistent with the currently enforced policy. Otherwise, starting the domain will fail with the error condition "operation not permitted".
ATTACHING A SECURITY LABEL TO A RESOURCE
The addlabel subcommand can also be used to attach a security label to a resource. Following the home banking example from above, we can label a disk resource (e.g., a physical partition or a file) to make it accessible to the home banking domain. The example policy provides a resource label, res_LogicalDiskPartition1(hda1), that is compatible with the HomeBanking domain label.
xm addlabel "res_LogicalDiskPartition1(hda1)" res phy:hda6
After labeling this disk resource, it can be attached to the domain by adding a line to the domain configuration file. The line below attaches this disk to the domain at boot time.
disk = [ ’phy:hda6,sda2,w’ ]
Alternatively, the resource can be attached after booting the domain by using the block-attach subcommand.
xm block−attach homebanking phy:hda6 sda2 w
Note that labeled resources cannot be used when security is turned off. Any attempt to use labeled resources with security turned off will result in a failure with a corresponding error message. The solution is to enable security or, if security is no longer desired, to remove the resource label using the rmlabel subcommand.
STARTING AND LISTING LABELED DOMAINS
xm create myconfig.xm xm list −−label Name ID ... Time(s) Label homebanking 23 ... 4.4 dom_HomeBanking Domain−0 0 ... 2658.8 dom_SystemManagement
LISTING LABELED RESOURCES
xm resources phy:hda6 policy: example.chwall_ste.client_v1 label: res_LogicalDiskPartition1(hda1) file:/xen/disk_image/disk.img policy: example.chwall_ste.client_v1 label: res_LogicalDiskPartition2(hda2)
We distinguish three representations of the Xen access control policy: the source XML version, its binary counterpart, and a mapping representation that enables the tools to deterministically translate back and forth between label names of the XML policy and label identifiers of the binary policy. All three versions must be kept consistent to achieve predictable security guarantees.
The XML version is the version that users are supposed to create or change, either by manually editing the XML file or by using the Xen policy generation tool (xensec_gen). After changing the XML file, run the makepolicy subcommand to ensure that these changes are reflected in the other versions. Use, for example, the subcommand cfgbootpolicy to activate the changes during the next system reboot.
The binary version of the policy is derived from the XML policy by tokenizing the specified labels and is used inside Xen only. It is created with the makepolicy subcommand. Essentially, the binary version is much more compact than the XML version and is easier to evaluate during access control decisions.
The mapping version of the policy is created during the XML-to-binary policy translation (makepolicy) and is used by the Xen management tools to translate between label names used as input to the tools and their binary identifiers (ssidrefs) used inside Xen.
Sean Dague <sean at dague dot net> Daniel Stekloff <dsteklof at us dot ibm dot com> Reiner Sailer <sailer at us dot ibm dot com>