Nutanix Community Edition (lab)

MetalSoft provides an application extension that installs Nutanix Community Edition (CE) 2.1 on bare-metal servers and creates a Nutanix cluster from them. Tenants deploy the cluster from the Infrastructure Designer; MetalSoft installs AOS and the AHV hypervisor on every node through an OS template, and the extension’s Ansible playbook then configures the Controller VMs and creates the cluster.

The extension definition and Ansible playbooks live in the Nutanix CE extension repository, and the OS template in the MetalSoft OS templates repository. Their READMEs document the full deployment flow. This page covers the MetalSoft-side setup.

Lab and evaluation use only

Community Edition is Nutanix’s free, community-supported edition. It runs unlicensed, needs internet access, is limited to 4 nodes, and is not supported for production. The CE binaries are Nutanix property under the CE EULA: keep your hosted copies on an internal or access-controlled repository.

What gets deployed

Component	Version
AOS (Controller VM software)	6.8.1
AHV (hypervisor)	20230302
Installer	Community Edition 2.1 (phoenix.x86_64-fnd_5.6.1_patch-aos_6.8.1_ga.iso)

Each node runs AHV on bare metal, with one Controller VM (CVM) that owns the node’s data disks. All nodes attach to one untagged management network that carries the AHV host IPs, the CVM IPs, the cluster virtual IP (Prism Element) and the data services IP. The cluster uses redundancy factor 1 on a single node and redundancy factor 2 on 3 or 4 nodes.

The deployment is fully unattended:

• No Foundation VM, web server or DHCP. The CE installer runs in its unattended mode, with each node’s answer file built into the per-instance ISO that MetalSoft generates from the OS template.
• No BMC access from the playbooks. MetalSoft installs the nodes over virtual media, as for any other OS template.
• Re-runs are safe. Every playbook step checks the current state first (AHV ready, CVM address set, cluster created, password set), so a failed run can be retried.

Prerequisites

1. A MetalSoft controller running release 7.5 or newer, with the Ansible Runner capability enabled on the site controller. See Enabling the Ansible Runner Capability for more details.
2. The OS template nutanix-ce-2-1-cluster-node registered at the target site, and its installation files hosted on a repository the servers can reach (see Registering the OS template).
3. A logical network profile at the target site for the cluster network (see Preparing the logical network).
4. Outbound internet access from the cluster network. Community Edition needs it to create the cluster and for the first Prism login.
5. Servers that meet the server requirements.
6. A My Nutanix (NEXT) account with the Community Edition role, for the first Prism login. Register for Community Edition on the Nutanix website with the account’s email address; the role is granted after the confirmation email.
7. The extension’s site configuration saved on every target site (see Creating the Nutanix CE application extension). Because the extension declares site-level configuration variables, a deployment fails until an operator opens the extension’s site config on each site and saves it. All variables ship with defaults, so this is a one-click save unless overrides are needed.
8. Optional: a DNS workflow extension, such as PowerDNS, so that the prism.<cluster>.<zone> record resolves. Without one, use the cluster_vip output.

Server requirements

Item	Requirement
Firmware	UEFI boot, Secure Boot disabled, BIOS clock set correctly (a wrong clock breaks cluster creation)
CPU	x86_64 with VT-x or AMD-V, at least 4 cores
Memory	At least 32 GB, 64 GB recommended. The installer stages its downloads in memory: servers with 8 GB fail to install
Disks	At least 3, presented raw (AHCI, or an HBA / IT-mode controller; no RAID volumes, HPE Smart Array in HBA mode): AHV boot disk (32 GB or more), CVM boot SSD (200 GB or more), and one or more data disks (200 GB or more each)
Network	At least one 1 GbE or faster NIC with a driver AHV supports, on plain switch ports (no LAG)

Use identical servers for all nodes of a cluster. All nodes share one disk layout (see Disk layout).

Registering the OS template

The extension requires the nutanix-ce-2-1-cluster-node OS template (Nutanix/CE-2.1/cluster-node in the MetalSoft OS templates repository). The template:

• boots the stock Community Edition installer ISO with a replacement boot menu that passes the node’s network settings and answer file to the installer;
• installs AHV and the CVM without keyboard input, and leaves the CVM without an external IP (the extension assigns it after the installation);
• fixes the server’s UEFI boot order before the installer reboots, so that the server boots AHV from its local disk rather than the still-attached installer ISO;
• uses the wait_for_ssh ready method, so MetalSoft hands the node over once AHV answers on SSH.

1. Host the installation files. The template expects three files on a web repository that the Global Controller (for the image build) and the servers (for the installer downloads) can reach:

   File	Used as
   phoenix.x86_64-fnd_5.6.1_patch-aos_6.8.1_ga.iso	The Community Edition installer ISO, downloaded from the Nutanix community portal after registering for CE. The template’s source image.
   nutanix_installer_package.tar	The AOS package, extracted from the ISO. Downloaded by the installer on every node.
   AHV-DVD-x86_64-el8.nutanix.20230302.101026.iso.iso	The AHV installer (the double .iso is the real name), copied from the ISO. Downloaded by the installer on every node.

   In unattended mode the installer downloads AOS and AHV over HTTPS rather than reading them from the attached ISO, so extract them from the ISO and host them next to it:

   xorriso -osirrox on -indev phoenix.x86_64-fnd_5.6.1_patch-aos_6.8.1_ga.iso \
     -extract /images/svm svm -extract /images/hypervisor/kvm kvm
   cat svm/nutanix_installer_package.tar.p00 svm/nutanix_installer_package.tar.p01 > nutanix_installer_package.tar
   cp kvm/AHV-DVD-x86_64-el8.nutanix.20230302.101026.iso.iso .

   Each node downloads about 6.2 GB. For large or remote sites, use a site-local mirror.

2. Point the template at your copies: the ISO URL in template.yaml, and the aos and ahv URLs in grub.cfg. Keep the MD5 sums next to the URLs in grub.cfg; the installer verifies both downloads against them.

3a. Create the template (with internet access):

metalcloud-cli os-template list-repo
metalcloud-cli os-template create-from-repo Nutanix/CE-2.1/cluster-node

3b. Create the template (air-gapped, or after editing the URLs). Copy the os-templates directory, including its .git files, to a machine that runs the CLI and has access to the Global Controller, then run:

metalcloud-cli os-template create-from-directory Nutanix/CE-2.1/cluster-node --dir /path/to/os-templates

Disk layout

The template installs AHV on sda, the CVM boot partition on sdb and the data on sdc, which matches a server with three disks on one controller. For another layout, set the following custom variables on the server instance group of the deployment. Custom variables set on the extension’s instance array reach only the Ansible playbook, not the OS template.

Custom variable	Example	Meaning
ce_hyp_boot_disk	sda	AHV boot disk
ce_cvm_boot_disks	nvme0n1	CVM boot SSD(s), comma-separated, 1 or 2
ce_cvm_data_disks	sdb,sdc	Data disks, comma-separated

The values are kernel device names as the installer sees them. To find them, boot any Linux live ISO on one server with the virtual CD attached (virtual media can shift sdX names) and run lsblk -d -o NAME,SIZE,ROTA,MODEL. Always set the AHV disk explicitly on servers with equal-size disks: left to itself, the installer may pick a disk intended for the CVM.

Preparing the logical network

The extension declares one logical network, ntnx-ce-mgmt. A logical network profile with the label referenced by its profileLabel (simple-vlan-network in the repository) must exist at every site where the cluster will be deployed. Go to Admin > Fabrics > fabric > Logical networks and create or adjust the profile:

• Untagged (native VLAN) on the server ports. The installer cannot tag the AHV host or CVM network.
• Not in 192.168.5.0/24, which Nutanix uses for the internal link between each AHV host and its CVM.
• Outbound internet access from the subnet, and access to the repository that hosts the installation files.
• Enough addresses. Each node uses two host addresses (see Limitations), and the extension reserves a 4-address range for the CVMs, plus one address each for the cluster virtual IP and the data services IP: 12 addresses for a 3-node cluster and 14 for a 4-node cluster. A /29 subnet is too small.

The extension attaches every server to the network untagged, active-backup, as the default route, with instance DNS records enabled. It also creates a prism.<cluster>.<zone> DNS record for the cluster virtual IP.

Creating the Nutanix CE application extension

1. Clone the repository containing the extension definition:

git clone https://github.com/metalsoft-io/metalsoft-extensions
cd metalsoft-extensions/nutanix-ce

The definition declares two assets: the Ansible bundle (AnsibleBundle) and the execution environment image (OciImage). Both are already built and hosted, so no repackaging is needed unless you change the playbooks. Adjust infrastructure.logicalNetworks[].profileLabel to match the logical network profile at your sites before you create the extension.

2. Create and activate the extension:

metalcloud-cli extension create "Nutanix Community Edition" application "Nutanix Community Edition (lab) on bare metal" --definition-source extension.json
metalcloud-cli extension activate <id-of-created-extension>

Activated extensions cannot be edited in place. To change the definition, bump extensionVersion and create the extension again. MetalSoft keeps each version as a separate extension.

3. Enable the extension on the site level and save its configuration. After activation, the extension must be activated per site on the site level configuration page. This is also where its configVars, the operator-set, site-level settings, are configured, and it must be done before the first deploy. To save the configuration from the CLI:

echo '[{"label":"DNSResolvers","value":"10.0.0.53"},{"label":"NTPServers","value":"pool.ntp.org"}]' | metalcloud-cli extension site-config set <id-of-created-extension> <site> --config-source pipe

You are now ready to deploy the Nutanix CE application from the Infrastructure Designer.

Site configuration (configVars)

An operator sets these once per site. They apply to every Nutanix CE deployment at that site:

Variable	Type	Default	Description
DNSResolvers	String	""	Comma-separated DNS servers for the cluster. Empty uses the site’s DNS resolvers.
NTPServers	String	""	Comma-separated NTP servers for the cluster. Empty uses the site’s NTP servers, else pool.ntp.org.

Deployment inputs

Each deployment created from the Infrastructure Designer takes the following inputs:

Input	Type	Default	Description
node_server_type	ServerType	—	Server type for the nodes (at least 32 GB of RAM).
node_count	Integer	3	Number of nodes: 1, 3 or 4 (Community Edition does not support 2-node clusters). Set at creation only.
node_os_template	OsTemplate	—	OS template for the nodes. Must be nutanix-ce-2-1-cluster-node.
prism_admin_password	String (password)	—	Password for the Prism admin user: 8–32 characters, and must not contain nutanix, ntnx or password. The playbook sets it on every run, overwriting a password changed in Prism.

The cluster is named after the extension instance.

Deployment flow

1. MetalSoft allocates the servers and the addresses, builds a per-node ISO from the OS template and installs every node over virtual media. The installer downloads AOS and AHV, installs both, fixes the UEFI boot order and reboots into AHV. This takes about 35 minutes.
2. AHV’s first boot defines and starts the CVM (15–20 minutes). MetalSoft hands the node over once AHV answers on SSH.
3. The extension’s postDeploy task runs deploy.yaml on the Site Controller:
   1. Preflight checks on the inputs and the allocated addresses.
   2. Waits until AHV first boot has finished on every node and the CVM is running.
   3. Gives each CVM its address from the reserved CVM range. Each AHV host reaches its own CVM over the internal 192.168.5.254 link. A CVM that already holds its address is left alone, and the run stops if another machine answers on a CVM address.
   4. On the first CVM, creates the cluster across all CVMs (unless it already exists), then sets the data services IP and the Prism admin password.
   5. Publishes the outputs.

A 4-node deployment typically takes 60–90 minutes from start to a running cluster.

Outputs

After a successful deployment, the extension publishes the following outputs on the extension instance:

Output	Description
prism_element_url	Prism Element URL, https://prism.<cluster>.<zone>:9440 (or the virtual IP when no DNS name is allocated)
cluster_name	Cluster name (the extension instance name)
cluster_vip	Cluster virtual IP
data_services_ip	Data services IP (iSCSI and Nutanix Volumes)
cvm_ips	CVM addresses, comma-separated, in node order
ahv_host_ips	AHV host addresses, comma-separated, in node order

Using the cluster

1. Open prism_element_url, or https://<cluster_vip>:9440.
2. Log in as admin with the prism_admin_password input. If the playbook warned that it could not set the password, log in with the Community Edition default password nutanix/4u and change it.
3. Sign in with your My Nutanix (NEXT) account. Community Edition requires this on the first login, and it has no API, so it stays a manual step. An account without the Community Edition role is rejected with “does not have the required Community Edition role”.

Default credentials

The AHV root user and the CVM nutanix user keep the Community Edition default password nutanix/4u. The extension only changes the Prism admin password. Change the others after handover if the cluster is reachable by anyone other than its users.

Execution environment

The extension’s Ansible playbooks run on the Site Controller inside an execution environment (ee-nutanix-ce), a container image holding all the dependencies the playbooks need. The extension ships its own image, declared as an OciImage asset in the extension definition and selected by the Ansible task through the ee option, so no Site Controller configuration is required. The image is based on UBI 10 and requires an x86-64-v3 (AVX2) CPU on the Site Controller. To customize it, rebuild the image from the execution-environment.yml recipe in the extension repository (on an x86_64 machine), and update the OciImage asset in the definition with a new tag.

Limitations

• Lab use only. Community Edition is unlicensed and not supported for production.
• No scaling. Community Edition supports only 1, 3 or 4 nodes, so node_count is set at creation only. On delete, MetalSoft wipes the servers as usual.
• Two host addresses per node. Each NIC on the management network receives its own address, but only the one carrying the default route is used (AHV bonds both NICs under it).
• Fixed versions. Community Edition 2.1 ships AOS 6.8.1 and AHV 20230302. Upgrade through LCM in Prism.
• Manual first login. The NEXT account sign-in in Prism cannot be automated.

Troubleshooting

Symptom	Cause and fix
The deployment fails because the extension is not enabled on the site	Save the extension site config on that site.
The OS image build fails with “Invalid encrypted data”	The template’s text assets must have the templating engine enabled. Re-register the template from the repository without changing the asset settings.
The installer console shows a Python traceback	The answer file was rejected, usually because of a disk name that does not exist on the server. Check the disk layout custom variables.
A node reinstalls in a loop (a fresh “Copying squashfs.img” after an install)	The server booted the installer ISO again. Select the local disk in the one-time boot menu once the installation finishes, then remove stale “Nutanix AHV” or “Unavailable” UEFI boot entries in the BIOS boot manager.
The playbook keeps waiting for AHV first boot	On the AHV host (root), check systemctl status ahv-define-cvm ahv-install-cvm ahv-configure-cvm ahv-status-cvm and virsh list --all. A failed ahv_first_boot.service is expected and harmless on this AHV version.
The run stops with “is answered by a different machine than this host’s CVM”	Another machine (for example another deployment) uses one of the CVM addresses. Free the address, then retry.
A warning that the CVM cannot reach my.nutanix.com	Usually transient right after the CVM gets its address. If it persists, the cluster network has no outbound internet access, and the first Prism login will fail.
Cluster creation fails	On the first CVM (nutanix), check cluster status and ~/data/logs/genesis.out. Check the BIOS clock and NTP.

Retrying the failed task re-runs the playbook from the current state. Each run’s output is available from the extension instance’s job log, and on the Site Controller with docker logs on the run’s container.