IP address Management (IPAM)

The MetalSoft built-in IPAM system provides IP address management. The service manages both:

  1. subnet segmentation (breaking a larger IP block such as a /24 into smaller blocks such as /30, /29 etc). We call the larger blocks Subnet Pools and the smaller ones Subnets.

  2. IP allocation from Subnets

How does it work?

MetalSoft differs from other solutions in that it is declarative. This means that instead of directly allocating an IP on a server, the users specify a specific source of IP addresses in the blueprint designer (Infrastructure editor). This is because blueprints can be deployed in any datacenter and on different hardware architectures and blueprints are meant to be portable.

When a blueprint is designed the user specifies a number of servers, the source subnet pools and can override certain parameters such as the VLAN ID. The ‘request’ is then translated into actual IPs and switch and server configuration.

WAN Subnets

We call “WAN” (wide area network) the networks that need to be reachable from outside an infrastructure, typically via a gateway. Each infrastructure receives a gateway interface private to that infrastructure and a private subnet from a WAN subnet pool. The gateway is usually a switch virtual interface (SVI), part of the same L2 broadcast domain (such as VLAN) and configured with an IP from the subnet that was allocated to the infrastructure.

Adding a WAN subnet

To add a WAN subnet pool go to the Subnets section of the admin and click on the Add Subnet button

The following parameters are used:

  • Datacenter The datacenter where the subnet is to be used.

  • Subnet The subnet in CIDR notation.

  • Routable Specifies if the IP addresses can be used to reach the internet. This flag is used in operating system templates to determine what gateway to set as a default gateway.

  • Destination

When the system needs a new subnet (for example for WAN or for LAN) it will verify that the destination of the subnet pool matches that of the subnet.

  • Infrastructure Allocation

    • If set to Automatic the system will automatically allocate subnets to infrastructures from this Subnet pool. If this Subnet Pool doesn’t have enough continuous IP space available for a new subnet the next Subnet Pool with the same destination will be used.

    • If set to Manual, the system will only allocate the subnets if configured to do so from the Advanced tab of an InstanceArray in the Infrastructure Editor. Checkout Enabling Secondary IPs and/or VLANs on the WAN interface for more information.

  • Single Subnet - This options is only available if Infrastructure Allocation is set to Manual. It effectively transforms this subnet pool into a single subnet which allows the allocation of individual IPs from it. This allows the subnet pool to be used to allocate IPs across infrastructures.

  • Gateways - If set will override the automatically allocated gateway from a subnet (usually the first ip from a block) to something else.

  • Default VRF -

  • Switch whitelisting - If enabled it restricts the allocation of this subnet to only certain switches.

  • Blacklisted IP - If set will not allocate IPs from the respective blacklist

  • Resource Pool - The resource pool to which this subnet belongs to

Notes:

  • The IPs allocated are also used to reach servers after deployment to test if the deployment is finished thus they need to be reachable from the Agents. For example if you configure an 192.168.0.1/24 subnet and configure it on an interface but without having a routing rule in the router VM (or the agent machine), a deploy will be stuck in the waitForSSH phase with an error similar to “host unreachable 192.168.0.22”

  • The IPs allocated by MetalSoft can be configured on the hosts either via DHCP or statically, there will be no overlap as they are tied to an Instance_id rather than to a MAC address. The configuration is done in the OS template.

When are subnets allocated?

Subnets are allocated when servers are provisioned, if needed, as follows: The size of the subnets (/30 or /29 or /28…) depend on the number of servers in an infrastructure in the same L2 broadcast domain. For example:

  1. If an infrastructure has a single server a /30 (2 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway SVI and the other one on the server.

  2. If an infrastructures has 2-5 servers a /29 is created (6 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway and the others on the servers.

  3. If an infrastructures has 5-13 servers a /28 is created (14 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway and the others on the servers.

  4. and so forth.

When a new server is added after the initial deploy:

  1. If the IP space large enough to hold the new serve count, next to the initial subnet is not allocated to another infrastructure the subnet can grow (turn a /30 into a /29 for example).

  2. If the IP space next to the allocated subnet is not available, a new subnet will be created.

When are subnets freed?

Automatically allocated subnets are de-allocated from an infrastructure when there are no longer instances with IPs allocated from it. Manually allocated subnets will not be automatically freed except when the infrastructure is deleted.

WAN Gateways

When an infrastructure is created that has at least a server in the WAN network, a new subnet is allocated and a gateway needs to be created. The system will create automatically one or more SVIs (two in the case of MLAG configurations) that act as gateways for the respective networks.

Note: If there is an existing SVI with the same VLAN ID the system will simply reuse that and will not delete it even if all infrastructures using that are deleted.

Custom VLAN IDs and Subnets

When a custom VLAN id is configured in the Advanced tab of the infrastructure editor (on a primary or secondary IP), that VLAN ID is used instead of random one that is allocated from the VLAN ranges specified on the Datacenter configuration page (WANVLANRange property).

Given the fact that the subnet cannot traverse multiple L2 broadcast domains (VLANs, EPG, VPLS, eVPN etc.) the VLAN ID is also tied to the subnet that has been created for that VLAN ID. This means that if a server is configured in the Advanced tab with a different VLAN id (or automatic) than the ones in the currently allocated subnets, a new subnet will be created from the specified subnet pool, with the new VLAN ID. To illustrate this consider the following:

In the following case there are 2 servers in this infrastructure, configured differently. Two different gateway interfaces and two different subnets are created for the two different vlan id.

If a new server is added, with the same source subnet pool but another vlan id a third subnet is created instead of the existing one expanded.

In this third scenario, the thrid server is configured with the same custom IP pool but with automatic VLAN ID. In this case a separate Subnet Pool is also created but tied to the infrastructure’s default VLAN ID and the gateway IP is added to the same switch SVI.

Secondary IPs vs Primary IPs

We have the concept the Primary IP and Secondary IPs. They can be allocated from different subnet pools and can have different custom VLAN IDs.

  1. If the secondary IP has the same VLAN ID as the Primary IP, both IPs are added to the same interface

  1. If the secondary IP has a different VLAN id as the Primary IP, the secondary IP is added on a separate loopback interface, configured with the custom VLAN id and the switch interface is configured in trunk mode to allow the custom VLAN.

Adding a new subnet pool using the CLI

The OOB subnet pool is used to allocate IPs on the OOB network to servers. It is advisable not to use routable IP addresses on this network.

datacenter: test-datacenter
prefix: 172.16.200.0
netmask: 255.255.255.0
size: 24
type: ipv4
routable: false
destination: WAN
manualAllocationOnly: true

Note the manualAllocationOnly that disables automatic allocation to avoid accidental allocation to a user using the automatic settings. The routable flag controls if the IP allocated can be used by the system to check if the OS is installed, to configure the firewall or to SSH into the OS to install applications such as Kubernetes.

Create the subnet using the CLI:

metalcloud-cli subnet-pool create -config ./subnet.yaml --return-id

Virtual Routing and Forwarding - VRFs

Virtual routing and forwarding (VRF) IP technology allows users to configure multiple routing table instances to simultaneously co-exist within the same router. Overlapping IP addresses can be used without conflicting because the multiple routing instances are independent, and can select different outgoing interfaces.

VRFs are also used for Symmetric IRB gateways.

Creating a VRF

Go to Subnets > VRFs tab. Click Add VRF Fill in the datacenter, VLAN ID, VNI ID

Adding a VRF-VLAN mapping

Go to Subnets > VRFs > VLAN mappings tab. Select the VRF and the VLAN ID to map.