IP address Management (IPAM)#
The MetalSoft built-in IPAM system provides IP address management. The service manages both:
subnet segmentation (breaking a larger IP block such as a /24 into smaller blocks such as /30, /29 etc). We call the larger blocks Subnet Pools and the smaller ones Subnets.
IP allocation from Subnets
How does it work?#
MetalSoft differs from other solutions in that it is declarative. This means that instead of directly allocating an IP on a server, the users specify a specific source of IP addresses in the blueprint designer (Infrastructure editor). This is because blueprints can be deployed in any datacenter and on different hardware architectures and blueprints are meant to be portable.
When a blueprint is designed the user specifies a number of servers, the source subnet pools and can override certain parameters such as the VLAN ID. The ‘request’ is then translated into actual IPs and switch and server configuration.
We call “WAN” (wide area network) the networks that need to be reachable from outside an infrastructure, typically via a gateway. Each infrastructure receives a gateway interface private to that infrastructure and a private subnet from a WAN subnet pool. The gateway is usually a switch virtual interface (SVI), part of the same L2 broadcast domain (such as VLAN) and configured with an IP from the subnet that was allocated to the infrastructure.
Adding a WAN subnet#
To add a WAN subnet pool go to the Subnets section of the admin and click on the Add Subnet button
The following parameters are used:
Datacenter The datacenter where the subnet is to be used.
Subnet The subnet in CIDR notation.
Routable Specifies if the IP address is routable.
When the system needs a new subnet (for example for WAN or for LAN) it will verify that the destination of the subnet pool matches that of the subnet.
If set to Automatic the system will automatically allocate subnets to infrastructures from this Subnet pool. If this Subnet Pool doesn’t have enough continuous IP space available for a new subnet the next Subnet Pool with the same destination will be used.
If set to Manual, the system will only allocate the subnets if configured to do so from the Advanced tab of an InstanceArray in the Infrastructure Editor. Checkout Enabling Secondary IPs and/or VLANs on the WAN interface for more information.
The IPs allocated are also used to reach servers after deployment to test if the deployment is finished thus they need to be reachable from the Agents. For example if you configure an 192.168.0.1/24 subnet and configure it on an interface but without having a routing rule in the router VM (or the agent machine), a deploy will be stuck in the waitForSSH phase with an error similar to “host unreachable 192.168.0.22”
The IPs allocated by MetalSoft can be configured on the hosts either via DHCP or statically, there will be no overlap as they are tied to an Instance_id rather than to a MAC address. The configuration is done in the OS template.
When are subnets allocated?#
Subnets are allocated when servers are provisioned, if needed, as follows: The size of the subnets (/30 or /29 or /28…) depend on the number of servers in an infrastructure in the same L2 broadcast domain. For example:
If an infrastructure has a single server a /30 (2 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway SVI and the other one on the server.
If an infrastructures has 2-5 servers a /29 is created (6 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway and the others on the servers.
If an infrastructures has 5-13 servers a /28 is created (14 usable addresses) is created. One of the usable IPs addresses is allocated on the gateway and the others on the servers.
and so forth.
When a new server is added after the initial deploy:
If the IP space large enough to hold the new serve count, next to the initial subnet is not allocated to another infrastructure the subnet can grow (turn a /30 into a /29 for example).
If the IP space next to the allocated subnet is not available, a new subnet will be created.
When are subnets freed?#
Automatically allocated subnets are de-allocated from an infrastructure when there are no longer instances with IPs allocated from it. Manually allocated subnets will not be automatically freed except when the infrastructure is deleted.
When an infrastructure is created that has at least a server in the WAN network, a new subnet is allocated and a gateway needs to be created. The system will create automatically one or more SVIs (two in the case of MLAG configurations) that act as gateways for the respective networks.
Note: If there is an existing SVI with the same VLAN ID the system will simply reuse that and will not delete it even if all infrastructures using that are deleted.
Custom VLAN IDs and Subnets#
When a custom VLAN id is configured in the Advanced tab of the infrastructure editor (on a primary or secondary IP), that VLAN ID is used instead of random one that is allocated from the VLAN ranges specified on the Datacenter configuration page (
Given the fact that the subnet cannot traverse multiple L2 broadcast domains (VLANs, EPG, VPLS, eVPN etc.) the VLAN ID is also tied to the subnet that has been created for that VLAN ID. This means that if a server is configured in the Advanced tab with a different VLAN id (or automatic) than the ones in the currently allocated subnets, a new subnet will be created from the specified subnet pool, with the new VLAN ID. To illustrate this consider the following:
In the following case there are 2 servers in this infrastructure, configured differently. Two different gateway interfaces and two different subnets are created for the two different vlan id.
If a new server is added, with the same source subnet pool but another vlan id a third subnet is created instead of the existing one expanded.
In this third scenario, the thrid server is configured with the same custom IP pool but with automatic VLAN ID. In this case a separate Subnet Pool is also created but tied to the infrastructure’s default VLAN ID and the gateway IP is added to the same switch SVI.
Secondary IPs vs Primary IPs#
We have the concept the Primary IP and Secondary IPs. They can be allocated from different subnet pools and can have different custom VLAN IDs.
If the secondary IP has the same VLAN ID as the Primary IP, both IPs are added to the same interface
If the secondary IP has a different VLAN id as the Primary IP, the secondary IP is added on a separate loopback interface, configured with the custom VLAN id and the switch interface is configured in trunk mode to allow the custom VLAN.