Network topology

A fundamental property of all MetalSoft managed environments is tenant separation which is achieved through network segmentation. From the network’s point of view a tenant is a group of users that manage a series of endpoints (such as servers) and the way those endpoints are connected to each other. Each tenant’s network is by default isolated from the others at L2. This eliminates the possibility of, among other things, ARP/ND poisoning between tenants.

The actual implementation depends on the Provisioner configured on the Datacenter object

  • EVPNVXLAN -> Uses L2 EVPN for tenant isolation. Supported on Dell OS10, Dell SONIC, Juniper, Cumulus (upcoming)

  • VLAN -> Uses simple VLANs for tenant isolation. Supported on Dell OS10, Dell SONIC, Juniper, Cumulus, HP Comware (Deprecated)

  • SDN -> This uses a 3rd party orchestrator. Currently supported on Cisco ACI. Uses EPGs for each network.

  • VPLS -> This uses BGP signaled MPLS for tenant isolation. Supported on HP Comware, H3C (Deprecated)

Data plane logical network topology

At a very high level MetalSoft manages a collection of isolated tenant networks. Tenants are typically either external clients or internal applications and contain one or more servers that need to talk to each other and maps to an Infrastructure.

From the tenant’s perspective the network only includes the servers and networks that are private to that tenant.

Within an infrastructure the tenants have a certain level of freedom to create private LANs, with separate L2 broadcast domains but also to join servers to the company (or “public”) WAN network that provides access to internet or to connect to external connections. The connections up to the WAN gateway is also a private L2 network to prevent lateral movement attacks and broadcast traffic sniffing.

SAN access is a special type of network. While many organizations have Fiber Channel type storage networks this document suggests using iSCSi or other IP-based solutions such as ROCEv2 enabling next generation NVMe-over-fabrics based storage and more.

SAN networks also need to be private as they can become an attack vector for lateral movement. Thus, this architecture suggests the use of separate SAN L2 networks per tenant terminated on virtual ports on the storage appliances.

Gateway management

Since each tenant could have separate L2 broadcast domains, they also need separate gateways. MetalSoft supports multiple approaches:

  1. MetalSoft managed gateway residing on leaf switches, one per L2 network (Default behavior)

  2. Externally managed gateway residing on external router/firewall devices

Note: To toggle this behavior, in the Admin UI, go to Datacenters > Datacenter name > Network Profiles > Click on the network profile and click on Edit next to the specific entry and tick the Provision subnet gateways as desired

MetalSoft-managed gateways

In the following diagram, MetalSoft has provisioned multiple L3 interfaces to act as gateways for the two different tenants (red and blue). Note that the actual number of gateways depends on the provisioner:

  • EVPN-VXLAN provisioner uses Symmetric Integrated routing and bridging (Symmetric IRB) whereby the same IP is present on multiple switches

  • VLAN provisioner (deprecated) uses a single L3 interface placed randomly on a leaf switch

  • SDN-Cisco ACI provisioner configures a bridge domain.

  • VPLS provisioner (deprecated) uses a single L3 interface on a ToR switch or, if the same L2 domain is spread across multiple ToR switches, it will reside on a separate “north” switch.

Externally managed gateways

In certain situations, the tenant gateway may reside on a firewall that is usually connected to an “edge router”. In this scenario, MetalSoft should be configured to not create the gateway interfaces and instead extend the L2 domain of the tenant to an “external connection” interface on a border switch, as a tagged VLAN. The firewalls will then use the respective VLAN and potentially VRRP to enable a redundant gateway and also apply firewall rules.