Managing firmware catalogs

Available since: version 6.1

Firmware catalogs allows you to filter firmware upgrades for certain components. This is required for VMWare ESXI or other hardware dependent operating systems. Vendors provide these catalogs and to use them a “Firmware catalog” record needs to be created and the binaries referenced by the catalogs need to be downloaded from the vendor repository and uploaded to the MetalSoft repository.

Preparing an HTTP repository

A local http repository is required to allow the agents to pull the binary assets from before uploading them to the hosts. This can be any repository including an object storage system with the caveat that the metalcloud-cli can only upload to SSH hosts at the moment but the files can be uploaded manually. This can be the same as the one used to store operating system images or different.

Recommended configuration for the http repository VM:

• Resources:

  • 2x CPU cores

  • 4GB RAM

  • 32GB volume mounted under the web server’s Document Root path (ex. /var/www/html)

• OS: latest Ubuntu LTS version (currently 22.04)

• Services:

  • Web server

    • Document Root folder set to the path where the extra storage is mounted (best to use the chosen web server’s default)

    • Allow auto indexing of files in the Document Root

  • SSH server

• Users:

  • a single non-admin user with read/write access to the Document Root path

  • the local user’s (running metalcloud-cli) public key added to ~/.authorized_keys on the server

• Firewall:

  • allow access to port 22 from local machine (the one metalcloud-cli is run from)

  • allow access to port 80 from the OOB network of your infrastructure

• Upgrades: set server to auto-apply security updates and auto-reboot as needed at a specified time of day using Ubuntu’s unatteneded-upgrades package

Configuring the CLI for managing firmware catalogs and firmware binaries

Before using the metalcloud cli to manage firmware catalogs and binaries, there are multiple environment variables that need to be set.

Repository-related variables

• METALCLOUD_FIRMWARE_REPOSITORY_URL: Holds the URL used to access the repo, for example, http://176.223.226.61/repo/firmware/. This property is overridden by the CLI parameter repo-http-url.

• METALCLOUD_FIRMWARE_REPOSITORY_SSH_PATH: Holds the SSH internal path in which the binaries will be uploaded, for example: /home/repo/firmware. This property is overridden by repo-ssh-path.

• METALCLOUD_FIRMWARE_REPOSITORY_SSH_PORT: Holds the SSH port number. The default is 22 if not given. Value gets overridden by repo-ssh-port.

• METALCLOUD_FIRMWARE_REPOSITORY_SSH_USER: Holds the SSH username. Value gets overridden by repo-ssh-user.

User-related variables

• METALCLOUD_USER_PRIVATE_OPENSSH_KEY_PATH: Holds the path to the private OpenSSH key. Currently, only keys with no password are supported. If not given, the default is the path formed by the home folder, .ssh folder, and id_rsa file. This property is overridden by user-private-ssh-key-path.

• METALCLOUD_KNOWN_HOSTS_FILE_PATH: Holds the path to the known_hosts file. The default is home folder, .ssh folder, and known_hosts file (for example on Linux ~/.ssh/known_hosts, on Windows C:\Users\Alex.ssh\known_hosts).

• METALCLOUD_API_KEY: The API key used to make calls to Metalsoft endpoints. This key is used when making the calls to create the catalog and batch import the firmware binaries.

• METALCLOUD_ENDPOINT: The endpoint of the environment where the catalog will be created.

HP-related variables

• HP_SUPPORT_TOKEN: For older generation servers, like gen9, in order to retrieve the catalog, a token is needed, which can be generated from the following link: HPE Software Delivery Repository. In general, if the HP catalogs can’t be retrieved due to authentication issues, a token needs to be provided.

Creating a firmware catalog record and uploading binaries

Dell example

1. Create a catalog record file, shown here in yaml format:

name: test-dell #name of the catalog
description: test  #description of the catalog
vendor: dell  #one of Dell, HP, Lenovo
catalog_url: https://downloads.dell.com/FOLDER04655306M/1/ESXi_Catalog.xml.gz #url towards the original catalog
local_catalog_path: ./ESXi_Catalog.xml #what XML to build based 
local_firmware_path: ./downloads

2. Create the record download and upload to the local repository.

metalcloud-cli firmware-catalog create --config-format yaml --raw-config .\example_dell.yaml --download-binaries --filter-server-types "M.32.32.2, M.32.64.2, M.40.32.1.v2" --repo-http-url http://176.223.226.61/repo/firmware/ --repo-ssh-path /home/repo/firmware --repo-ssh-port 22 --repo-ssh-user root --user-private-ssh-key-path ~/.ssh/id_rsa

The command retrieves firmware binary information from the given catalog, downloads the binaries locally (either to a folder or temporarily in the tmp folder), and then uploads the binaries to an HTTP repository (unless the skip-upload-to-repo property is set to true). The user must provide a path to the private SSH key in OpenSSH format for authentication and the public key must be set on the server holding the repository in ~/.ssh/authorized_keys. Currently, only keys with no password are supported.

CLI firmware-catalog create command behavior

After setting up the environment variables and/or overriding them with the CLI parameters a user can use the repository to upload the binaries. By default, if a binary with the same name already exists, it won’t be replaced. Force replacement can be issued by using the --replace-if-exists CLI parameter. If the user doesn’t want to upload the binaries at all just use the --skip-upload-to-repo parameter.

When connecting to the repository, the keys found in the known_hosts file will be used for authentication. If no key is found for a hostname, the user will be prompted to accept this new connection, otherwise, the connection will be aborted. If --skip-host-key-checking is set to true, then the user will not be prompted to accept.

The authenticity of host '192.168.74.1:4022' can't be established.
SSH key is ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtdJbYCWJobS1wlVe4O3yItBUVNyRZsN8m+CqbmAgrQB1J1ksKqM5DEKXyM3Rf4KpfDSQgcBK5LAi8DASPZUH0=.
This key is not known by any other names.
It will be added to known_hosts file C:\Users\Alex\.ssh\known_hosts.
Are you sure you want to continue connecting (yes/no)?
yes
Added key ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtdJbYCWJobS1wlVe4O3yItBUVNyRZsN8m+CqbmAgrQB1J1ksKqM5DEKXyM3Rf4KpfDSQgcBK5LAi8DASPZUH0= to known_hosts file C:\Users\Alex\.ssh\known_hosts.Established connection to hostname 192.168.74.1:4022.

If a key is already found and it doesn’t match the host public key from the repository, an error message will be shown to change manually the key. Either the hostname/key changed or a man-in-the-middle attack is happening.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The key sent by the remote host is
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtdJbYCWJobS1wlVe4O3yItBUVNyRZsN8m+CqbmAgrQB1J1ksKqM5DEKXyM3Rf4KpfDSQgcBK5LAi8DASPZUH0=.
Please contact your system administrator.
Add correct host key in C:\Users\Alex\.ssh\known_hosts to get rid of this message.
Host key for 192.168.74.1:4022 has changed and you have requested strict checking.
Host key verification failed.

Customizing a catalog

Catalogs can be customized to allow only certain firmware components to be used. This is done by using the --raw-config, --config-format, and filter-server-types parameters.

With the raw-config parameter, a path to a file is passed, which holds information about that catalog and from where to retrieve it while config-format holds the format of the file, which can be either YAML or JSON. The keys differ depending on the format:

• filter-server-types is used to filter the server types that will be included in the catalog. The default is all server types that are found in the environment where the catalog will be created. Be careful when having a lot of server types, because if no filtering is done a lot of binaries will have to be uploaded to the repository. Only Metalsoft server types are allowed. If invalid values are given, a user will be prompted with what are the available values. An example of this parameter is:

--filter-server-types "M.32.32.2, M.32.64.2, M.40.32.1.v2"

The configuration file passed to raw-config has the following properties for a JSON file:

• name: Name of the catalog

• description: Description of the catalog

• vendor: The vendor of the servers we are working on, can be either Dell, Lenovo, or HP

• catalogUrl: The URL of the online catalog if one is used

• serversList: a list of servers with machineType and serialNumber, to be used only for Lenovo when filtering for specific servers, not just server types

• localCatalogPath: the local path where the catalog will be downloaded. If the file already exists it will be used, instead of downloading a new one, unless the overwrite property is enabled.

• overwriteCatalogs: if set to true, it will overwrite the local catalog found at localCatalogPath with the new downloaded one

• localFirmwarePath: the local path where the binaries will be downloaded if the user chose to. If binaries are downloaded and they already exist, a file hash check is performed to see if the file is the same. If not, they will be overridden.

The YAML equivalent names of the properties are: name, description, vendor, catalog_url, servers_list machine_type, serial_number, local_catalog_path, overwrite_catalogs, local_firmware_path.

Vendor specific details

For Dell and HP vendors, we provide an online catalog URL from where to download it, and the file is saved to the local catalog path (if not present or overriding it). For Lenovo, there are some queries done in order to generate the catalog. For this, we need the machine type and serial numbers. They can be either given in the config file or extracted from the server properties. Lenovo is a special case in which 2 types of filtering can be done, once with the server type filter and another one by machine and serial number. Also, the local catalog path is a folder, as there are as many catalogs as there are machine/serial number combinations.

When calling the create function, errors will be thrown if the parameters are not set correctly or not populated. For example, the name and vendor properties are required for all vendors, but some other properties are optional or not used at all. Catalog URL is not used by Lenovo for instance.

Periodic update

Note that MetalSoft does not update the catalog automatically. An Admin should automate this process if automatic upgrade is required, typically via a CRON job on the server that has internet access.