Page tree
Skip to end of metadata
Go to start of metadata

Background

This document describes how to build an image that can be deployed to your platform, which allows you to create servers based on the image. For information about deploying an image to your platform, see Creating an Image

This documentation is meant to be a guide for technically astute readers. None of the processes described here require anything other than normal customer access. No programming is required; whilst it is possible to automate the tasks described here with the API, this is beyond the scope of this article. You will, however, need experience with basic Linux system administration, and familiarity with block devices and file systems in particular.

Images are essentially byte for byte copies of a disk. They are deployed onto new servers by thin provisioning. They differ from snapshots only in that they do not retain a reference to the disk from which they were created, and in that they are included in the list presented to the user of images on which servers can be based.

Customers can create private images, and then share them with other customers, a specific billing entity, or the entire platform (subject to the configuration dictated by the platform licensee). The licensee can associate them with a product offer that will generate appropriate billing if required.

Two steps are thus required to create a new public image:

  1. Creation of a functional private image.
  2. Publishing of that private image into a shared/public image.

The second of these steps is detailed under the Managing an Image page,  so this document focuses on the first of these steps.

There are two main ways to build an image:

  • Configuration and snapshotting of a server or disk.
  • Off-server image creation.

These are alternative processes. The first method is the easiest and the most user-friendly. The second is the most flexible.

Configuration and Snapshotting of a server or disk

This is the simplest method. The steps are as follows:

  1. Create a server using an existing operating system image as supplied on your platform. You will have to abide by the licensing terms of this operating system.
  2. Install your own custom components.

    If you are using VMware as your hypervisor, we recommend that these components include VMware Tools, as VMware requires these components in order to shut virtual machines down safely. For instructions on how to install VMware tools, see http://www.vmware.com/support/ws55/doc/new_guest_tools_ws.html.

  3. Test your server, and make any required adjustments.
  4. Stop your server.
  5. Using the image widget, make an image of the server’s primary disk, or for VMware or Hyper-V, the server itself.
  6. Create a new server, based on this image.
  7. Test this; if it is unsuccessful, delete the server created in step 6 and the image, and repeat from step 3.

In order for your image to present well to end-users, you will have to take some care to ensure that inappropriate data is not included within it. This is especially relevant for operating system images (rather than appliance images). This means that you should ensure that:

  1. Your own test scripts are erased.
  2. No network-accessible logins are available (so, on a UNIX platform, there are no entries in /etc/passwd or /etc/shadow with fixed passwords).
  3. Logs and usage information are cleared (consider /var/logutmpwtmp, and dmesg files).
  4. Information which is supposed to be unique per server is regenerated (for instance ssh keys).

The easiest way to ensure these happen is by a “first boot” script. This can also set a unique initial password for one user (see section 4). As an alternative, it is possible to modify the disk directly. To do this:

  1. Stop your test server.
  2. Create a new server (call this “utility server”).
  3. Detach the disk from your test server.
  4. Attach the disk to your utility server.
  5. Start the utility server.
  6. Mount the new disk (e.g. ‘mount /dev/sdb /mnt’).
  7. Make the appropriate modifications to the mounted disk.
  8. Unmount the new disk (e.g. ‘umount /dev/sdb’).
  9. Stop the utility server.
  10. Detach the disk from the utility server.
  11. Reattach the disk to the test server (If you used the above workaround, also configure it as the primary disk).
  12. Create the image as described above.

Off-Server image creation

Flexiant Cloud Orchestrator will boot any disk that a normal x86 server will boot, provided it will run as a virtual machine (Xen/KVM/VMWare subject to the choice of hypervisor by the licensee). It is thus possible to create images on a separate server either manually, or using packages such as vmbuilder. You can even use a virtual machine running on the platform to do this, which makes transfer for images faster.

This process consists of two parts:

  • Making the binary disk image.
  • Transferring it into your platform.

Making the binary disk image

You can use any method you like to make a binary disk image (even a ‘dd’ from a server on your desk), but we suggest you use our tool-chain. This currently consists of two tools:sparsecopy and fsmaker. These tools are optimised for and only tested on a Linux platform, though sparsecopy should work on other platforms. However, this does not restrict your choice of operating system within your image; for instance, there is nothing to stop you using this to create an OpenBSD based image.

In short, what these do is as follows:

  • sparsecopy is a block copy utility rather like dd, which works with sparse files and does intelligent things with partition tables. Sparse file support means it works more quickly, uses less disk space for the resultant images, and is more compatible with thin provisioning.
  • fsmaker takes a filing system root partition that does not necessarily have a kernel or a boot sector (such as you might use in Amazon's EC-2 service, or on Eucalyptus), and makes it into a bootable hard disk image by adding grub and a optionally a kernel and an initrd.

Note that fsmaker utility is not supported by Flexiant.

You will need sfdisk and getext2fs installed to use the fsmaker utility (both of these are available in Ubuntu and Debian stock distributions, and may be available elsewhere).fsmaker also requires that sparsecopy is installed.

To compile and install the tool-chain from scratch, download sparsecopy and fsmaker source code from here. Expand each tar archive into somewhere convenient, and type ‘make’ and (as root) ‘make install’. Alternatively, on a  debian or ubuntu amd64 system, download the precompiled .deb files from the same link.

Both utilities will give appropriate usage instructions if invoked with no parameters.

To use fsmaker, you will first require a copy of your file-system (to be installed in the first partition) of your image. You could create a 2GB file-system, for instance, by:

sparsecopy -f2G /dev/null myfs
mkfs -t ext3 myfs
mount -o loop myfs /mnt
rsync -axSHAXD /my/files/to/copy /mnt
umount -d /mnt

Alternatively, you could use a utility such as vmbuilder to perform this operation, or you could use the img file from an existing AMI image (you may wish to modify it to removeec2init and ec2init-userdata which are Amazon specific items).

Then use fsmaker to build your image. In its simplest form this is:

fsmaker myfs myimage.img

fsmaker will use grub, a recent known good kernel and initrd to boot your image, though on first boot you may wish to replace these with kernels inside your file-system. It is possible to specify your own kernel, ram-disk, and even grub installation through command-line options.

Transferring your image to the platform

You can upload images to your platform using the Create Disk or Create Image buttons within the Disk and Image widgets. The only difference between these two buttons is that the former makes a disk and the latter makes an image. It is possible to make a disk/server into an image by using the “New Image” button on the Images widget. An image can be made into a disk either by provisioning a new server using that image, or by cloning it.

Transferring your image to the platform using the API.

Exactly the same functionality as set out above is available through the API. Please refer to the API call FetchDisk.

For more details, please refer to the API documentation.

Transferring your image to the platform manually

For completeness, images can also be uploaded manually. This is occasionally useful for debugging, or when developing on the platform itself. To achieve this:

  1. Create a utility server (i.e. a Linux server of any sort).
  2. Create a new 20GB blank disk.
  3. Attach the blank disk to the server.
  4. Boot the server.
  5. If you have not done so already, install the tool-chain on the server (as above).
  6. Compress your image (e.g. ‘gzip myimage.img’).
  7. Transfer your compressed image (myimage.img.gz in the above example) to your server, using normal UNIX tools (e.g. scp, rsync –e ssh, wget, etc.).
  8. Uncompress the image directly onto the second hard disk (e.g. 'gzcat myimage.img.gz | sparsecopy - /dev/sdb').
  9. Reread the partition table (e.g. 'sfdisk –R /dev/sdb'.
  10. Stop the server.
  11. Create an image of the second disk.

First password creation

As any server created from a given image on Flexiant Cloud Orchestrator will have an identical disk configuration, there is an inherent risk to storing passwords (encrypted or otherwise) in a public image, as the password will be the same for all users. Unless the image is appropriately locked down (so, for instance, passwords can only be entered through the console interface) this would present a security risk as any person familiar with the passwords for that image could log into a freshly deployed server using that image via its network interface; this is clearly undesirable.

To avoid this problem, the Flexiant Cloud Orchestrator platform generates a unique initial password that a ‘first boot’ script can obtain and assign as the password to any user on the system. Example scripts are provided as a 'Firstboot' tarball and debian package here

These simple scripts can be installed using either of the above two methods. These scripts work by using the Metadata service to retrieve the password information, amongst other details that are available.

ISO support

The platform can also support disks in ISO format (subject to hypervisor support). This is particularly useful in order to upload ISO images of install CDs or DVDs. The platform will recognise an ISO format disk as such, and mounts it as a virtual CD drive rather than a virtual hard disk. Remaining disks will be mounted as hard disks in disk order. Boot order is set to CD then hard disk.

  • No labels