Recover a corrupted instance on the IBM SmartCloud Enterprise

I learned today a very interesting tip from the IBM SmartCloud Enterprise support and I would like to share it with you as I think this could help some of you. This is only applicable to Linux instances.

A customer has corrupted the sudoer file and thus wasn’t able to connect anymore to his instance as root and he has no recent backup of this instance. My first thought was to announce to the customer, with regrets, that we can’t do anything but I was wrong. Here the great trick I learned.

As you know, since release 2.0 we can export images on a volume. The export process copy the root file system on the volume as a directory, once you know that thing become very simple, it will require time and effort to recover your instance but it will be possible.

What you have to do is:

1) Create an image from your instance. This can be done via the portal.

2) Create a persistent storage on the same data-center as your image. To be sure, order a persistent storage a bit bigger than your instance;

3) Find you image in the asset catalog and set the attribute ‘Copy Allowed’ to ‘Y’. To edit the image in the asset catalog you have basically to click on the ‘pencil’ at the top right corner, change the attribute’ and then hit the ‘update’ button. See article:Copy an image on a persistent storage on the IBM SmartCloud Enterprise

4) Export this image using the command-line API “ic-copy-to” on the newly created persistent storage. See this article to know more about command-line APIs and how to setup it

5) Create a new Linux instance (it can be a copper size) and attach the created persistent storage where you copy your image to it and set a mount point (ie:/mnt/myimage). Once the instance is created, your persistent storage should be mounted, if not read the User Guide to mount it manually.

6) Access the instance via ssh, run sudo bash and mount the exported image.

mount the exported image to /mnt/oldimage

7) Now, you can navigate in <mountpoint> (above /mnt/oldimage) and correct what you did wrong on the original instance which prevent it to run correctly.

8 ) Unmount the exported image.

unmount the exported image

9) Delete the Linux instance created at step 5. By the way, once the attach/detach volume will be available on SCE, this procedure can be simplified because if the persistent storage can be attached and detached to/from an existing instance and thus no need to provision/deprovision an instance for each try.

10) Use the command-line API to import the content of the persistent storage as an image in your private catalog (“ic-import-image”).

11) Recreate an instance from this private image.

I guess you will find this tip very interesting and thanks again to IBM SmartCloud Enterprise support.

One thought on “Recover a corrupted instance on the IBM SmartCloud Enterprise

  1. Pingback: IBM SmartCloud Enterprise Unleased III – The Security Issue | Edwin Schouten

Comments are closed.