Patching ExaCS clusters using OCI-CLI

It's important to keep your ExaCS clusters up to date. In this article, colleague Mark Kempers, Infrastructure Specialist at MCX, explains some basic steps.

"In my case I made an interactive script so I can do the whole patch cycle in my terminal without clicking through the console. It lets me choose the environment, which update I would like to precheck/apply and provides me the status of the patch.

This can become helpful if not every employee can access the OCI console, but needs to patch the system. For example your DBA or Service Desk Engineer can do it without any tools. Just run the script!

So, let's take a dive into the scripting.

Right now, I will focus on fetching the GRID/OS patches for one VM cluster of the ExaCS infrastructure.

Let's start with the prerequisites.

Make sure you have a Linux machine installed with the oci-cli and jq. Refer to this piece of documentation on how in to install and configure the oci-cli.

In this case I have an ExaCS infrastructure with 4 different clusters. One for each DTAP (Development, Test, Acceptance and Production) environment. The goal is to keep all environments the same.

Building the script

Make sure you start with declaring all variables you need. In this case we need the OCIDs for all ExaCS VM clusters and today's date in UTC. Which will look something like this:

We will use this variables later in the scripting.

The next step will be to fetch all the patches available for a cluster. Of course we want a friendly formatted output. As oci-cli gives the output in JSON, we will format it to a readable table. Here is an example:

When running this, the result will be a table in your terminal with the OCID of the patch, description, version, release time and available action (ROLLING-APPLY/PRECHECK).

Now it is important that you save the OCID of the specific patch you would like to install so we can kick off the precheck for that patch. Here is how to start the precheck.

So now we would like to know when the precheck is finished. As soon as the precheck starts we run a script to ask for the current status of the precheck. Every job that is running will already be stored in the history, but with a specific lifecycle state. The three most important lifecycle-states are FAILED, IN PROGRESS or SUCCEEDED. So let's see how that will look like.

The script below will ask for the status of that specific job every 60 seconds for 60 minutes. It will keep going for the time the status is 'IN PROGRESS'. It will finish as soon as it reach the status FAILED or SUCCEEDED. When reaching FAILED it will exit the script, when SUCCEEDED it will continue to apply the patch.

Assuming the patch precheck succeeds, we will continue to apply the patch. The command looks similar to the PRECHECK command. Let's take a look.

As soon as the patch APPLY runs, you can run the same check loop as mentioned above. Instead of filtering the output on PRECHECK, filter on the APPLY. The full command looks like this. Can you see the difference?

When the patch is installed successfully, it will tell you so. That's all it is! With some bash scripting skills, you can combine everything together and patch your environment with just one single command!

As you can see, you can change a variable to apply this to all of your clusters.

If you like some details on the full script I built for the patching cycles, don't hesitate to reach out! Happy to help."