Shutdown Hours Lab

Determine if an Ocean-GKE Shielded Node workaround is required

To implement shutdown hours in Ocean, we must first consider in the GKE cluster has Shielded Node enabled. To check this, refer to the GKE cluster settings to determine if Shielded Nodes are enabled.

If the Shielded Node configuration is enabled, we must first apply a workaround to ensure the Ocean controller pod can continue to run during off hours.

To check if Shielded Node is enabled, refer to the the GKE cluster configuration page, look under the Security group and find the key “Shielded GKE nodes”.

gke-shielded-node-config

Implementing the Shielded Node shutdown hours workaround

To implement the workaround, consider the following:

Workaround –> Terraform a small, single zone GKE managed node pool that the ocean controller pod can run on when the Ocean managed nodes are shut down.

The following terraform code will:

  • Create a low cost, single node pool of VM type e2-small
  • Deploy the node in a single node pool in a (zonal) availability zone
  • This node pool will be managed exclusively by GKE and will remain running during shutdown hours
resource "google_container_node_pool" "shutdown_node" {
  name       = "ocean-node-for-shutdown-hours"
  location   = var.region
  node_locations = [var.zone]
  cluster    = google_container_cluster.primary.name
  node_count = 1
  project    = var.project_id

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]

    labels = {
      env = var.project_id
    }

    preemptible  = false
    machine_type = "e2-small"
    tags         = ["ocean--shutdown-hours-node", "${var.project_id}-gke"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

Implementing shutdown hours in terraform

Now its time to implement shutdown hours for you GKE cluster. Lucky for you, this step is nothing more than adding a single config element to your existing Ocean-GKE terraform code. In the example below you will see a new config parameter “shutdown_hours” that has been added to the configuration. In this example, we have instructed Ocean to shutdown during non-business hours. The times are defined in GMT format.

module "ocean-gcp-k8s" {
  source     = "spotinst/ocean-gcp-k8s/spotinst"

  cluster_name                      = var.cluster_name
  location                          = var.region
  shutdown_hours = {
    is_enabled = true
    time_windows = [ 
      "Fri:23:30-Mon:13:30", 
      "Mon:23:30-Tue:13:30", 
      "Tue:23:30-Wed:13:30",
      "Wed:23:30-Thu:13:30",
      "Thu:23:30-Fri:13:30",
    ]
  }
}