HOME BLOG

How to use Kubernetes jobs to run short one-off tasks

Posted on: February 25th, 2022 by Olu No Comments

Hi folks,

In this post I discuss how to use jobs to run short one-off tasks in applications that use Kubernetes for container orchestration.

In your application, you may have certain one-off tasks, e.g. pre-populating your database with certain information. How do you go about doing this in an automated fashion?

You can do this using Kubernetes Job.

The idea is to create an idempotent script that can populate the database as needed. Then create a Kubernetes job to run that script. This way, whenever you deploy your application, a job will be created to run the script to populate your database.

Why should the script be idempotent? Well, by making it idempotent, you can extend your job to handle new data, e.g. add more data to use for pre-populating your app database. Then when next the job runs, only the data data will be processed.

An advantage of performing short tasks in a separate task is that if the task fails, it’s easy to tell by looking at the status of the job, as opposed to if you bundle both short tasks with the main application pod. This applies if your main application is something like a web service which is usually a long-running application.

In summary, you can run short one-off tasks using Kubernetes jobs. By making your tasks idempotent, you can run them safely every time you deploy your app. That’s all for now. Till next time, happy software development.

How to build and deploy application container images with Skaffold and Kaniko

Posted on: January 19th, 2022 by Olu No Comments

Hi folks,

In this post I talk about an interesting way to build your application Docker containers and deploy them into your artifact repository e.g. Artifactory. These tools are useful if you use Kubernetes to orchestrate your container deployment.

You can perform builds using a tool called Skaffold. Skaffold can be used in multiple ways. You can perform your build one-time or you can have Skaffold watch your project and automatically build Docker containers when the code changes. Read more about Skaffold here.

Kaniko is a tool to build container images from a Dockerfile, inside a container or Kubernetes cluster. Kaniko doesn’t depend on a Docker daemon and executes within a Dockerfile completely in userspace. This allows us to easily build container images in environments where it’s not convenient to run a Docker daemon e.g. in a standard Kubernetes cluster. Read more about Kaniko here

Using these two tools together you can have your CI server build and deploy app containers to your artifact repo when certain events occur e.g. when code is merged into your development branch.

That’s all for now. Till next time. Happy software dev.

References

1. Skaffold. https://skaffold.dev/
2. Kaniko. https://github.com/GoogleContainerTools/kaniko

OpenShift

Posted on: January 15th, 2022 by Olu No Comments

Hi folks,

In this post I talke about a very interesting I have only come across recently. It’s called OpenShift Container Platform. It’s an on-premise platform-as-a-service built around Linux containers, orchestrated by Kubernetes on a foundation of Red Hat Enterprise Linux. OpenShift is developer by Red Hat.

It’s an amazing tool that makes it easy to manage your applications if you use Kubernetes to orchestrate your app containers.

It provides a nice web interface that allows you view the various Kubernetes resources for your app like services, pods, stateful sets, etc. It also allows you view logs, terminals, events, metrics, etc. for your various Kubernetes pods. This makes it a lot easier to administer your application and means you don’t have to always resort to CLI tools like kubectl to monitor your application resources.

OpenShift also provides a really nice CLI tool called oc that allows you check on your Kubernetes resources from a terminal. oc is a tool that provides a superset of the functionality of kubectl.

The main difference between OpenShift and vanilla Kubernetes is the concept of build-related artifacts. In OpenShift, such artifacts are first-class Kubernetes resources upon which standard Kubernetes operations can apply.

So, if you plan to use Kubernetes to manage app deployment, I recommend giving OpenShift a shot. That’s all for now. Till next time, happy software development.

References

1. OpenShift. https://en.wikipedia.org/wiki/OpenShift

Tips for deploying apps with Kubernetes

Posted on: November 13th, 2021 by Olu No Comments

Hi folks,

In this post i go over a few tips to keep in mind when deploying applications with Kubernetes.

1. After updating a Kubernetes deployment, double-check to make sure the deployment is up-to-date. You can run command

kubectl get deployment

There will be a column UP-TO-DATE.¬† Make sure it is 1. If it isn’t, then check the state of the deployment rollout. If there is any issue with rolling out your deployment, you can scale down and then scale up replicas to update the pods.

2. Make sure your cluster has enough resources (memory, CPU) for your application’s needs. E.g. If your pod requests more memory than the cluster has available, Kubernetes will not allow it run.

That’s all for now. Till next time, happy software development.

How to search and replace unprintable character with Vim

Posted on: October 14th, 2021 by Olu No Comments

Hi folks,

In this post I talk about how to search and replace unprintable character with Vim. I once found myself in a situation where some code wouldn’t run in my file due to an unknown character. It turned out the character wasn’t visible, so I had to find a way to remove it. If you find yourself in a similar situation here’s what to do:

First try to set encoding type and see if that works.

:set encoding=utf-8

If that doesn’t work, you can place the cursor on the unprintable character and press keys ga. This will show the decimal/hexadecimal/octadecimal code for the character. You can then replace it with command:

:%s/\%xYY/yournewchar/g

Or for multibyte character use:

:%s/\%uYYYY/yournewchar/g

See the following for details

:help character-class

 

Reference

How to search and replace an unprintable character. Stackoverflow. https://stackoverflow.com/questions/2798398/how-to-search-and-replace-an-unprintable-character/2801132

How to view Kubernetes resources of a specific API version

Posted on: September 20th, 2021 by Olu No Comments

Hi folks,

Here’s a quick tip on how to view Kubernetes resource of a specific API version. For example, there are different types of Network Policy resources that Kubernetes supports.

There’s the default network policy which has kind NetworkPolicy under apiVersion networking.k8s.io/v1.
Then let’s say there’s another resource of kind NetworkPolicy under apiVersion projectcalico.org/v3.

How do you view all network resources under apiVersion projectcalico.org/v3?

Here’s the command for it:

kubectl get networkpolicy.v3.projectcalico.org

That’s all for now. Happy Software Development.

How to run a one-off job from a Kubernetes Cronjob

Posted on: September 7th, 2021 by Olu No Comments

Hi folks,

In this post I talk about how to manually run a job from a Kubernetes cronjob.

Assume the cronjob is called mycronjob, you can use the following command:

 

kubectl create job --from=crojob/mycronjob name-of-job

That’s all for now. Happy software development.

How to prevent web application from revealing git repository details

Posted on: August 21st, 2021 by Olu No Comments

Hi folks,

In this post I will discuss a security tip. If your web application uses Git for version control, then there would usually be a .git folder in the project’s root folder. Here’s a quick way to prevent users from accessing details under your .git folder from a web browser.

Implement a 404 redirect for any request starting with /.git.

If you use Apache web server and your project users a .htaccess file, you can do this by adding a simple rule to your .htaccess file as follows:

RedirectMatch 404 /\.git

That’s all for now. Happy software development.

How to get process id of last executed background process in Shell script

Posted on: August 10th, 2021 by Olu No Comments

Hi folks,

In this short post I will describe how to get the process id of the last process that was executed in the background.

You do that using the symbol $!

E.g. If you run the commands:

date &
echo $!

you will get an id like 12345 corresponding of the process id for the date command you ran in the background previously.

The process id of the last run background process could be very useful if you want to do things like killing the process when a certain condition is meant. For example, when deploying multicontainer pods in Kubernetes using the sidecar pattern, you may want to terminate the sidecar container once the main container’s process finishes. One way to do this is in the sidecar is to start the sidecar process in the background, fetch its process id, wait for a trigger file which will be written by main container to signal process completion, then kill the sidecar process once the trigger file is found.

See the article How to terminate a side-car container in Kubernetes Job [1] for details.

That’s all for now. Happy software development.

 

Reference

1. How to terminate a side-car container in Kubernetes Job https://cotton-ori.medium.com/how-to-terminate-a-side-car-container-in-kubernetes-job-2468f435ca99

2. Does $! mean something in shell scripting. Stackoverflow. https://stackoverflow.com/questions/18462916/does-mean-something-in-shell-scripting

How to add Search feature to your Dango app with Haystack and Solr

Posted on: August 4th, 2021 by Olu No Comments

Hi folks,

In this post I talk briefly about how you can add search functionality to your Django app. Why add search platforms like Solr, Elasticsearch,e tc. to your app? Because it makes it easier for users to find relevant information, especially when it may be resource-intensive to perform those searches with your database alone. There’s an interesting article on this topic: Solr and RDBMS: Designing your application for the best of both. https://lucidworks.com/post/solr-and-rdbms-the-basics-of-designing-your-application-for-the-best-of-both/. There are several search tools you can use including Solr, Sphinx, etc. let’s assume you want to use Solr.

There’s a modular library you can use called Haystack (https://django-haystack.readthedocs.io/en/master/toc.html). Haystack supports several search tools, including Solr, Elasticsearch, etc.

If you want to use Haystack with Solr, you’ll need to have Solr running first. You can install Solr on different platforms. Haystack config details are found¬† at https://django-haystack.readthedocs.io/en/master/tutorial.html#installation.

Details about installing Solr for use with Haystack can be found here https://django-haystack.readthedocs.io/en/master/installing_search_engines.html.

When you install and start Solr, you would usually want to create something called a core. You perform indexing in that core and perform your searches against the core. Multiple cores can be created on a single Solr instance. When referencing Solr in an application, you would usually use a url like http://127.0.0.1/solr/<yourcore>. Haystack provides management command for interacting building search index and updating search index. You can install Solr using a docker container. See https://hub.docker.com/r/geerlingguy/solr/.

That’s all for now. Till next time, happy software development.