My Linux Stories

Setting up k8's to use nvidia Setting up k8's to use nvidia Prerequisites Quick Start Preparing your GPU Nodes Enabling GPU Support in Kubernetes Checks Sample yaml file References Destroy Prerequisites The list of prerequisites for running the NVIDIA device plugin is described below: NVIDIA drivers ~= 410.48 nvidia-docker version > 2.0 (see how to install and it's prerequisites) docker configured with nvidia as the default runtime . Kubernetes version >= 1.10 kubeadm installation . Post installation use flannel as the network plugin - $ sudo kubeadm init --pod-network-cidr=10.244.0.0/16 - $ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml - $ kubectl taint nodes --all node-role.kubernetes.io/master- Quick Start Preparing your GPU Nodes The following steps need to be executed on all your GPU nodes. This README assumes that the NVIDIA drivers

Issue Have you ever felt that write to a partition has become relatively slower than previously?? Ever wondered how to debug such kind of issues Then this post is for you. Useful commands are lsblk iotop iostat dd Check the partition type using  df -Th Debugging dd command is used to emulate disk write to the device.  In this example we are just writing the stream of  /dev/zero of size 1G to our partition. A key observation was that the real time was way higher than sys and user time combined. for i in {0..60}; do time dd if=/dev/zero of=/data/1GB.bin bs=1G count=1; done output real 31m27s user 00m00s sys 00m806s dd process was observed to be in uninterruptible sleep (D) state using the following command for the most part of the time as the above command was under execution. iotop (run as a root user) showed that there was no disk activity on any of the disks on the device. Few takeaways: /dev/zero produces a continuous stream of NULL (

Introduction If you have ever installed Mongo 3.* version and worrying as to how to upgrade it to Mongo 4.x. Then this story is for you, Mongo has made the upgrades comparatively easy. This story is a brief overview as to how to upgrade Mongo and it is divided into three segments Identification While Migrating Databases from one node to another using mongodump --host sourcehost:27017 --db client_c_n --archive | mongorestore --host targethost:27017 --archive We landed into Unrecognized field 'snapshot'. mongo this error. Debugging We saw that there is a version mismatch as in source host being 4.2.x and target being 3.6.17 between both the mongo servers. Now we have to take an immediate step to upgrade mongo and maintain the same version of mongo across the nodes. So that the dumps can be transferred between nodes. Solution Please note that you can update mongo DB only on Stepwise. For instance, To upgrade from a version earlier than the 3.6-series, yo