When installing Rancher’s K3s on a “containerd”-based platform (so not using Docker to run your containers), it might not be as obvious as one expects to gain root access inside the containers.
But why might you need this at all? Typically, in a Kubernetes environment you’ll not be in touch with the containers directly, and that’s a good thing. But sometimes you might need to diagnose some problem, or things didn’t work out as expected (like having wrong permissions on a PV, so that the de-privileged software in the container cannot write to that storage). And sometimes, even if you can start a shell inside the container using i. e. “kubectl exec -ti <podname> — sh” (or similar, depending on what shell is available in the specific container), you may find yourself running the shell with a non-root user and no way to up your privileges (no “su” binary, no root password, or similar). Continue reading
Posted in Linux
Tagged containerd, K3s
Now that the Corona Pandemic is taking its second (or is it its third?) swing, online collaboration seems almost like “state of the art” and many of us are used to using it, one way or another.
Despite of having many Opensource options, a number of commercial solutions seem to dominate the market, Microsoft Teams being one of them. And on top of the browser-based access path, Microsoft even offers a native Linux client! That’s great, Microsoft now belongs to the good guys. Or don’t they? Well, it certainly depends on your point of view. And what you may notice: Microsoft Teams on Linux is not what Microsoft Teams is on Windows. The net is full of user feedback i. e. asking for remote control via Linux desktop Teams client, there even is a MS user feedback forum you get redirected to by Microsoft employees:
“It recommends you give a feedback to https://microsoftteams.uservoice.com/forums/555103-public/suggestions/39624940-remote-control-on-linux.
Microsoft will always focus on customer’s feedback and experience. Some new features would be added to the services based on customers’ feedback in the future, and your good ideas will be very helpful for them to improve the service.”
Said forum thread was started February 4, 2020, but up to today no Microsoft feedback was received, nor was that functionality added to the software. But wait: Maybe the feature already is available?
This is what “Murphy’s Law” is all about: You change a minor issue and spend the day fixing big trouble.
In this specific case, I had to reboot one of our more important servers because of problems with persistent network names. But after the reboot, the machine didn’t come up as expected (or should I rather say “hoped for”?), but instead went into Dracut rescue mode, AKA “initrd shell”.
We quickly noticed that only some of the LVM volume groups were activated and noticed that the volume group carrying the system LVs was unavailable. “lvm pvscan” not only gave a too short list of PVs (the two disks for system VG missing), but also spat out a number of error messages: That it wouldn’t activate our system volume group, that it could not work with a standalone PV, and “read-only locking type set. write locks are prohibited”. Now what’s that about? Continue reading