K3S MultiCloud

| Ref : https://zenn.dev/yude/scraps/32c811301f9ee1

設定 config

兩個版本都順便寫進 tailscale 安裝需求

Proxmox 版本

features: mount=fuse,nesting=1
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
lxc.apparmor.profile: unconfined
lxc.cap.drop:
lxc.cgroup.devices.allow: a
lxc.mount.auto: proc:rw sys:rw
lxc.mount.entry: /dev/fuse dev/fuse none bind,create=file 0 0

Lxd Profile

| ref : https://radar231.com/posts/k3s-nodes-in-lxd-containers/

config:
    raw.lxc: |
        lxc.cgroup2.devices.allow= c 10:200 rwm
        lxc.mount.entry= /dev/net/tun dev/net/tun none bind,create=file
        lxc.apparmor.profile= unconfined
        lxc.cap.drop=
        lxc.cgroup.devices.allow= a
        lxc.mount.auto= proc:rw sys:rw
        lxc.mount.entry= /dev/fuse dev/fuse none bind,create=file 0 0\
    limits.memory.swap: "false"
    linux.kernel_modules: overlay,nf_nat,ip_tables,ip6_tables,netlink_diag,br_netfilter,xt_conntrack,nf_conntrack,ip_vs,vxlan
    security.nesting: "true"
    security.privileged: "true"
description: ""
devices:
  kmsg:
    path: /dev/kmsg
    source: /dev/kmsg
    type: unix-char
  root:
    path: /
    pool: k3s
    type: disk
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
name: k3s
used_by: []

設定 kmsg

| 只對 proxmox 做就好 lxd 的版本我們放進 profile 了

#!/bin/sh -e

# Kubeadm 1.15 needs /dev/kmsg to be there, but it's not in lxc, but we can just use /dev/console instead
# see: https://github.com/kubernetes-sigs/kind/issues/662
if [ ! -e /dev/kmsg ]; then
    ln -s /dev/console /dev/kmsg
fi

# https://medium.com/@kvaps/run-kubernetes-in-lxc-container-f04aa94b6c9c
mount --make-rshared /

chmod +x /etc/rc.local
reboot

Tailscale 安裝

curl -fsSL https://tailscale.com/install.sh | sh

sudo tailscale up --auth-key=<key>

k3s 安裝

第一台

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server \
    --cluster-init \
    --advertise-address <tailscale-ip> \
    --node-ip <tailscale-ip> \
    --node-external-ip <tailscale-ip>" \
    sh -s

第二台 多 master

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server \
    --server https://<第一台 k3s IP>:6443 \
    --token \
    <token> \
    --advertise-address <tailscale-ip> \
    --node-ip <tailscale-ip>  \
    --node-external-ip <tailscale-ip> " \
    sh -s

worker

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent \
    --server https://<第一台 k3s IP>:6443 \
    --token \
    <token> \
    --node-ip <tailscale-ip> \
    --node-external-ip <tailscale-ip>" \
    sh -s