When researching LLMs, I found these two terms often used interchangeably — distillation and quantization. After some thought, I began to see them like the process of storing a digital image.

When you take a picture of an object, you’re not capturing the object itself but recording a representation — something that can later be projected or reconstructed. You decide how many pixels to keep, and for each pixel, how many bits to allocate. Every choice trades fidelity for practicality.

Distillation is like reducing the pixel count. You retrain a smaller model to reproduce the behavior of a larger one, keeping structure and meaning while dropping fine details. It captures the “shape” of knowledge, not every contour. The result: faster, lighter, and usually good enough for the intended resolution.

Quantization is like lowering the bit depth. The architecture stays the same — same number of layers, parameters, and connections — but each weight or activation is stored with fewer bits. You keep the shape, but reduce the number of shades you can represent. Like playing a color movie on a black and white TV.

Both methods are forms of compression, but they act on different dimensions:

Distillation trims the model’s space — fewer neurons and layers.
Quantization trims the model’s depth — fewer bits per value.

Which to use depends on your target “display.” A small mobile device may need both: fewer pixels and lower bit depth. A server model with room to breathe might only quantize for efficiency.

Like photography, model optimization is about preserving what matters most for the final audience. Every reduction is a decision about what’s worth keeping.

WSL on Windows is great. After using it for some months, I have learned way more about how Windows and Linux systems interact.

One of the things that bothered me was that for some reason my SSH keys on the WSL system never works. On my previous machine running Ubuntu, the SSH Agent would automatically select the correct key to use in my ~/.ssh folder.

Discovery

I stumbled across this project https://github.com/rupor-github/wsl-ssh-agent, and it seemed promising, so I tried it out.

After downloading the binary, I extracted the contents to C:\tools\wsl-ssh-agent. Then from Powershell I ran wsl-ssh-agent-gui.exe -socket c:\tools\wsl-ssh-agent\ssh-agent.sock. From WSL terminal EXPORT SSH_AUTH_SOCK=/c/tools/wsl-ssh-agent/ssh-agent.sock.

Now that the SSH_AGENT is setup, we need to register our ssh keys with the Windows ssh-agent. From Powershell, run ssh-add ${path to ssh private key}.

Now when I run ssh from WSL, it will automatically run through the available ssh keys in my system to log into the remote machine. Win!

Persisting

Now we need a way to persist this setup across reboots.

First is to add wsl-ssh-agent-gui.exe to start up on boot, next is to add the environment variable in WSL to use the correct ssh-agent.

To add to startup, navigate to C:\Users\[User Name]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup. From there, right click and create a new shortcut. Set Target to C:\tools\wsl-ssh-agent\wsl-ssh-agent-gui.exe -socket C:\tools\wsl-ssh-agent\ssh-agent.sock and Start in to C:\tools\wsl-ssh-agent.

In WSL, paste EXPORT SSH_AUTH_SOCK=/c/tools/wsl-ssh-agent/ssh-agent.sock in ~/.profile

Create the shared folder

From virt-manager Add Hardware then choose FileSystem

The source path is the path to the folder on your KVM host

Target path refers to the mount point on your KVM guest

In this case, on the guest I want to mount it to /host

mount -t 9p -o trans=virtio,version=9p2000.L,rw /dev/backup /host

If you try to create any file on /host now, you will get permission denied (even as root!)

You need to set the permission for the guest to write to folders on your host machine

On your host machine

sudo setfacl -m libvirt-qemu:rwx /files/backup/fm

`/files/backup` is not referring to my backup location, it's just a convenient way for me to note that the contents in this folder needs to be backed up.

Now you should be able to write files from your guest machine to the host

To mount the point on boot, add the following to /etc/fstab

/host   /dev/backup    9p  trans=virtio,version=9p2000.L,rw    0   0

*There are definitely security concerns with mounting your host disk on your kvm guest. I'm doing this on my homelab, as I trust what I run and it makes sense for what I am trying to accomplish.

Due to how docker volumes are mounted on Windows, I had to checkout git repositories on Ubuntu WSL to the Windows directory mount, eg. /c/workspace/...

File permissions don't translate well from Linux to Windows, hence, the git files have their permissions "changed" to 755, which is the default mounted permissions. Now when viewing changes from WSL, it is crowded by these "changes", which makes it hard to do git add . without adding all the unchanged files.

The way is to add a git config on WSL to ignore the file permission changes

git config --global core.filemode false

If you need to change permission on a file in the future, eg. add an executable bit

git update-index --chmod=+x 'scriptname.ext'

It takes an extra step to remember, but it doesn't happen too often so I'm okay with it.

Scripts to install docker on Ubuntu (with docker-compose)

curl -fsSL get.docker.com -o get-docker.sh
sh get-docker.sh
rm get-docker.sh

COMPOSE_VERSION=`git ls-remote https://github.com/docker/compose | grep refs/tags | grep -oP "[0-9]+\.[0-9][0-9]+\.[0-9]+$" | tail -n 1`
sudo sh -c "curl -L https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose"
sudo chmod +x /usr/local/bin/docker-compose
sudo sh -c "curl -L https://raw.githubusercontent.com/docker/compose/${COMPOSE_VERSION}/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose"

I first encountered ReactiveX while working on NEM2 blockchain, aka Catapult. On why ReactiveX was chosen rather than Promises, etc, there is a good writeup here. (The original article is no longer available).

Writing Reactive apps requires a change in view in how the app works. The following is one of the most common mistake/frustration I encountered while working with RxJS.

Avoid nested subscribe()

Functions referenced here are from https://github.com/nemtech/nem2-sdk-typescript-javascript

<Observable>.subscribe() should only be used at the end. If you are using nested observables, most likely you are doing it wrong.

One of the most common reason for using nested subscribe is that we want to chain observables yet not care for what the first observable returns.

For example:

Having a list of transaction hashes, I want to be able to determine if the transaction that I announce is successful or not.

To do this, I can use transactionHttp.getTransactionsStatuses(transactionHashes) to retrieve the current status of the transactions on chain.

However, not all transactions are included in the same block, so I would run a while loop to keep checking the blockchain until all transactions have status success or failure.

Resulting in

while (transactionHashes.length > 0) {
	transactionHttp.getTransactionsStatuses(transactionHashes).subscribe(
	(transactionStatusResponses) => {
        for each transactionStatusResponse.hash 
        	if transactionStatusResponse is sucessful remove from transactionHashes
    })
}

Now, it is obvious that we can improve this.

Rather than fetching transaction status in a loop, we can do so only if a new block is harvested.

So our resulting function will look like this

Listener.newBlock().subscribe(
	() => {
            transactionHttp.getTransactionsStatuses(transactionHashes).subscribe(
        (transactionStatusResponses) => {
            for each transactionStatusResponse.hash 
                if transactionStatusResponse is sucessful remove from transactionHashes
      	}
    )
})

Unfortunately there is no way for us to retrieve the transactionStatusResponse. Also the newBlock observable will not know when to exit since it cannot read the status of the inner subscription.

We need a more reactive way to do this.

What we need is to be able to listen to new blocks, then fetch the transactionStatuses within a single subscription.

First we start listening to new blocks

const ob1 = Listener.newBlock()

Now, instead of subscribe, we will use pipe() to keep the reactiveness.

Listener.newBlock().pipe(
	// Do stuff each time newBlock gives us a new block
)

Within pipe() we can specify operators that help us do stuff. However, if we subscribe to Listener.newBlock() right now, it will only return us BlockInfo. To be able to return a different value, we use the switchMap operator

// Within pipe()
switchMap(
	() => {
return transactionHttp.getTransactionsStatuses(transactionHashes)
	}
),

Now, if we do ob1.subscribe() we will get transactionStatus objects.

Using docker to build binaries can cause the docker image to be really large as it includes all the source code, transient build files, linkers, etc. For production it is recommended to us docker multi-stage builds to produce a container that only has the binary and needed libraries to run.

It is easy if the binary produced has all the needed libraries statically linked. But.. what if there are dynamically linked libraries? How do we find them?

ldd

ldd would provide a list of dynamically linked libraries given a binary. However, it doesn't do multiple binaries at once. Hence pipes to the rescue.

find * -type f -perm /a+x -exec ldd {} \; \
| grep so \
| sed -e '/^[^\t]/ d' \
| sed -e 's/\t//' \
| sed -e 's/.*=..//' \
| sed -e 's/ (0.*)//' \
| sort \
| uniq -c \
| sort -n

The following will give us a list of dynamically linked libraries for all binaries found in the current folder.

Now we just have to copy the list of libraries with the binaries to a scratch docker image. #Win

Source: https://stackoverflow.com/a/50218/8827732

I faced an issue with my Ubuntu boxes where the WiFi doesn't come back on when the machine wakes up.

After trying a few suggestions on stackoverflow, I finally found a command that works.

sudo service network-manager restart

And to automate it to work on wake. We follow the steps here

https://askubuntu.com/questions/741620/what-are-the-possible-commands-to-reset-a-wifi-connection

Recently I had to split a 30 thousand line csv file into smaller files of 1 thousand lines. Naturally I had to look for a tool that already does this and I found that this utility split comes installed on most Ubuntu versions (sweet)!

So I removed the first line (which is the header) and went to town.

On first try

split -l 1000 -d 30k-lines-file.csv 1k-lines-file-

Ok.. but it produced the following file names

1k-lines-file-1
1k-lines-file-2
1k-lines-file-3
...

Not good. how do I get back the .csv extension

A little more googling and I found a patch from the split mailing list from 2007! Apparently there is a parameter --additional-suffix

split -l 1000 -d --additional-suffix=.csv 30k-lines-file.csv 1k-lines-file-

Ahh... a little better

Now to add back the header to the first line of each file.

sed -i '1 i header1, header2' 1k-lines-file-*.csv

That's all folks!

Cover photo by Jaxon Lott on Unsplash

Can't seem to find where docker.sock is on your Windows machine?

After trying many docker management GUIs, I finally settled on Portainer. It is very simple to use and the ability to manage more than one docker environment makes it perfect to run small scale testing.

Unfortunately on my Windows machine, I can only run docker using Docker Toolbox, which uses VirtualBox to create a virtual environment for Docker. So where can I bind to docker.sock?

After days of unsuccessfully scouring the internet for a solution, I finally found it! Apparently all you need is to double backslash it.

So using the Portainer example from https://www.portainer.io/installation/

$ docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

becomes

$ docker run -d -p 9000:9000 -v //var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

Easy

If you are running low on RAM on your system, you might want to consider adding some swap space. The following will allocate and assign a 2GB swap space.

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon --show

It is possible to assign a second swap space. Just repeat the above commands but replace /swapfile with another name, eg. swapfile2

Removing swap space

sudo swapoff /swapfile
sudo rm /swapfile

To make permanent

Add the following line to /etc/fstab

/swapfile swap swap defaults 0 0

Most UNIX-like operating systems, including Linux and macOS, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis. These “ulimits” prevent single users from using too many system resources. (https://docs.mongodb.com/manual/reference/ulimit/)

How to increase ulimit

# vi /etc/sysctl.conf
fs.file-max = 500000
#sysctl -p

# vi /etc/security/limits.conf
* soft nofile 60000
* hard nofile 60000

Note: '*' would apply the limit to all users on your system (except for root). If you only want the limit to apply for a single user or for the root user, you would need to replace * with the user.

reboot

Resource

https://glassonionblog.wordpress.com/2013/01/27/increase-ulimit-and-file-descriptors-limit/

A simple way to set redirect to https for any site on your nginx/apache configuration.

Apache

RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R=301,L]

Nginx

server {
	listen 80;
	server_name ${DOMAIN};
	return 301 https://$server_name$request_uri;
}

When testing distributed systems, sometimes it is necessary to find out the network tolerance before production deployment.

Network simulation in essence involves 3 parts

1. Latency
2. Bandwidth
3. Packet loss

Setup

The following assumes you have two nodes set up

1. server
2. client

Latency

Before starting any tests, it is good to get a benchmark of your current setup.

From your client node run

ping <server-ip-address>

Now let's add a 500ms latency

# tc qdisc add dev eth0 root netem delay 500ms

If you run ping again, you will get a different result

Let's say we want to add some variance to the latency

Here we add a 10ms variance along a normal distribution in addition to the 500ms latency

Notice the use of  change instead of add

tc qdisc add dev eth0 root netem delay 500ms 10ms distribution normal

Packet loss

The following line adds a 10% packet loss in addition to a 250ms delay

tc qdisc change dev eth0 root netem loss 10% delay 250ms

Note: These changes do not persist across restarts

Bandwidth

Using Wondershaper

https://github.com/magnific0/wondershaper

For bandwidth control, you can use wondershaper

After installation you can run the following

wondershaper -a eth0 -u 30000 -d 10000
wondershaper -c -a eth0

Test bandwidth

Use iperf to test if bandwidth limitations are applied

On the server run iperf -s to start the server

On the second node run iperf -c <server-ip-address>