Yet another coding blog

Want to help test the merge? Run a node on Kintsugi

Want to help test the merge? Run a node on Kintsugi

#TestingTheMerge

Luis Naranjo's photo
Luis Naranjo
·

14 min read

Why bother?

If you want to help test the merge:

  • This article will show you how to run your own node on the Kintsugi merge testnet.
  • This is just one of the ways that you can help test the merge
  • If you're curious about other ways you can help, Marius van der Wijden has a great post on the subject outlining needed contribution areas for people with different backgrounds.

Recap

Ethereum is planning to transition from proof-of-work to proof-of-stake this year. This event is being referred to as the merge, and it's a big deal to the Ethereum community. It's been a long time coming.

But before that happens, the Ethereum community has a chance to test the merge using various testnets. If you're not sure what that is, all you need to know is that it is a non-production environment where we can simulate the merge and identify issues ahead of time. This merge testnet activity is super valuable because any issues that we can proactively find and fix now will go a long way to making the merge successful on mainnet (which is a major engineering feat).

You can help by running a node

By the way, Kintsugi is the name of the public testnet whose sole reason for existing is to test the merge. Anyone (including you) is able to run their own node on the Kintsugi testnet and help test the merge!

If you care about Ethereum and you want to contribute your time to help test the merge you're in luck. There are tons of ways you can help, and there is room for all sorts of people (non technical, technical, and super technical).

This article will focus on just one of those ways (running a node on the Kintsugi testnet). In my opinion, this is one of the most accessible ways for everyday Ethereum users to help test the merge. The goal of this blog post is to be a guide for anyone who wants to do it.

How is running a node helpful?

  • It might not seem like it's doing anything at first glance, but your node will be talking to other nodes, and checking their work, and calling out errors, etc.... This is useful to the client dev teams because you add traffic to the network and more data points = more confidence
  • You might decide to provide feedback to the dev teams on the installation/configuration experience, if you discover any UX friction points for the clients you are using
  • If you run into any bugs, please report them to the appropriate dev team.

I asked Remy Roy to help me quantify the impact of running a Kintsugi node (on a scale of 1-10), and he shared this, which I found useful

Running a node on Kintsugi: 8 Running a validator on Kintsugi: 8 Getting others to run a validator on Kintsigu: 9.5

Probably no one is going to say thank you, to you personally, for spending your time running a node on Kintsugi. But you will get the satisfaction of knowing that you contributed value by testing the merge (even if you don't find or report any issues).

This can also be a useful learning opportunity for you! If you're curious about operating a node on mainnet or even turning that mainnet node into a validator to stake your ETH, this could be a cool win/win. You get to help Ethereum test the merge, and in return, you get some hands on experience learning what it's like to operate a node.

Disclaimer about difficulty

This blog post will require you to have some technical knowledge (know how to navigate the command line, conceptualize how the tech stack is working, etc...).

It's ok to follow this guide even if you don't fully understand what is going on, as long as you understand that this is more of a learning exercise for you (and hopefully a helpful one). Everyone has to start somewhere. Don't let the technical jargon dissuade you from learning something new.

Disclaimer about security

Following this guide will lead you to open ports on your router and on your host machine.

This is a relatively dangerous thing to do (exposing ports to the internet), so if you're not comfortable doing that or don't understand the risks, please stop.

This is especially pertinent if you are planning to do this on your home network, where you live and maybe even work.

If you are in a situation where you are not comfortable doing that, but want to help anyways, you might consider spinning up a cloud VM and doing this there to avoid changing your home network security profile.

Additionally, you will end up installing some software (Java, Besu, and Teku). As with installing any new software, there are risks of supply chain vulnerabilities. Fortunately, because Besu and Teku share the same tech stack and are developed by the same company, you get to reduce your risk profile a little bit.

Ok, let's do it

First, let's set expectations for what the high level steps are so we can keep break it down step by step

  1. Acquire hardware that can run the node
  2. Install and configure pre-requisite software
  3. Install and configure a execution client (we will be using Besu for this write-up)
  4. Install and configure a consensus client (we will be using Teku for this write-up)
  5. Wait for both clients to finish syncing

That's it! Let's go through it step by step.

We will be using Besu as our execution client, and Teku as our consensus client. I am picking these clients for a few reasons

  1. They are both minority clients, so we can promote the client diversity by adopting them
  2. They are both compiled with Java, so we can build both projects from source easily
  3. They are both produced and maintained by the same company, so they have some synergy and similarities in usage patterns.

Step 1: Acquire hardware that can run the node

This first step is likely to be the one with the highest barrier to entry, because it requires access to physical hardware. And if you don't have access to that, or the resources to acquire access, you might be out of luck :(

The software requirements for running a node are pretty low, actually (by design). But still, you need something to run the software.

Normally, for running on mainnet or even a long running testnet, you would want to prefer hardware that you actually have at home over a cloud VM (for better decentralization). But for the sake of a short lived testnet, a VM should be totally fine if that's your preference.

You can technically run on Mac or Windows, but I wouldn't recommend it. I am going to assume your node is running Linux.

Step 2: Install and configure pre-requisite software

This step will involve a few house keeping things that we're just going to quickly get out of the way

Open ports

First, you will need to open some ports on your machine so that our Besu and Teku clients can operate freely. I won't be covering this, because there are many guides out there that can do a better job of explaining than me. Here is one for example.

If you are running this on a machine at home behind a router, you will likely need to configure your router so that it forwards traffic on that external port to your machine in your LAN. You will need to do this for each port you open. You can use canyouseeme.org to check this once you think you've configured it correctly.

Using port 30303 for example, internet traffic flow should look like this conceptually

  1. INTERNET (external request)
  2. Your router, on your public WAN IP assigned by your ISP (port 30303)
  3. Forwarded to your machine on your LAN, probably 192.168.1.0/255 (port 30303)
  4. Besu receives request because it listens to port 30303 by default).

Open the following ports (TCP+UDP)

  • 30303 (default Besu port used for peer discovery)
  • 9000 (default Teku port used for peer discovery)

For more information about ports, check out the documentation for Besu and Teku.

If you have advanced use cases, you may wish to open other ports such as 8545 if you wish to allow public access to the HTTP RPC API for the execution layer. But for the sake of keeping things simple, you don't need to do that right now (as long as Teku can access that port internally so it can talk to Besu).

If you are on a home router behind a NAT, you may wish to take a look at this section of the Teku documentation for improving Teku's peering performance. The equivalent documentation for Besu is here.

Create linux users

We are going to create a linux user for each client. One called "besu", and another called "teku". This is just a good security practice so that we can sandbox file permissions

sudo useradd --no-create-home --shell /bin/false besu
sudo useradd --no-create-home --shell /bin/false teku

Create data directories

Each client will need a data directory, so go ahead and create one wherever you want. Make sure it has plenty of storage capacity (ideally 2 TB). For my example, I will be putting my folders in /datadrive, which is where I have gone ahead and mounted a 2 TB SSD.

mkdir /datadrive/besu
mkdir /datadrive/teku

Now that we have folders, let's restrict ownership to their respective Linux groups. This will ensure that only the Besu user can edit files in the besu folder, and vice versa.

sudo chown -R besu:besu  /datadrive/besu
sudo chown -R teku:teku  /datadrive/teku

Install Java

Now we are going to install Java 11 (or higher). We will use this later to compile Besu and Teku from source. This is convenient because when we need to install updates, we can just pull down the latest source with git and recompile easily.

sudo apt install default-jre default-jdk

Clone testnet config

Later on, we are going to need access to the Kintsugi testnet config. Go ahead and clone that repository somewhere you won't forget. I am putting it in my home directory.

git clone https://github.com/eth-clients/merge-testnets.git

Step 3: Installing and configuring Besu

Now we are going to get Besu (our execution client) all set up.

First, clone the source code and check out the "merge" branch (important!)

git clone -b merge https://github.com/hyperledger/besu --recursive
cd besu

Next, compile it. This may take at least a few minutes.

./gradlew installDist distTar

Once compilation has succeeded, we will copy the binaries to a folder that is on our $PATH

sudo cp -a build/install/besu/. /usr/local/bin/besu
cd ..

Now, we are going to create a systemd file. This will allow us to start a systemd service that will help us manage the lifecycle of our client. It's also where we will supply custom command line arguments

sudo nano /etc/systemd/system/besu.service

Then dump this fun wall of text in there. You will need to update the parameters in ExecStart to match your paths. 

[Unit]
Description=Ethereum Besu execution client
After=network.target
Before=network.target

[Service]
User=besu
Group=besu
Type=simple
Restart=always
RestartSec=5
Environment="JAVA_OPTS=-Xmx4g"
ExecStart=/usr/local/bin/besu/bin/besu \
    --data-path="<path to your besu data dir>" \
    --genesis-file="/home/<username>/merge-testnets/kintsugi/besu_genesis.json" \
    --rpc-http-enabled \
    --rpc-http-api=ADMIN,MINER,ETH,NET,DEBUG,TXPOOL,EXECUTION \
    --network-id=1337702 \
    --p2p-enabled=true \
    --Xmerge-support=true \
    --miner-coinbase=0x0000000000000000000000000000000000000000
    --bootnodes=enode://6f377dd1ef5a3272d7e02fac9064c4f95d74f7edfd866e59ded774ee5b4649ff61c3f24c95f5c3d07d692b447f0569716b8921b6861810b96a705c92e1d27ff9@161.35.67.219:30303,enode://08e5308bca664503e023e10ff39795041a5d49d0384a071554b155af1e82e99afc487c3ec25e41b8910f633039b193902d41ecd7ae42ca031929558eb8f94222@161.35.79.127:30303,enode://b7abf51b9496011789a59cf3ea6df653a4a2aadec9e152b81fbfbf704245ad624e99960c11ad38a6c5b30bb5791d7431477b375be806c24bcee4886124b54c05@64.225.4.226:30303,enode://371ef6897c38b451afd1864f1c3e6697d39ec9d509803a540195e873816c441cc82415a1f01c2c98441d0cbad06b2958e2aaa3c23b27304841f1492a04a6d47b@165.232.177.121:30303,enode://c20047e975f562131d0211b1a36f955b821663bd83a69edd725b221b70db0d01320716dd6a45d87e4e8afc1bc53439ff001212a70be0f1064db65c82d7ebbc9d@161.35.67.221:30303,enode://11b10968e46d30e3013b376e941eed90a3472ddebef0df004fa9dc20644e6b9fddb7ec1fe5aa06c57650e2752115e9f93c0049e4618b9f811acffb1ca7e402ec@161.35.75.78:30303,enode://57745805245c441b71a9f3b3e7d78f75dd576d36b236b9f64cf9a9cccdcb574ec1f64d69c05add598ef26e3d7f646534b4c9976ca53551f71ad579a472635086@165.232.185.207:30303

[Install]
WantedBy=multi-user.target

If you want to learn more about any of these command line options, definitely check out the official Besu Options documentation.

Then run these commands

sudo systemctl daemon-reload
sudo systemctl start besu
sudo systemctl status besu
sudo systemctl enable besu
sudo journalctl -fu besu.service

That last command should show you the logs of your service. Hopefully everything looks good and there are no errors! Ideally, you should be able to watch Besu start syncing blocks (which will take a while).

Step 4: Installing and configuring Teku

Now we are going to get Teku (our consensus client) all set up.

First, clone the source code

git clone https://github.com/ConsenSys/teku.git
cd teku

Next, compile it. This may take at least a few minutes.

./gradlew installDist

Once compilation has succeeded, we will copy the binaries to a folder that is on our $PATH

sudo cp -a build/install/teku/. /usr/local/bin/teku
cd ..

Now, we are going to create a systemd file. Same thing as list time.

sudo nano /etc/systemd/system/teku.service

Then dump this fun wall of text in there. You will need to update the parameters in ExecStart to match your paths. 

[Unit]
Description=Ethereum Teku consensus client
After=network.target
Before=network.target

[Service]
User=teku
Group=teku
Type=simple
Restart=always
RestartSec=5
Environment="JAVA_OPTS=-Xmx4g"
ExecStart=/usr/local/bin/teku/bin/teku \
    --data-path="<path to teku data dir>" \
    --network=kintsugi \
    --Xee-endpoint=http://localhost:8545 \
    --log-destination=console

[Install]
WantedBy=multi-user.target

If you want to learn more about any of these command line options, definitely check out the official Teku Options documentation.

Then run these commands

sudo systemctl daemon-reload
sudo systemctl start teku
sudo systemctl status teku
sudo systemctl enable teku
sudo journalctl -fu teku.service

Same drill as before. Hopefully everything looks good and there are no errors! Ideally, you should be able to watch Teku start syncing blocks (which will take a while).

Wait for both clients to finish syncing

This may take a while. You might be wondering - "how can I tell when each client is done syncing?".

You can use a similar strategy to check the sync status for your execution client and consensus client. You can ask each client - "what is the current block height?". If the block heigh closely matches the block height you see on the blockchain explorer, then you know you are up to date. If your block height is a lot smaller, you know you need to wait for your client to catch up.

This is because each client needs to sync to the very latest head block (height = N) of the blockchain, starting from the genesis block (height = 0). This takes time, and it's not going to text you when it's done ;)

Besu

If you remember, we specified the --rpc-http-enabled flag when we ran Besu. This turns on the RPC HTTP server at the default port (8545). This means you can make HTTP requests to talk to Besu, and ask it for the current block height

curl -X POST --data '{"jsonrpc":"2.0","id":"1","method":"eth_blockNumber","params":[]}' http://127.0.0.1:8545

The result should look like this

{
  "jsonrpc" : "2.0",
  "id" : "1",
  "result" : "0x26c89"
}

The value in the "result" field is the current block height in hexadecimal format (base 16). For this particular value, you can easily convert it to base 10 with a calculator app on the programmer view (both Windows and Mac support this).

To get the current block height, you can take a look at the Kintsugi Ethereum Explorer.

Additionally, there is a "eth_syncing" method you can call which seems even more straightforward (yes or no result).

curl -X POST --data '{"jsonrpc":"2.0","id":"1","method":"eth_syncing","params":[]}' http://127.0.0.1:8545

The result should look like this

{
  "jsonrpc" : "2.0",
  "id" : "1",
  "result" : false
}

Teku

For Teku, we will use a slightly different technique to observe the current block height (according to our node), because we did not enable the HTTP RPC server (and therefore cannot query it).

We are going to take a look at the live systemd logs, like this

sudo journalctl -fu teku.service

The output should look similar to this

an 26 02:25:00 kintsugi2-vm teku[12159]: 2022-01-26 02:25:00.100 INFO  - Epoch Event *** Epoch: 9125, Justified checkpoint: 6698, Finalized checkpoint: 6697, Finalized root: a253d50c06db0eae41abbde3b4e53bc4f7ce9447f71a2b13923829f9bf3dd519
Jan 26 02:25:04 kintsugi2-vm teku[12159]: 2022-01-26 02:25:04.101 INFO  - Slot Event  *** Slot: 292000, Block:    ... empty, Justified: 6698, Finalized: 6697, Peers: 74
Jan 26 02:25:16 kintsugi2-vm teku[12159]: 2022-01-26 02:25:16.101 INFO  - Slot Event  *** Slot: 292001, Block:    ... empty, Justified: 6698, Finalized: 6697, Peers: 74
Jan 26 02:25:28 kintsugi2-vm teku[12159]: 2022-01-26 02:25:28.101 INFO  - Slot Event  *** Slot: 292002, Block:    ... empty, Justified: 6698, Finalized: 6697, Peers: 74

Make note of the epoch and slot numbers you see in the output. Then head over to the Kintsugi beacon chain block explorer and compare the epoch and slot numbers to what your node sees.

Once again, if you're really far behind, chances are you just need to wait and give your node time to catch up.

Now what?

Congrats! You now have a node running an execution client and a consensus client on the Kintsugi testnet. You are providing a valuable service to help test the merge.

If you want to do even more, you could upgrade your node into a validator! But that will be a topic for another day, because this article is already pretty long.

If you want to continue on your journey, Remy Roy has a guide on this for a different client combo that you may be able to re-use here

There is no launchpad for Kintsugi yet but you can follow along on that progress here

Collecting feedback

If you discovered any technical issues or are seeing anything weird, you may want to consider providing feedback to the dev team.

But before you do that, I would encourage you to try the #testingthemerge channel in the ethstaker Discord. There are lots of community leaders and educators in that channel who may be able to assist you or filter out non-issues before creating triage work for the dev teams.

Besides the EthStaker discord, you could also try the Eth R&D Discord which has Teku, Besu, and merge related channels.

Finally, if you've found a very specific issue that you want to report, you can file an issue in the Besu and Teku Github repos. But please carefully review their contribution guidelines and fill out their template so that they can prioritize responding to real actionable issues.

Thanks for reading!

Ethereum