Troubleshooting

Need help solving an issue? Check to see if this has already been answered below. If you can’t find an answer to your issue, reach out to the SKALE Network developer community on Discord.

Ask a question

How do I change my Node’s IP address?

See Changing Node IP.

How should validators configure back up and restore?

Validators are responsible for backing up everything on the node. SKALE Network is relying on validators to complete the backup procedure for each of the nodes. SKALE Network will be replicating the data for each SKALE Chain across 16 nodes, adding an additional layer of data availability. If a node goes down for a long time the other 2/3 of 15 will be responsible for taking the SKALE Chain’s snapshot and shuffling the SKALE Chain in the SKALE Network. If a validator node is down for a long time, it may be reallocated to service new SKALE Chains depending on the length of the downtime.

Please check node back up and sgx_backup to learn more about how to back up node or SGXWallet, and restore.

Where are the network communications between containers, shared configuration files and volumes stored?

You can find these details in GitHub [skale-node](https://github.com/skalenetwork/skale-node) repository and [docker-compose.yml](https://github.com/skalenetwork/skale-node/blob/f928b95e69c548f12b4b21bd11a16fe2d239b83b/docker-compose.yml)

How do I access the SKALE Chain’s IP and ports?

You can use the node-cli to view firewall info and SKALE Chain info:

# Show firewall rules for IP and Ports
skale schains show-rules

# Show the SKALE Chain Config
skale schains config [SCHAIN_NAME]

In this file you can see all the SKALE Chain information:

5d9f9cb5adfc337b00747f66 Screen%20Shot%202019 10 10%20at%201.53.21%20PM

What are the following Node ports used for?

Port Description Notes

3009/TCP

Watchdog service provides sla-agent and SKALE Monitoring/Metrics site with data about all containers statuses on all SKALE nodes

Used by nginx as reverse proxy for Flask based Watchdog

8080/TCP

Used by cAdvisor (Prometheus exporter) - to analyze and expose resource usage and performance data from running docker containers

Running cAdvisor container is optional (MONITORING_CONTAINERS=True)

9100/TCP

Used by Prometheus Node Exporter - to measure host resources such as memory, disk and CPU utilization etc.

Running Node exporter container is optional (MONITORING_CONTAINERS=True)

22

Used by the node operator to ssh into the machine

10000–11500/TCP

These ports should be open for communications inside sChains between different nodes.

When there are no sChains these ports are closed by iptables rules. It’s important to not add another firewall, because sChain configuration is created dynamically as well as iptables rules.

Can I use Kubernetes?

Yes. However, you will need to make sure that your platform is compatible with SKALE. In particular, your platform must allow docker-lvmpy to dynamically partition the external drive for each SKALE chain.

Where can I find the SKALE Node logs

Validators can run these commands to check their logs in their node:

Here are some logs and commands you can use for troubleshooting or provide logs to the core team

the most used and first place to look at

docker logs skale_api

or

docker logs skale_admin

transaction manager logs

docker logs skale_transaction_manager

all running containers logs

skale logs dump [PATH]

SGX certification in validator node:

ls -l ~/.skale/node_data/sgx_certs/

sgxwallet logs:

docker logs runsgx_sgxwallet_1

node-cli debugging logs

~/.skale/.skale-cli-log/debug-node-cli.log

docker-lvmpy logs

cat /var/log/docker-lvmpy/lvmpy.log

Why is SGX crashing after machine reboot?

SGXWallet may crash after a reboot if you didn’t disable automatic updates. This is because SGXWallet is based on new low level technology, and kernel updates may break the system. It’s recommended to only update the SGXWallet server if there are critical security fixes.

Node update procedure

Updating system packages is important part of node maintenance and security. Updates may contain fixes for recent security issues. However, newer updates may be incompatible with SKALE node software. Therefore, updating should be executed carefully.

Make sure that the infrastructure provider doesn’t update packages after system is restarted.

The following list contains packages that require additional attention.

Critical risk (updates should be performed only if there are some messages from core team)

  • kernel updates on SGXWallet server.

Currently SGX is tested on 4.15-* kernels. It’s best to avoid minor version updates too.
High risk (updates should be avoided in general)

  • kernel on SKALE node server

  • docker

  • docker-compose

  • btrfs-progs

Medium risk (updates should be performed very carefully)

  • lvm2

  • iptables

  • iptables-persistent

  • python

=== General tips

  1. Carefully check which packages will be updated when executing apt upgrade. You can use apt list --upgradable

  2. Avoid executing apt dist-upgrade

  3. Disabling updates for certain packages can be done using apt-mark hold (https://manpages.debian.org/stretch/apt/apt-mark.8.en.html). For example to forbid kernel updates you should run:

sudo apt-mark hold linux-generic linux-image-generic linux-headers-generic

Another option is to use unattended upgrades (https://wiki.debian.org/UnattendedUpgrades) which allows to automatically keep system current with the latest security updates. There is an option to exclude some packages from update list using Package-Blacklist section in the configuration file. For example to disallow linux kernel updates you should place the following lines in /etc/apt/apt.conf.d/50unattended-upgrades file:

Unattended-Upgrade::Package-Blacklist {
        "linux-generic";
        "linux-image-generic";
        "linux-headers-generic";
};

SKALE Network has many resources designed to help you get your questions answered. You can reach out to the SKALE Network developer community on discord, or submit a support request below.

Contact Support

Edit on Github