(Docker) thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CreatedNewConfigHalt

[Slluxx]

Following the documentation and trying this command

mkdir -p ./storage/database \
  && mkdir -p ./storage/ssl_certificates \
  && export TORRUST_TRACKER_USER_UID=1000 \
  && docker run -it \
    --user="$TORRUST_TRACKER_USER_UID" \
    --publish 6969:6969/udp \
    --publish 7070:7070/tcp \
    --publish 1212:1212/tcp \
    --volume "$(pwd)/storage":"/app/storage" \
    torrust/tracker:3.0.0-alpha.3

results in:

Loading configuration from config file ./config.toml
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CreatedNewConfigHalt { location: Location { file: "packages/configuration/src/lib.rs", line: 520, col: 27 }, path: "./config.toml" }', src/bootstrap/app.rs:83:52
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[josecelano]

Hi @Slluxx, I'm sorry, the documentation is unclear wrong in this section.

There are two ways to inject configuration into the application: a config.toml file in the root folder or an environment variable TORRUST_TRACKER_CONFIG with the same content as the file.

When you start the application, it fails if you do not provide one of those two options, and it creates a default config file in the root folder. Since the docker command only mounts the storage folder, the newly generated config file is not persisted.

If you only want to use docker without cloning the repo, you must inject the configuration. You can do it with:

TORRUST_TRACKER_USER_UID=$UID
TORRUST_TRACKER_CONFIG=$(wget -qO- "https://raw.githubusercontent.com/torrust/torrust-tracker/develop/config.toml.local")

mkdir -p ./storage/database \
  && mkdir -p ./storage/config \
  && mkdir -p ./storage/ssl_certificates \
  && touch ./storage/database/data.db \
  && echo ";" | sqlite3 ./storage/database/data.db \
  && docker run -it \
    --user="$TORRUST_TRACKER_USER_UID" \
    --publish 6969:6969/udp \
    --publish 7070:7070/tcp \
    --publish 1212:1212/tcp \
    --env TORRUST_TRACKER_CONFIG="$TORRUST_TRACKER_CONFIG" \
    --volume "$(pwd)/storage":"/app/storage" \
    torrust/tracker:3.0.0-alpha.3

It's explained in the specific documentation for docker.

You might have problems with file permission for the database file data.db if you run docker as root. In that case, you can also run the docker command as root.

Alternatively, you can build and run the docker dev env with:

git clone git@github.com:torrust/torrust-tracker.git
cd torrust-tracker
./docker/bin/install.sh   
./docker/bin/run.sh

[Slluxx]

Thanks for the detailed response!
I actually tried pulling the repo and use the command from inside, hoping that the config file would apply but i must have done something wrong. Probably because the file belongs in the storage folder.

The backend allows changing the default config file location with an env var

Usually i try to not use config files at all, make the most necessary bits an ENV var and then move everything else into some database (usually sqlite) to eliminate need for other services, make the project smaller as well as giving the option to edit anything on the fly via some kind of (web)interface. A database is created anyway, so there is no reason not to use it, i believe.

Anyway, thanks for taking this seriously and answering in such great detail. I will try to set this up in a few hours again.

BTW: I don't think it exists yet, so a docker ARM build would be nice. I know i could build it myself but compiling on a single board computer can be cumbersome

[josecelano]

Hi @Slluxx

Thanks for the detailed response! I actually tried pulling the repo and use the command from inside,

Do you mean the ./bin/install.sh command?

hoping that the config file would apply but i must have done something wrong. Probably because the file belongs in the storage folder.

The backend allows changing the default config file location with an env var

Just for the record in case other people have problems, what I usually do for debugging this type of problem is log into the container.

1.- Run this command to login into the container:

docker run -it \
  --user="$TORRUST_TRACKER_USER_UID" \
  --entrypoint="/bin/sh" \
  torrust/tracker:3.0.0-alpha.3

I do not need to mount volumes or bind ports.

2.- Run the tracker inside the container:

/app $ ./torrust-tracker 
Loading configuration from config file ./config.toml
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CreatedNewConfigHalt { location: Location { file: "packages/configuration/src/lib.rs", line: 520, col: 27 }, path: "./config.toml" }', src/bootstrap/app.rs:83:52
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

As you can see it creates the default config.toml but it's not persisted because we are not mounting that file or the file is not located in a mounted volume (for example: ./storage/config/config.toml).

$ ls -al
total 10836
drwxr-xr-x    1 appuser  appuser       4096 Aug  2 07:57 .
drwxr-xr-x    1 root     root          4096 Aug  2 07:56 ..
-rw-r--r--    1 appuser  appuser        715 Aug  2 07:57 config.toml
-rwxr-xr-x    1 appuser  appuser   11080768 Jul 10 11:03 torrust-tracker

Usually i try to not use config files at all, make the most necessary bits an ENV var and then move everything else into some database (usually sqlite) to eliminate need for other services, make the project smaller as well as giving the option to edit anything on the fly via some kind of (web)interface. A database is created anyway, so there is no reason not to use it, i believe.

Yes, I totally agree. In fact, we want to do that on the Index Backend.

Anyway, thanks for taking this seriously and answering in such great detail. I will try to set this up in a few hours again.

No worries, thank you for opening the issue. That makes the app better. I'm going to add some more points to the TODO list I've started writing and move it to a new comment so we can link it directly. I think we could also extract a command to run the app with docker and also test the command with a workflow or something, as Rust does for the code in the comments. To make sure the infrastructure commands in the documentation work.

BTW: I don't think it exists yet, so a docker ARM build would be nice. I know I could build it myself but compiling on a single board computer can be cumbersome

Yes, I totally forgot about that. When I added support for docker, I struggled to use the cache for the cargo dependencies. If I'm not wrong I had to specify the target otherwise the docker layer to build the dependencies was always re-executed. But I'm not sure now. That's definitely something we have to do. And also review support for Windows. I've been working primarily on writing documentation and testing recently. I've opened a feature request for that.

I also want to give you a little bit of extra context. You can also mount the config.toml in the root directory like this:

wget https://raw.githubusercontent.com/torrust/torrust-tracker/develop/config.toml.local
mv config.toml.local config.toml

mkdir -p ./storage/database \
  && mkdir -p ./storage/ssl_certificates \
  && export TORRUST_TRACKER_USER_UID=1000 \
  && docker run -it \
    --user="$TORRUST_TRACKER_USER_UID" \
    --publish 6969:6969/udp \
    --publish 7070:7070/tcp \
    --publish 1212:1212/tcp \
    --volume "$(pwd)/storage":"/app/storage" \
    --volume "$(pwd)/config.toml":"/app/config.toml":ro \
    torrust/tracker:3.0.0-alpha.3

Notice the new line --volume "$(pwd)/config.toml":"/app/config.toml":ro. The first two lines are just to get a copy of the default configuration file without running the app.

That command should work and It could be the one-command setup I mentioned before. But I have been trying to avoid this solution because all hosting provider (as far as I know) do not allow to mount single files.

And finally, I wrote an article for the torrust blog:

How To Run A Local Demo

It uses docker-compose. But it's not only for the tracker but for the whole solution Tracker+Index.

[Slluxx]

Glad to see that ARM is important to you.

Do you mean the ./bin/install.sh command?

No, it was super late and i wasnt giving it enough brainpower, so i cloned this repo and ran the docker command from inside. Don't mind that though, that was just me at 3am with to little sleep haha.

And finally, I wrote an article for the torrust blog:

Yes, that is actually how i was introduced to this project. However i was (and still am) unable to run this command git clone git@github.com:torrust/torrust-compose.git

C:\Users\Slluxx\Desktop\test>git clone git@github.com:torrust/torrust-compose.git
Cloning into 'torrust-compose'...
The authenticity of host 'github.com (140.82.121.4)' can't be established.
ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Ive always had issues with git@.*.git cloning. Not sure if that's because of me or because https://github.com/torrust/torrust-compose does not exist. I assume you might have renamed compose to tracker at one point? Or something else that i cant think about

[josecelano]

Glad to see that ARM is important to you.

Do you mean the ./bin/install.sh command?

No, it was super late and i wasnt giving it enough brainpower, so i cloned this repo and ran the docker command from inside. Don't mind that though, that was just me at 3am with to little sleep haha.

And finally, I wrote an article for the torrust blog:

Yes, that is actually how i was introduced to this project. However i was (and still am) unable to run this command git clone git@github.com:torrust/torrust-compose.git

C:\Users\Slluxx\Desktop\test>git clone git@github.com:torrust/torrust-compose.git
Cloning into 'torrust-compose'...
The authenticity of host 'github.com (140.82.121.4)' can't be established.
ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Ive always had issues with git@.*.git cloning. Not sure if that's because of me or because https://github.com/torrust/torrust-compose does not exist. I assume you might have renamed compose to tracker at one point? Or something else that i cant think about

Hey @Slluxx, sorry, I've just realized the repo was still private :-/. I think you have been the very first to follow the blog post.

[Slluxx]

Haha, absolutely no problem. Glad we figured this out together :D
I will give it a try tomorrow evening!

[josecelano]

Hi @Slluxx I've created a new issue with all the tasks I want to do #361

I will fix this one by only replacing the wrong command you tried:

mkdir -p ./storage/database \
  && mkdir -p ./storage/ssl_certificates \
  && export TORRUST_TRACKER_USER_UID=1000 \
  && docker run -it \
    --user="$TORRUST_TRACKER_USER_UID" \
    --publish 6969:6969/udp \
    --publish 7070:7070/tcp \
    --publish 1212:1212/tcp \
    --volume "$(pwd)/storage":"/app/storage" \
    torrust/tracker:3.0.0-alpha.3

With the fixed version:

wget https://raw.githubusercontent.com/torrust/torrust-tracker/v3.0.0-alpha.3/config.toml.local -O config.toml \
  && mkdir -p ./storage/database \
  && mkdir -p ./storage/ssl_certificates \
  && export TORRUST_TRACKER_USER_UID=1000 \
  && docker run -it \
    --user="$TORRUST_TRACKER_USER_UID" \
    --publish 6969:6969/udp \
    --publish 7070:7070/tcp \
    --publish 1212:1212/tcp \
    --volume "$(pwd)/storage":"/app/storage" \
    --volume "$(pwd)/config.toml":"/app/config.toml":ro \
    torrust/tracker:3.0.0-alpha.3

[josecelano]

Hi @Slluxx, I've just realized there is something else that will only work if you change the default configuration.

The default bound IP for the API socket is the localhost: 127.0.0.1. The reason why it's the default one is we do not want to make the API publicly accessible by default. If you use docker, the API will not be accessible from the host. You need to change the API configuration to :

[http_api]
enabled = true
bind_address = "0.0.0.0:1212"
ssl_enabled = false
ssl_cert_path = ""
ssl_key_path = ""

[josecelano]

Hi @Slluxx, I've just realized there is something else that will only work if you change the default configuration.

The default bound IP for the API socket is the localhost: 127.0.0.1. The reason why it's the default one is we do not want to make the API publicly accessible by default. If you use docker, the API will not be accessible from the host. You need to change the API configuration to :

[http_api]
enabled = true
bind_address = "0.0.0.0:1212"
ssl_enabled = false
ssl_cert_path = ""
ssl_key_path = ""

It's mentioned in the docker docs.

[Slluxx]

I have not yet been able to try to make this work any further (life, sigh). I would have completely missed that too so its good that you brought this up. I hope that in the future, this is all either part of env variables anyone can set in the compose file or manageable via some webinterface.

That aside, i feel like that something should work "out of the box" (especially using Docker). If the entire system doesn't work with the default configuration provided, its more a hassle than anything else. I can definitely see why this choice was made, however it works against UX if you expect something to work and then need to "troubleshoot" in the docs.

Basically "Here is a command/compose file to set everything up to a basic but working state, over here are all the things you can configure.", instead of "This command sets it up but it wont run and how to make it run you can read here and here".

That would make it more accessible to everyone, lower frustration and minimize possible github issues like this one haha

[josecelano]

I have not yet been able to try to make this work any further (life, sigh). I would have completely missed that too so its good that you brought this up. I hope that in the future, this is all either part of env variables anyone can set in the compose file or manageable via some webinterface.

That aside, i feel like that something should work "out of the box" (especially using Docker). If the entire system doesn't work with the default configuration provided, its more a hassle than anything else. I can definitely see why this choice was made, however it works against UX if you expect something to work and then need to "troubleshoot" in the docs.

Basically "Here is a command/compose file to set everything up to a basic but working state, over here are all the things you can configure.", instead of "This command sets it up but it wont run and how to make it run you can read here and here".

That would make it more accessible to everyone, lower frustration and minimize possible github issues like this one haha

Hi @Slluxx thank you for your feedback. I totally agree.

In this case, it's not that it would not work, it would work but without exposing the API. The default configuration is intentionally more restrictive.

But I agree that we have to provide a more consistent, stable, tested, documented and friendly UX experience. Most of the docker support was done by me, and my main goal at the time was to setup environments for testing. @da2ce7 is now reviewing the docker configuration. Regarding the setup, I think we have to improve a lot of things:

First of all, I see three different phases:

1. Setup system dependencies: you need the OS and set up some external dependencies like MySQL or SQLite. Notice that they can be optional.
2. Then you need to setup the minimal app dependencies to run it. For example, the storage folder, the initialized database (if you are using a database) and the configuration config.yml. This is typically done before running the application.
3. Finally, the application could have some ways to customize it while running. We want to move most options in the configuration from phase 2 to this phase. Ideally, we could just provide a directory and the DB connection (if needed) to start the application.

For each phase, we should provide a nice UX. But docker is optional. There is no official way to deliver the application. In the long term, we plan to move most of the configuration options to phase 3. Then the first time you run the app (or whenever you can to set it up) you can run an installation/configuration assistant. That could be a webapp as you mentioned or a console assistant.

Secondly, the purpose of the installation could be different. For example:

• Development environment
• Demo environment with preset configuration
• Production environment (fully customized, HTTPS support, etcetera)

The problem with the current docker scripts is I'm mixing concepts from the 3 phases and also the purposes for the final deployment. In the current version, scripts are mixing the three phases and the purpose it's mainly demo or test env.

For this issue, my intention for that script was to provide a quick way for people who just can give it a try. I think the best way to achieve it would be:

• Provide a docker image you can run from the prebuilt public image. You only need to mount a storage folder. This image only provides the system dependencies.
• By default, the application also has a website assistant available at locahost:3000 or maybe you can log in in the container and run a tracker --install command, depending on the type of installer we want to provide.
• The output of the previous could be a config file in the storage folder or even some configuration values in a settings DB table.

If you do not want to use docker you can just setup the dependencies manually and run the installation/assistant process. Somehow we have a basic assistant now, if you do not provide a config file, the application will create the default one. But it's not user and docker friendly.

Anyway, suggestions to improve the process are welcome. I'm working on a PR to improve the documentation. FOr now, It's just an small improvement.