Allow binaries written in Python via a "platform python" style approach

[cgwalters]

Like https://fedoraproject.org/wiki/Changes/Platform_Python_Stack

But we can do it in even stronger way: if we wanted to make it a whole lot harder for external people to use it, we could randomize the location of the binary, e.g. /usr/libexec/platform-python-$(uuidgen).

(Walking over every python binary and rewrite the shebangs)

[cgwalters]

I could pretty easily add an rpm-ostree compose option to do this.

[bgilbert]

/usr/libexec/platform-python-$(uuidgen) is a pretty cool idea if we end up needing to ship Python. It'd still be better not to have to maintain a Python at all, though.

[ajeddeloh]

I'd really really really like to not ship python. It's a huge chunk of the dependency tree we don't need to worry about. Less to ship is less to break. I'd also like to not ship an entire language to support one package.

I really want to keep FCOS as minimal as possibe.

[dustymabe]

small nit: with the /usr/libexec/platform-python-$(uuidgen) approach it would mean files in subsequent composes have different checksums, right? that being said I'm sure the same goal could be achieved

To re-iterate some previous conversations we had I think (I'm working off of memory here) there were multiple reasons we wanted to keep python out of the base:

1. prevent users from running scripts directly on the host
2. prevent shipping/maintaining python
3. prevent issues where user's python script needs library X that isn't installed
4. prevent security issues in python requiring a respin

This proposal helps with the 1st and 3rd of those since user's wouldn't be able to run their own scripts. The others would still be issues.

[dustymabe]

Added the meeting label so we can discuss during the community meeting today.

[dustymabe]

Discussed in the community meeting today (see the log starting at 16:59:17).

We added two more smallish reasons why it would be nice to remove python from the base. The list of reasons now comes out to:

1. prevent users from running scripts directly on the host
2. prevent shipping/maintaining python
3. prevent issues where user's python script needs library X that isn't installed
4. prevent security issues in python requiring a respin
5. less space used on disk + less data transmitted for updates
6. better perception we're a minimal OS

Each reason was then analyzed and we determined

• #1 is our primary concern
• #3 is also important so we don't break users when we make changes
• #2 is mostly handled for us because we are part of Fedora, but there is still a cost associated to finding/fixing bugs related to python and tools that use it, so it won't be non-zero if we do include a system python.
• #4 we have an action item to evaluate how often python (or python related libraries) in Fedora CoreOS need security updates (can use package set in Fedora Atomic Host as the reference).
• #5/#6 minimal is good, but we don't think we'd lose a significant number of users/community members if we shipped a system python, vs no python.

I'll dig into the investigation for #4 this week and we'll revisit in the community meeting next wednesday.

[glevand]

I feel the discussion should be If python support is not included, what do we need to do?, as apposed to If python support is included, what do we need to do?.

In the meeting this week the question came up about selinux and python. CL has a python USE flag, and support for that was added to the selinux packages to either not build certain parts of the packages, or not install certain parts of the packages. See the python USE flag commits of my PR that updates CL selinux. I guess we would need to do the same for FCOS.

[dustymabe]

I feel the discussion should be If python support is not included, what do we need to do?, as apposed to If python support is included, what do we need to do?.

I don't disagree. Unfortunately we're working a bit backwards here because this discussion was born out of the firewall discussion in #26. So it was more Is firewalld an option? and then we started this conversation to see if it was an option.

[dustymabe]

OK. I tried to take a stab at looking at our current python content in Fedora CoreOS
based on the image I create using the configs in the fedora-coreos-configs
repo. I concentrated on python3 since python2 should be completely gone soon.

I gathered the list of packages by running rpm -e python3

[root@coreos ~]# rpm -e python3
error: Failed dependencies:
        python(abi) = 3.7 is needed by (installed) python3-setuptools-39.2.0-7.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-pip-18.0-3.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-libselinux-2.8-3.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-setools-4.1.1-12.1.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-sssdconfig-2.0.0-3.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-libsemanage-2.8-3.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-pyyaml-4.2-0.1.b4.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-ply-3.9-8.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-bind-32:9.11.4-5.P1.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-pytoml-0.1.18-1.fc29.noarch
        python(abi) = 3.7 is needed by (installed) atomic-registries-1.22.1-27.gitb507039.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-IPy-0.81-23.fc29.noarch
        python(abi) = 3.7 is needed by (installed) python3-audit-3.0-0.2.20180808git77fbcf3.fc29.x86_64
        python(abi) = 3.7 is needed by (installed) python3-policycoreutils-2.8-6.fc29.noarch
        python(abi) = 3.7 is needed by (installed) authconfig-7.0.1-7.fc29.x86_64
        python3 is needed by (installed) python3-bind-32:9.11.4-5.P1.fc29.noarch
        python3 is needed by (installed) authconfig-7.0.1-7.fc29.x86_64
        /usr/bin/python3 is needed by (installed) python3-setuptools-39.2.0-7.fc29.noarch
        /usr/bin/python3 is needed by (installed) python3-pip-18.0-3.fc29.noarch
        /usr/bin/python3 is needed by (installed) python3-libsemanage-2.8-3.fc29.x86_64
        /usr/bin/python3 is needed by (installed) bind-utils-32:9.11.4-5.P1.fc29.x86_64
        /usr/bin/python3 is needed by (installed) atomic-registries-1.22.1-27.gitb507039.fc29.x86_64
        /usr/bin/python3 is needed by (installed) python3-policycoreutils-2.8-6.fc29.noarch
        /usr/bin/python3 is needed by (installed) policycoreutils-python-utils-2.8-6.fc29.noarch
        /usr/bin/python3 is needed by (installed) authconfig-7.0.1-7.fc29.x86_64
        /usr/bin/python3 is needed by (installed) nfs-utils-1:2.3.2-1.rc3.fc29.x86_64
        /usr/bin/python3 is needed by (installed) xfsprogs-4.17.0-3.fc29.x86_64
        /usr/bin/python3 is needed by (installed) setools-console-4.1.1-12.1.fc29.x86_64

I then looked at the bodhi updates for each one including python itself. I
grouped together security updates that seemed to be related but for several
releases of fedora at the same time (i.e. a security update for foo-2.1 for
f24, f25, f26).

python itself

• ~12 issues over past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=python
• https://bodhi.fedoraproject.org/updates/?packages=python2
• https://bodhi.fedoraproject.org/updates/?packages=python3

python3-setuptools

• no listed security issues in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=python-setuptools

python3-pip

• ~4 listed security issues in past 5 years
• https://bodhi.fedoraproject.org/updates/?packages=python-pip

python3-libselinux

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=libselinux

python3-setools & setools-console

• no listed security issues in past 2 years
• https://bodhi.fedoraproject.org/updates/?packages=setools

python3-sssdconfig

• 2 listed security issues in past 5 years
• https://bodhi.fedoraproject.org/updates/?packages=sssd

python3-libsemanage

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=libsemanage

python3-pyyaml

• 2 listed security issues in past 6 years
• https://bodhi.fedoraproject.org/updates/?packages=PyYAML

python3-ply

• no listed security issues in past 5 years
• https://bodhi.fedoraproject.org/updates/?packages=python-ply

python3-bind & bind-utils

• ~13 listed security issues in the past 5 years
• https://bodhi.fedoraproject.org/updates/?packages=bind
• note that these bulletins are for bind an all subpackages. In many cases I would
  bet the "clients" wouldn't be affected.

python3-pytoml

• no listed security issues in past 2 years
• https://bodhi.fedoraproject.org/updates/?packages=python-pytoml

atomic-registries

• no listed security issues in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=atomic

python3-IPy

• no listed security issues in past 8 years
• https://bodhi.fedoraproject.org/updates/?packages=python-IPy

python3-audit

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=audit

python3-policycoreutils & policycoreutils-python-utils

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=policycoreutils

authconfig

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=authconfig

nfs-utils

• no listed security issues in past 4 years
• https://bodhi.fedoraproject.org/updates/?packages=nfs-utils

xfsprogs

• 1 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=xfsprogs

note that we do not currently have firewalld installed but I'll look
at issues for firewalld too:

• 3 listed security issue in past 3 years
• https://bodhi.fedoraproject.org/updates/?packages=firewalld

Most packages have very few security updates but there are some that have
more than 10.

[embik]

Just to add my two cents as a present Container Linux and future FCOS user: We use (self-deployed) Python on Container Linux, mainly because Ansible is our automation driver for Kubernetes clusters. With kubeadm, we need to run a few tasks on the host itself like deploying updated config files and restarting services (I don't see a way around that at the moment). I would love to see Python to be included in FCOS .