Can't install Azure packages with pip: ruamel.yaml error-CodePudding

I'm having trouble installing the following packages in a new python 3.9.7 virtual environment on Arch Linux.

My requirements.txt file:

joblib
python-dotenv
azure-cli==2.29.2
azureml-core

Steps to reproduce:

python3 -m venv venv
source venv/bin/activate
pip install -U pip
pip install -r requirements.txt

Here's the relevant part of the output of the last command, it errors when trying to install ruamel.yaml:

Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.
Installing collected packages: pycparser, urllib3, idna, chardet, cffi, certifi, six, requests, PyJWT, oauthlib, cryptography, requests-oauthlib, python-dateutil, isodate, azure-nspkg, wrapt, tabulate, pyyaml, PySocks, pyopenssl, PyNaCl, pygments, portalocker, msrest, msal, MarkupSafe, jmespath, colorama, bcrypt, azure-mgmt-nspkg, azure-core, argcomplete, applicationinsights, adal, websocket-client, vsts, pyparsing, pyasn1, psutil, pkginfo, pathlib2, paramiko, msrestazure, msal-extensions, knack, jinja2, jeepney, invoke, humanfriendly, deprecated, backports.weakref, azure-mgmt-datalake-nspkg, azure-mgmt-core, azure-common, azure-cli-telemetry, xmltodict, sshtunnel, semver, SecretStorage, scp, ruamel.yaml, pytz, PyGithub, pathspec, packaging, ndg-httpsclient, jsonpickle, jsondiff, javaproperties, fabric, docker, distro, contextlib2, backports.tempfile, azure-synapse-spark, azure-synapse-managedprivateendpoints, azure-synapse-artifacts, azure-synapse-accesscontrol, azure-storage-common, azure-multiapi-storage, azure-mgmt-web, azure-mgmt-trafficmanager, azure-mgmt-synapse, azure-mgmt-storage, azure-mgmt-sqlvirtualmachine, azure-mgmt-sql, azure-mgmt-signalr, azure-mgmt-servicefabricmanagedclusters, azure-mgmt-servicefabric, azure-mgmt-servicebus, azure-mgmt-security, azure-mgmt-search, azure-mgmt-resource, azure-mgmt-reservations, azure-mgmt-relay, azure-mgmt-redis, azure-mgmt-redhatopenshift, azure-mgmt-recoveryservicesbackup, azure-mgmt-recoveryservices, azure-mgmt-rdbms, azure-mgmt-privatedns, azure-mgmt-policyinsights, azure-mgmt-network, azure-mgmt-netapp, azure-mgmt-msi, azure-mgmt-monitor, azure-mgmt-media, azure-mgmt-marketplaceordering, azure-mgmt-maps, azure-mgmt-managementgroups, azure-mgmt-managedservices, azure-mgmt-loganalytics, azure-mgmt-kusto, azure-mgmt-keyvault, azure-mgmt-iothubprovisioningservices, azure-mgmt-iothub, azure-mgmt-iotcentral, azure-mgmt-imagebuilder, azure-mgmt-hdinsight, azure-mgmt-extendedlocation, azure-mgmt-eventhub, azure-mgmt-eventgrid, azure-mgmt-dns, azure-mgmt-devtestlabs, azure-mgmt-deploymentmanager, azure-mgmt-datamigration, azure-mgmt-datalake-store, azure-mgmt-datalake-analytics, azure-mgmt-databoxedge, azure-mgmt-cosmosdb, azure-mgmt-containerservice, azure-mgmt-containerregistry, azure-mgmt-containerinstance, azure-mgmt-consumption, azure-mgmt-compute, azure-mgmt-cognitiveservices, azure-mgmt-cdn, azure-mgmt-botservice, azure-mgmt-billing, azure-mgmt-batchai, azure-mgmt-batch, azure-mgmt-authorization, azure-mgmt-applicationinsights, azure-mgmt-appconfiguration, azure-mgmt-apimanagement, azure-mgmt-advisor, azure-loganalytics, azure-keyvault-keys, azure-keyvault-administration, azure-keyvault, azure-identity, azure-graphrbac, azure-functions-devops-build, azure-datalake-store, azure-cosmos, azure-cli-core, azure-batch, azure-appconfiguration, antlr4-python3-runtime, python-dotenv, joblib, azureml-core, azure-cli
    Running setup.py install for ruamel.yaml: started
    Running setup.py install for ruamel.yaml: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8y_97owt/install-record.txt --single-version-externally-managed --compile --install-headers /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml
         cwd: /tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/
    Complete output (649 lines):
    sys.argv ['/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py', 'install', '--record', '/tmp/pip-record-8y_97owt/install-record.txt', '--single-version-externally-managed', '--compile', '--install-headers', '/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml']
    test compiling test_ruamel_yaml
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.9
    creating build/lib.linux-x86_64-3.9/ruamel
    copying .ruamel/__init__.py -> build/lib.linux-x86_64-3.9/ruamel
    creating build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./main.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./util.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./tokens.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./scalarfloat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./scanner.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./comments.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./loader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./serializer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./resolver.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./nodes.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./dumper.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./configobjwalker.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./timestamp.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./error.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./reader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./cyaml.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./parser.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./__init__.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./representer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./events.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./scalarbool.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./composer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./scalarint.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./emitter.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./anchor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./compat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./constructor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./scalarstring.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
    copying ./LICENSE -> build/lib.linux-x86_64-3.9/ruamel/yaml
    running build_ext
    building '_ruamel_yaml' extension
    creating build/temp.linux-x86_64-3.9
    creating build/temp.linux-x86_64-3.9/ext
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include -I/usr/include/python3.9 -c ext/_ruamel_yaml.c -o build/temp.linux-x86_64-3.9/ext/_ruamel_yaml.o
    In file included from ext/_ruamel_yaml.c:523:
    ext/_ruamel_yaml.h:10: warning: "PyString_CheckExact" redefined

<a whole bunch of compiling errors & warnings>

The full log file can be found here

Note the line Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.. If I first install the wheel package, the same error appears when installing the packages.

Installing ruamel.yaml manually works fine, but the azure packages still can't be installed.

EDIT: Installing every package one by one installs everything without errors. Why?

Changing step 3 to pip install -U pip wheel ruamel.yaml doesn't change anything.

CodePudding user response：

The ruamel.yaml documentation states that it should be installed using:

pip install ruamel.yaml

so using the legacy [python] setup.py install should not be used.

Your full log indicates that it is tried to install two versions of ruamel.yaml:

Collecting ruamel.yaml<=0.17.16,>=0.17.10
Using cached ruamel.yaml-0.17.16-py3-none-any.whl (109 kB)

and

Collecting ruamel.yaml<=0.15.89,>=0.15.35
Using cached ruamel.yaml-0.15.89.tar.gz (306 kB)

Apart from a potential problem with two versions being installed, the second one will not succeed as the 0.15 ruamel.yaml series (the last one uploaded to PyPI in July 2019) doesn't support Python 3.9 first released in Oct 2020.

So when you install each package step by step, you get the latest version or ruamel.yaml. Why that step by step installation doesn't at with some package tries to install ruamel.yaml==0.15.89 as a dependency is unclear.

When I install the four packages listed in your requirements.txt, then ruamel.yaml doesn't get installed but pipdeptree indicates that azureml-core==1.36.0.post2 has possible conflicting dependencies.

When I use your requirements.txt I get the same error.

When I use your requirements.txt in a Python 3.7 venv (so that the old ruamel.yaml wheel can be installed) installation works. From that you can see that the package azureml-core==1.14.0 is installed instead of the azureml-core==1.36.0.post2