I want to create a default python environment with many external libraries pre-installed to be used in many servers that do not have internet connection, I can only transfer files using sftp protocol. What are the options I have?
CodePudding user response：
You can download the appropriate .tar.gz files for the libraries you want from pypi.org and install them with pip install. The .tar.gz will still need to be built on the computer they are installed to, but for pure Python packages, that won't be a problem if Python is already installed.
For cases where that doesn't work, you can try obtaining a pre-built .whl file from an online source you trust - for example Christoph Golkhe (https://www.lfd.uci.edu/~gohlke/pythonlibs/) has a good collection here, but again, only use if you trust them to build safe software.
Note that PyPI will also have the .whl for many pre-built packages. For example, have a look at the files page for pandas: https://pypi.org/project/pandas/#files
But some libraries don't, for example GDAL: https://pypi.org/project/GDAL/#files - this is a good example of a package that may be hard to build on just any machine and third party repositories may offer pre-built binaries like this: https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
However, you will need the .whl that matches the version of Python and the OS and system architecture (e.g. not just "GDAL", but "GDAL for Python 3.9 on Windows 64-bit").
To create a default environment, your best option is probably to write an installation script that:
- installs Python
- creates a virtual environment
- installs all of the packages you bundle with it
and package the script together with everything else for easy upload. You include use a requirements.txt referring to the local files to make installation in the correct order easy without having to change the script later, if you add packages.
CodePudding user response：
You should definitely look at using automation to solve this problem. I would use something like Fabric to set up a script to:
- SSH to a host
- Check python version
- Copy python installer
- Uninstall / install latest python
- copy required libs
- install required libs
- Confirm python install and libs are functioning correctly
Once you have a script that does this, you can feed it a list of server IPs and have it SSH into each one and run through the same steps, so you can hit "go" and have it do this for every machine without manual intervention.
There may be some better ways to achieve some of the things you mentioned (like installing the latest python version), but we'd need to know which operating system(s) and versions, and which package managers are used.