Docker Compose cannot find pip installed package-CodePudding

Docker Compose gives ModuleNotFoundError: No module named 'django' error. Docker Compose has not installed my pip installed packages pip install -r requirements.txt but running the image any other way shows they are installed and this issue is only with docker-compose, why?

Compose

version: '3.8'

services:
  web:
    build: ./
    user: python
    volumes:
      - ./:/app
    ports:
      - 8000:8000
    env_file:
      - ./.env.dev

Dockerfile

# Base image  
FROM python:3.9.6

ENV HOME=/app

# create directory for the app user
RUN mkdir -p $HOME

# set work directory
WORKDIR $HOME
 
# install psycopg2 dependencies
RUN apt-get update \
    && apt-get -y install libpq-dev gcc \
    && pip install psycopg2 \
    && apt-get -y install gunicorn3

RUN pip install --upgrade pip
ADD requirements*.txt .
RUN pip install -r requirements.txt
COPY python . .

ENTRYPOINT ["/app/entrypoint.sh"]

EXPOSE 8000

Problem:

I have created the following Dockerfile which runs in production and even runs locally outside of docker-compose without any issues i.e the following works with no errors docker run -p 8000:8000 web/lastest.

However, when I run this via docker-compose it fails to find my installed pip packages.

For example:

docker-compose build (successful)
docker-compose up

Error

web_1  | ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
web_1  | [2022-01-04 14:55:05  0000] [1] [INFO] Starting gunicorn 20.1.0
web_1  | [2022-01-04 14:55:05  0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
web_1  | [2022-01-04 14:55:05  0000] [1] [INFO] Using worker: sync
web_1  | [2022-01-04 14:55:05  0000] [8] [INFO] Booting worker with pid: 8
web_1  | [2022-01-04 14:55:05  0000] [8] [ERROR] Exception in worker process
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/arbiter.py", line 589, in spawn_worker
web_1  |     worker.init_process()
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 134, in init_process
web_1  |     self.load_wsgi()
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/workers/base.py", line 146, in load_wsgi
web_1  |     self.wsgi = self.app.wsgi()
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/app/base.py", line 67, in wsgi
web_1  |     self.callable = self.load()
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 58, in load
web_1  |     return self.load_wsgiapp()
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
web_1  |     return util.import_app(self.app_uri)
web_1  |   File "/usr/lib/python3/dist-packages/gunicorn/util.py", line 384, in import_app
web_1  |     mod = importlib.import_module(module)
web_1  |   File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
web_1  |     return _bootstrap._gcd_import(name[level:], package, level)
web_1  |   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
web_1  |   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
web_1  |   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
web_1  |   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
web_1  |   File "<frozen importlib._bootstrap_external>", line 790, in exec_module
web_1  |   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
web_1  |   File "/app/app/wsgi.py", line 12, in <module>
web_1  |     from django.core.wsgi import get_wsgi_application
web_1  | ModuleNotFoundError: No module named 'django'

Running which python outputs /usr/local/bin/python on both running the image directly and using docker-compose.

Running docker run -it 43d991d65c02 /bin/bash I can see and run Django. Only when running docker-compose is Django not installed, why?

CodePudding user response：

In the containerfile presented, we work in the container-directory /app. But at runtime, we mount a volume to /app. Hence, all content that is generated during image build time that is stored in /app is overridden by the volume mount. If the dependencies at runtime were installed in /app, then they are overridden by the volume mount.

To fix this issue, two possibilities come to my mind:

We can remove the volume mount. This will, however, devoid us of the capability of "hot reloading".
We can re-run pip install -r requirements.txt at container startup, before starting the application. This would mean adding the line pip install -r requirements.txt to the entrypoint.sh-script.

CodePudding user response：

When docker-compose mounts the volume to /app folder its previous structure become hidden and new structure overrides previous one.