Host directory mounted as volume in docker compose - cannot write to host directory from within docker container

I needed to write to the host directory mounted as a volume in my docker container.  Running on my local environment worked great but failed when deployed to the test environment.

Environment:

• Ubuntu 20.04.6 LTS
• Docker Engine - Community Version 26.0.1

The host machine was a Ubuntu 20.04 virtual machine with user "develop".  Note that the "develop" user was in the "docker" group so didn't need to run sudo on any docker commands.  I only saw this problem when I deployed to the test environment because the "develop" user had a different user ID/group ID compared to my development environment.  On my development environment the "develop" user had user ID 1000 and group ID 1000, but on the test environment the "develop" user had user ID 1011 and group ID of 1011.

Problem

Here's my original code with the problem.

docker-compose.yml before (The $USER environment variable holds value "develop"):

services:

  cherry_container:

    container_name: cherry_container

    build:

      image: ubuntu:20.04

      args:

        SERVICE_USER: $USER

    volumes:

      - /cherryshoe_shared_volume:/home/$USER/cherryshoe_shared_volume

    extra_hosts:

      - "cs-test:172.16.0.0"

    environment:

      - "CS_EXTRA_ENV_VAR=example"

Dockerfile before:

# https://hub.docker.com/layers/library/ubuntu/20.04/images/sha256-b2339eee806d44d6a8adc0a790f824fb71f03366dd754d400316ae5a7e3ece3e

FROM ubuntu:20.04



# install OS tools

RUN apt-get update && apt-get install -y vim && \

apt-get clean all



# arguments available during docker build

ARG SERVICE_USER



ENV USERNAME $SERVICE_USER

RUN useradd -U -ms /bin/bash $USERNAME \

  && usermod -aG sudo $USERNAME \

  && echo "$USERNAME:$USERNAME" | chpasswd



# start in $USERNAME home as $USERNAME

WORKDIR /home/$USERNAME

USER $USERNAME

   

# https://serverfault.com/a/1084975

# keep container running for use by shelling into it with 'docker exec -it <container_name> bash'

CMD sh -c 'trap "exit" TERM; while true; do sleep 1; done'

Ran "docker compose config" to check that the args in the docker-compose.yml file were replaced as I expected them to be.

Shell into the cherryshoe_container: docker exec -it cherryshoe_container bash

The below commands show that a file could not be created while inside the cherryshoe_container docker container.  Notice the 1011 for user ID and group ID for the cherryshoe_shared_volume folder.

develop@0d8c5da6978e:~/cherryshoe_shared_volume$ ls -la

total 8

drwxrwxr-x 2 1011 1011 4096 Jun 26 17:10 .

drwxr-xr-x 1 develop develop 4096 Jun 26 17:11 ..

develop@0d8c5da6978e:~/cherryshoe_shared_volume$ touch hi.txt

touch: cannot touch 'hi.txt': Permission denied

Solution

In order for the "develop" user to write to the cherryshoe_shared_volume folder while inside the docker container was to have the user ID and group ID synchronized on the host and in the docker container.

Update "develop" user .bashrc file on the HOST with the user ID and group ID; source or restart the session:

export SERVICE_USER_ID=1011

export SERVICE_USER_GROUP_ID=1011

You can find out the user ID and group ID with these commands:

SERVICE_USER_ID=$(id -u)

SERVICE_USER_GROUP_ID=$(id -g)

docker-compose.yml after:

services:

  cherry_container:

    container_name: cherry_container

    build:

      image: ubuntu:20.04

      args:

        SERVICE_USER: $USER

        SERVICE_USER_ID: $SERVICE_USER_ID

        SERVICE_USER_GROUP_ID: $SERVICE_USER_GROUP_ID

    volumes:

      - /data:/home/$USER/data

    extra_hosts:

      - "cs-test:172.16.0.0"

    environment:

      - "CS_EXTRA_ENV_VAR=example"

Dockerfile after:

# https://hub.docker.com/layers/library/ubuntu/20.04/images/sha256-b2339eee806d44d6a8adc0a790f824fb71f03366dd754d400316ae5a7e3ece3e

FROM ubuntu:20.04



# install OS tools

RUN apt-get update && apt-get install -y vim && \

apt-get clean all



# arguments available during docker build

ARG SERVICE_USER

ARG SERVICE_USER_ID

ARG SERVICE_USER_GROUP_ID



# create USER environment variable to match Ubuntu host

ENV USER $SERVICE_USER 



# synchronize the service user inside the container with the service user outside the container so that they have the same permission to the mounted volume.

RUN addgroup --gid $SERVICE_USER_GROUP_ID $SERVICE_USER \

   && useradd -ms /bin/bash --uid $SERVICE_USER_ID --gid $SERVICE_USER_GROUP_ID $SERVICE_USER \ 

   && usermod -aG sudo $SERVICE_USER \

   && echo "$SERVICE_USER:$SERVICE_USER" | chpasswd



# start in $SERVICE_USER home as $SERVICE_USER

WORKDIR /home/$SERVICE_USER

USER $SERVICE_USER

   

# https://serverfault.com/a/1084975

# keep container running for use by shelling into it with 'docker exec -it <container_name> bash'

CMD sh -c 'trap "exit" TERM; while true; do sleep 1; done'

Ran "docker compose config" to check that the args in the docker-compose.yml file were replaced as I expected them to be.

Shell into the cherryshoe_container: docker exec -it cherryshoe_container bash

The below commands show that a file could now be created while inside the cherryshoe_container docker container.  Notice the user ID and group ID for the cherryshoe_shared_volume folder are no longer numbered but the username for the folder.

develop@0d8c5da6978e:~/cherryshoe_shared_volume$ ls -la

total 8

drwxrwxr-x 2 develop develop 4096 Jun 26 17:10 .

drwxr-xr-x 1 develop develop 4096 Jun 26 17:11 ..

develop@0d8c5da6978e:~/cherryshoe_shared_volume$ touch hi.txt