在Docker映像中运行气流命令错误：[Errno 13]权限被拒绝：'/ opt / airflow / logs / scheduler /

如何解决在Docker映像中运行气流命令错误：[Errno 13]权限被拒绝：'/ opt / airflow / logs / scheduler /

我正在docker hub的apache/airflow:latest上创建一个映像，该映像通过本地dag和插件复制。构建本地气流图像后，我运行命令docker run -it local_airflow:latest list_dags列出了最近复制的dag，但是得到了输出：

Unable to load the config,contains a configuration error.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/logging/config.py",line 565,in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/local/lib/python3.6/logging/config.py",line 738,in configure_handler
    result = factory(**kwargs)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/log/file_processor_handler.py",line 50,in __init__
    os.makedirs(self._get_log_directory())
  File "/usr/local/lib/python3.6/os.py",line 220,in makedirs
    mkdir(name,mode)
PermissionError: [Errno 13] Permission denied: '/opt/airflow/logs/scheduler/2020-09-03'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow",line 25,in <module>
    from airflow.configuration import conf
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__init__.py",line 47,in <module>
    settings.initialize()
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/settings.py",line 402,in initialize
    LOGGING_CLASS_PATH = configure_logging()
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/logging_config.py",line 68,in configure_logging
    raise e
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/logging_config.py",line 63,in configure_logging
    dictConfig(logging_config)
  File "/usr/local/lib/python3.6/logging/config.py",line 802,in dictConfig
    dictConfigClass(config).configure()
  File "/usr/local/lib/python3.6/logging/config.py",line 573,in configure
    '%r: %s' % (name,e))
ValueError: Unable to configure handler 'processor': [Errno 13] Permission denied: '/opt/airflow/logs/scheduler/2020-09-03'

这是在基本气流图像之上构建的Dockerfile：

FROM apache/airflow:latest

USER airflow

ARG REQUIREMENTS_TXT=""
ENV REQUIREMENTS_TXT=${REQUIREMENTS_TXT}

COPY $REQUIREMENTS_TXT $REQUIREMENTS_TXT

ARG AIRFLOW_CONSTRAINTS_URL=""
ENV AIRFLOW_CONSTRAINTS_URL=${AIRFLOW_CONSTRAINTS_URL}

RUN if [ ! -z "${REQUIREMENTS_TXT}" ]; then pip install --user --upgrade pip && \
    pip install --user -r "${REQUIREMENTS_TXT}" \
    --constraint "${AIRFLOW_CONSTRAINTS_URL}"; fi

ARG DAGS_FOLDER="dags/"
ENV DAGS_FOLDER=${DAGS_FOLDER}

COPY dags/ $AIRFLOW_HOME/dags/

ARG PLUGINS_FOLDER="plugins/"
ENV PLUGINS_FOLDER=${PLUGINS_FOLDER}

COPY plugins/ $AIRFLOW_HOME/plugins/

这是docker build命令

docker build . \
    --tag local_airflow:latest \
    --build-arg DAGS_FOLDER="dags/" \
    --build-arg PLUGINS_FOLDER="plugins/" 

解决方法

问题是这些文件的所有者（几乎可以肯定）是root。因此，您无权在此进行任何更改。

我建议将COPY命令更改为如下形式：

COPY --chown=airflow ... ...

这会将文件/文件夹的所有权更改为airflow。

有关更多信息，请查看文档：{​​{3}}