如何解决StreamSets服务器缺少服务com.streamsets.pipeline.api.service.dataformats.DataFormatParserService
我创建了StreamSets服务器的Docker容器，并且可以正常启动它。
但是，在创建第一个管道时，系统会要求我安装streamsets-datacollector-dev-lib阶段库。
这样做并重新启动容器后，该服务在控制台中显示以下错误：
Abnormal exit: java.lang.RuntimeException: Missing services: Stage com_streamsets_pipeline_stage_devtest_rawdata_RawDataDSource is missing service com.streamsets.pipeline.api.service.dataformats.DataFormatParserServiceCheck STDERR for more details
streamsets_streamsets.1.9oqwp3hfpzox@stephane-pc |
streamsets_streamsets.1.9oqwp3hfpzox@stephane-pc | java.lang.RuntimeException: Missing services: Stage com_streamsets_pipeline_stage_devtest_rawdata_RawDataDSource is missing service com.streamsets.pipeline.api.service.dataformats.DataFormatParserService
删除streamsets-datacollector-dev-lib文件时，容器可以正常重启。
Dockerfile文件：
FROM streamsets/datacollector
docker-compose-dev.yml文件：
version: "3.7"
services:
streamsets:
image: localhost:5000/streamsets:latest
ports:
- "18630:18630"
networks:
streamsets:
common:
volumes:
- "~/dev/docker/projects/streamsets/volumes/data:/data"
- "~/dev/docker/projects/streamsets/volumes/stage-libs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
- "~/dev/docker/projects/streamsets/volumes/stage-libs-extra:/opt/stage-libs-extra"
- "~/dev/docker/projects/streamsets/volumes/logs:/logs"
environment:
SDC_CONF_STREAMSETS_LIBRARIES_EXTRA_DIR: "/opt/stage-libs-extra"
HOST_USER_ID: ${CURRENT_UID}
HOST_GROUP_ID: ${CURRENT_GID}
deploy:
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
window: 10s
healthcheck:
test: curl --fail http://localhost:18630 || exit 1
interval: 1m
timeout: 3s
retries: 3
networks:
streamsets:
name: streamsets
common:
external: true
name: common
启动命令：
docker stack deploy --compose-file docker-compose-dev.yml streamsets
体积：
stephane@stephane-pc:~/dev/docker/projects/streamsets$ tree volumes/stage-libs
volumes/stage-libs
├── streamsets-datacollector-dev-lib
│ └── lib
│ ├── automaton-1.11-8.jar
│ ├── commons-lang3-3.5.jar
│ ├── generex-1.0.2.jar
│ ├── jackson-annotations-2.8.0.jar
│ ├── jackson-core-2.8.9.jar
│ ├── jackson-databind-2.8.9.jar
│ ├── javafaker-0.13.jar
│ ├── streamsets-datacollector-dev-lib-3.18.1.jar
│ └── streamsets-datacollector-stagesupport-3.18.1.jar
└── streamsets-datacollector-mysql-binlog-lib
└── lib
├── guava-19.0.jar
├── HikariCP-3.2.0.jar
├── jackson-annotations-2.8.0.jar
├── jackson-core-2.8.8.jar
├── jackson-databind-2.8.8.jar
├── log4j-1.2.17.jar
├── mysql-binlog-connector-java-0.13.0.jar
├── slf4j-log4j12-1.7.7.jar
├── streamsets-datacollector-mysql-binlog-lib-3.18.1.jar
└── streamsets-datacollector-stagesupport-3.18.1.jar
4 directories,19 files
stephane@stephane-pc:~/dev/docker/projects/streamsets$ tree volumes/stage-libs-extra/
volumes/stage-libs-extra/
0 directories,0 files
更新：我注意到当没有任何卷映射时，启动容器将显示以下库：
bash-4.4$ ls -l /opt/streamsets-datacollector-3.18.1/streamsets-libs
total 20
drwxrwxr-x 3 1000 root 4096 Aug 26 18:55 streamsets-datacollector-basic-lib
drwxrwxr-x 3 1000 root 4096 Aug 26 18:55 streamsets-datacollector-dataformats-lib
drwxrwxr-x 3 1000 root 4096 Aug 26 18:55 streamsets-datacollector-dev-lib
drwxrwxr-x 3 1000 root 4096 Aug 26 18:55 streamsets-datacollector-stats-lib
drwxrwxr-x 3 1000 root 4096 Aug 26 18:55 streamsets-datacollector-windows-lib
但是具有卷映射时：
- "~/dev/docker/projects/streamsets/volumes/stage-libs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
- "~/dev/docker/projects/streamsets/volumes/stage-libs-extra:/opt/stage-libs-extra"
主机目录中没有这些库。
更新：无法解决这些权限问题，我求助于创建Docker卷而不是使用主机卷。
docker volume create --name sdc-data;
docker volume create --name sdc-stagelibs;
在docker-compose-dev.yml文件中，我现在有：
volumes:
- "sdc-data:/data"
- "sdc-stagelibs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
现在可以安装其他舞台库了。
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。