We have some application that use Docker in Azure Vm (centos7). We use systemd in order to manage their lifecycle in case of failure. A typical example of this is this service unit:
[Unit]
Description=Service app-subs
After=docker.service
Requires=docker.service
[Service]
Environment="JAVA_OPTS=-XX:+UseContainerSupport -XX:MaxRAMPercentage=70 -XshowSettings:vm"
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop app-subs
ExecStartPre=-/usr/bin/docker rm app-subs
ExecStartPre=/usr/bin/docker login registry.azure.io -u USER -p PASSWORD
ExecStart=/usr/bin/docker run \
--init \
--rm \
-p 8236:8080 \
registry.azure.io /app-subs:0.2.22
[Install]
WantedBy=multi-user.target
We work like this for all our docker app since 2 year and we never had problem with this type of service and we do not make any update. Today one of our vm trigger this error:
Apr 11 04:46:20 vm22 systemd: Found dependency on docker.service/start
Apr 11 04:46:20 vm22 systemd: Breaking ordering cycle by deleting job app-subs.service/start
Apr 11 04:46:20 vm22 systemd: Job app-subs.service/start deleted to break ordering cycle starting with docker.service/start
This make the service inactive that launch our application. We don’t make any change since a long time and do not update this service recently. So we tried to figure out why we have an “ordering cycle” in our systemd services with these commands:
sudo systemctl show -p Requires,Wants,Requisite,BindsTo,PartOf,Before,After app-subs.service
Result:
Requires=docker.service system.slice basic.target
Requisite=
Wants=
BindsTo=
PartOf=
Before=shutdown.target multi-user.target
After=basic.target docker.service systemd-journald.socket system.slice
Docker service:
sudo systemctl show -p Requires,Wants,Requisite,BindsTo,PartOf,Before,After docker.service
Result:
Requires=basic.target docker.socket containerd.service system.slice
Requisite=
Wants=network-online.target
BindsTo=
PartOf=
Before= shutdown.target app-subs.service
After=basic.target docker.socket network-online.target multi-user.target systemd-journald.socket system.slice firewalld.service containerd.service
If i understand well Docker have to be launch before app-subs.service but after multi-user.target and our app have to be launch before multi-user.target but after docker.service. But multi-user.target seem to be a step trigger by this in our services.
[Install]
WantedBy=multi-user.target
that allow our service to be launch at startup.
We have the same configuration in multiple machine and app. Only one machine trigger this error and i cannot find out what happened for this. The machine reboot in the night without explanation.
Does anyone have any idea what might have happened or how to make sure it doesn’t happen again?